[2025-11-04T17:00:01.515354] [INFO ] [MAIN ] Using last completed trading day: 2025-11-04 [2025-11-04T17:00:01.695203] [DEBUG] [R-STATUS ] Registered 'tdm' as a dependent service. [2025-11-04T17:00:01.695368] [INFO ] [STATE ] Waiting for all dependent services to be clear... [2025-11-04T17:00:01.811879] [WARN ] [tdm-status] Service 'tdm' is not running. [2025-11-04T17:00:01.812187] [WARN ] [STATE ] State set to: RUNNING [2025-11-04T17:00:01.812444] [INFO ] [MAIN ] Refreshing symbols list before update... [2025-11-04T17:00:01.812486] [INFO ] [MAIN ] Starting TDM symbol refresh process. [2025-11-04T17:00:01.812565] [INFO ] [MAIN ] Fetching latest NYSE symbols list. [2025-11-04T17:00:01.812600] [DEBUG] [SYMBOL ] Starting fetch of NYSE Symbols. [2025-11-04T17:00:03.242574] [DEBUG] [SYMBOL ] Found 2419 valid NYSE symbols. [2025-11-04T17:00:03.242771] [INFO ] [SYMBOL ] Completed fetch of NYSE Symbols. [2025-11-04T17:00:03.242976] [INFO ] [MAIN ] Filtered out 54 blacklisted symbols. [2025-11-04T17:00:03.243442] [INFO ] [LOCAL ] Symbols cache updated with 2367 symbols. [2025-11-04T17:00:03.243499] [INFO ] [MAIN ] Successfully refreshed symbols list. [2025-11-04T17:00:03.243531] [INFO ] [MAIN ] Finished TDM symbol refresh process. [2025-11-04T17:00:03.243584] [INFO ] [MAIN ] Pruning stale symbols... [2025-11-04T17:00:03.243615] [INFO ] [MAIN ] Starting TDM stale symbol pruning process. [2025-11-04T17:00:03.243670] [INFO ] [LOCAL ] Reading cached symbols list. [2025-11-04T17:00:03.244037] [DEBUG] [LOCAL ] Found 2367 symbols in cached list. [2025-11-04T17:00:03.244072] [INFO ] [LOCAL ] Completed reading cached symbols list. [2025-11-04T17:00:03.244222] [INFO ] [MAIN ] Including 54 blacklisted symbols for pruning. [2025-11-04T17:00:03.244253] [INFO ] [MAIN ] Pruning data for symbols not in current list of 2367 valid symbols. [2025-11-04T17:00:03.244282] [INFO ] [LOCAL ] Starting pruning of local symbol history data for inactive/invalid symbols. [2025-11-04T17:00:03.244312] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/primary_storage' [2025-11-04T17:00:03.269753] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/derivative_storage' [2025-11-04T17:00:03.294126] [INFO ] [LOCAL ] Deleted 0 obsolete data files. [2025-11-04T17:00:03.294254] [INFO ] [LOCAL ] Completed pruning of unlisted local symbol history data. [2025-11-04T17:00:03.294290] [INFO ] [MAIN ] Successfully pruned stale symbol data. [2025-11-04T17:00:03.294321] [INFO ] [MAIN ] Finished TDM stale symbol pruning process. [2025-11-04T17:00:03.294377] [INFO ] [LOCAL ] Reading cached symbols list. [2025-11-04T17:00:03.294748] [DEBUG] [LOCAL ] Found 2367 symbols in cached list. [2025-11-04T17:00:03.294801] [INFO ] [LOCAL ] Completed reading cached symbols list. [2025-11-04T17:00:03.294833] [INFO ] [MAIN ] Starting incremental update for 2367 symbols. [2025-11-04T17:00:03.294861] [INFO ] [MAIN ] Running in parallel mode [2025-11-04T17:00:03.295024] [INFO ] [MAIN ] Starting parallel primary update with 4 worker threads for 2367 symbols [2025-11-04T17:00:03.850565] [INFO ] [MAIN ] AA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:03.955467] [INFO ] [MAIN ] A: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:03.999291] [INFO ] [MAIN ] AAMI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:04.029912] [INFO ] [MAIN ] AAM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:04.031110] [DEBUG] [LOCAL ] AAM: No existing data file found in primary storage. [2025-11-04T17:00:04.031167] [WARN ] [MAIN ] AAM: No existing data found, performing full refresh instead. [2025-11-04T17:00:04.031302] [INFO ] [MAIN ] AAM: Wiping existing primary data before full refresh. [2025-11-04T17:00:04.031413] [INFO ] [LOCAL ] AAM: Starting deletion of primary data. [2025-11-04T17:00:04.031493] [DEBUG] [LOCAL ] AAM: No primary data file found to delete. [2025-11-04T17:00:04.031551] [INFO ] [MAIN ] AAM: Starting full primary refresh. [2025-11-04T17:00:04.299561] [DEBUG] [LOCAL ] AAMI: Loaded 23763 existing records from primary storage. [2025-11-04T17:00:04.318432] [INFO ] [MAIN ] AAMI: Loaded 23763 existing records. [2025-11-04T17:00:04.515429] [DEBUG] [LOCAL ] A: Loaded 50436 existing records from primary storage. [2025-11-04T17:00:04.552049] [DEBUG] [LOCAL ] AA: Loaded 53622 existing records from primary storage. [2025-11-04T17:00:04.552380] [INFO ] [MAIN ] A: Loaded 50436 existing records. [2025-11-04T17:00:04.557858] [INFO ] [MAIN ] AA: Loaded 53622 existing records. [2025-11-04T17:00:04.699162] [INFO ] [REMOTE ] AAM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:04.786039] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-01. [2025-11-04T17:00:04.926696] [WARN ] [MAIN ] AAMI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:04.976286] [INFO ] [MAIN ] AAMI: Wiping existing primary data before full refresh. [2025-11-04T17:00:05.043670] [WARN ] [MAIN ] A: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:05.072633] [WARN ] [MAIN ] AA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:05.073210] [INFO ] [MAIN ] A: Wiping existing primary data before full refresh. [2025-11-04T17:00:05.072774] [INFO ] [LOCAL ] AAMI: Starting deletion of primary data. [2025-11-04T17:00:05.073369] [INFO ] [MAIN ] AA: Wiping existing primary data before full refresh. [2025-11-04T17:00:05.073519] [INFO ] [LOCAL ] A: Starting deletion of primary data. [2025-11-04T17:00:05.073723] [INFO ] [LOCAL ] AA: Starting deletion of primary data. [2025-11-04T17:00:05.074075] [DEBUG] [LOCAL ] A: Deleted primary data file. [2025-11-04T17:00:05.074014] [DEBUG] [LOCAL ] AA: Deleted primary data file. [2025-11-04T17:00:05.073979] [DEBUG] [LOCAL ] AAMI: Deleted primary data file. [2025-11-04T17:00:05.074374] [INFO ] [MAIN ] A: Starting full primary refresh. [2025-11-04T17:00:05.074596] [INFO ] [MAIN ] AA: Starting full primary refresh. [2025-11-04T17:00:05.074502] [INFO ] [MAIN ] AAMI: Starting full primary refresh. [2025-11-04T17:00:05.663125] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T17:00:05.766912] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-02. [2025-11-04T17:00:06.069458] [INFO ] [REMOTE ] AA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:06.113685] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-01. [2025-11-04T17:00:06.202906] [INFO ] [REMOTE ] A: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:06.220345] [INFO ] [REMOTE ] A: Fetching primary data for 2025-01. [2025-11-04T17:00:06.222351] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-11-04T17:00:06.225230] [INFO ] [REMOTE ] AAMI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:06.226337] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-03. [2025-11-04T17:00:06.227311] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-01. [2025-11-04T17:00:06.456367] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-03 (0.23 seconds). [2025-11-04T17:00:06.457375] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-04. [2025-11-04T17:00:06.678517] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-04 (0.22 seconds). [2025-11-04T17:00:06.679508] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-05. [2025-11-04T17:00:07.035918] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-11-04T17:00:07.037154] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-06. [2025-11-04T17:00:07.073852] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-11-04T17:00:07.077605] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-02. [2025-11-04T17:00:07.236707] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-06 (0.20 seconds). [2025-11-04T17:00:07.237555] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-07. [2025-11-04T17:00:07.406384] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T17:00:07.416065] [INFO ] [REMOTE ] A: Fetching primary data for 2025-02. [2025-11-04T17:00:07.471069] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-07 (0.23 seconds). [2025-11-04T17:00:07.472044] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-08. [2025-11-04T17:00:07.587508] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T17:00:07.591437] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-03. [2025-11-04T17:00:07.694302] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-08 (0.22 seconds). [2025-11-04T17:00:07.695324] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-09. [2025-11-04T17:00:07.929356] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-09 (0.23 seconds). [2025-11-04T17:00:07.930280] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-10. [2025-11-04T17:00:08.271250] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-11-04T17:00:08.277247] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-04. [2025-11-04T17:00:08.344247] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-01 (2.23 seconds). [2025-11-04T17:00:08.354687] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-02. [2025-11-04T17:00:08.711847] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T17:00:08.721517] [INFO ] [REMOTE ] A: Fetching primary data for 2025-03. [2025-11-04T17:00:09.237766] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T17:00:09.238841] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-11. [2025-11-04T17:00:09.312334] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-11-04T17:00:09.318852] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-05. [2025-11-04T17:00:09.614862] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T17:00:09.615498] [INFO ] [REMOTE ] AAM: Full fetch complete (915 records). [2025-11-04T17:00:09.615567] [DEBUG] [VALIDATOR ] AAM: Validating primary data. [2025-11-04T17:00:09.834972] [WARN ] [VALIDATOR ] AAM: Validation failed - insufficient historical data. Have 185 unique days, need 200 days [2025-11-04T17:00:09.835244] [FATAL] [MAIN ] AAM: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:00:09.835402] [WARN ] [MAIN ] AAM: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:00:09.835558] [INFO ] [LOCAL ] AAM: Starting deletion of primary data. [2025-11-04T17:00:09.835688] [DEBUG] [LOCAL ] AAM: No primary data file found to delete. [2025-11-04T17:00:09.835747] [INFO ] [LOCAL ] AAM: Starting deletion of derivative data. [2025-11-04T17:00:09.835791] [DEBUG] [LOCAL ] AAM: No derivative data file found to delete. [2025-11-04T17:00:09.835857] [INFO ] [MAIN ] Update Progress: 1/2367 symbols completed. AAM took 6.5s. Estimated remaining: 4h 17m 56s. Rate: 9.2 symbols/min [2025-11-04T17:00:10.045402] [INFO ] [MAIN ] AAP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:10.046058] [DEBUG] [LOCAL ] AAP: No existing data file found in primary storage. [2025-11-04T17:00:10.046235] [WARN ] [MAIN ] AAP: No existing data found, performing full refresh instead. [2025-11-04T17:00:10.046412] [INFO ] [MAIN ] AAP: Wiping existing primary data before full refresh. [2025-11-04T17:00:10.046540] [INFO ] [LOCAL ] AAP: Starting deletion of primary data. [2025-11-04T17:00:10.046674] [DEBUG] [LOCAL ] AAP: No primary data file found to delete. [2025-11-04T17:00:10.046821] [INFO ] [MAIN ] AAP: Starting full primary refresh. [2025-11-04T17:00:10.271094] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-11-04T17:00:10.325443] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-03. [2025-11-04T17:00:10.475848] [INFO ] [REMOTE ] AAP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:10.476549] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-01. [2025-11-04T17:00:10.524988] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T17:00:10.532846] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-11-04T17:00:10.544436] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-06. [2025-11-04T17:00:10.546453] [INFO ] [REMOTE ] A: Fetching primary data for 2025-04. [2025-11-04T17:00:11.407908] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T17:00:11.412926] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-07. [2025-11-04T17:00:12.234910] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-11-04T17:00:12.247448] [INFO ] [REMOTE ] A: Fetching primary data for 2025-05. [2025-11-04T17:00:12.265947] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T17:00:12.272850] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-08. [2025-11-04T17:00:12.310601] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-11-04T17:00:12.323759] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-04. [2025-11-04T17:00:12.370441] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-11-04T17:00:12.380601] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-02. [2025-11-04T17:00:13.179875] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-11-04T17:00:13.185126] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-09. [2025-11-04T17:00:13.591474] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T17:00:13.600987] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-03. [2025-11-04T17:00:13.897535] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-11-04T17:00:13.909078] [INFO ] [REMOTE ] A: Fetching primary data for 2025-06. [2025-11-04T17:00:14.077038] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-11-04T17:00:14.083454] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-10. [2025-11-04T17:00:14.120866] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-11-04T17:00:14.135176] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-05. [2025-11-04T17:00:15.055381] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T17:00:15.061582] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-11. [2025-11-04T17:00:15.115304] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-11-04T17:00:15.127008] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-04. [2025-11-04T17:00:15.225244] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T17:00:15.235278] [INFO ] [REMOTE ] A: Fetching primary data for 2025-07. [2025-11-04T17:00:15.825490] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-11-04T17:00:15.837223] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-06. [2025-11-04T17:00:16.196695] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T17:00:16.198286] [INFO ] [REMOTE ] AAMI: Full fetch complete (33323 records). [2025-11-04T17:00:16.198401] [DEBUG] [VALIDATOR ] AAMI: Validating primary data. [2025-11-04T17:00:16.199195] [FATAL] [VALIDATOR ] AAMI: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:00:16.199310] [FATAL] [MAIN ] AAMI: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:00:16.199395] [WARN ] [MAIN ] AAMI: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:00:16.199504] [INFO ] [LOCAL ] AAMI: Starting deletion of primary data. [2025-11-04T17:00:16.199614] [DEBUG] [LOCAL ] AAMI: No primary data file found to delete. [2025-11-04T17:00:16.199722] [INFO ] [LOCAL ] AAMI: Starting deletion of derivative data. [2025-11-04T17:00:16.199961] [DEBUG] [LOCAL ] AAMI: Deleted derivative data file. [2025-11-04T17:00:16.200693] [INFO ] [MAIN ] Update Progress: 2/2367 symbols completed. AAMI took 12.8s. Estimated remaining: 4h 14m 21s. Rate: 9.3 symbols/min [2025-11-04T17:00:16.374099] [INFO ] [MAIN ] AAT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:16.426446] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-11-04T17:00:16.439012] [INFO ] [REMOTE ] A: Fetching primary data for 2025-08. [2025-11-04T17:00:16.449552] [DEBUG] [LOCAL ] AAT: Loaded 29937 existing records from primary storage. [2025-11-04T17:00:16.450098] [INFO ] [MAIN ] AAT: Loaded 29937 existing records. [2025-11-04T17:00:16.659770] [WARN ] [MAIN ] AAT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:16.660081] [INFO ] [MAIN ] AAT: Wiping existing primary data before full refresh. [2025-11-04T17:00:16.660245] [INFO ] [LOCAL ] AAT: Starting deletion of primary data. [2025-11-04T17:00:16.660510] [DEBUG] [LOCAL ] AAT: Deleted primary data file. [2025-11-04T17:00:16.660619] [INFO ] [MAIN ] AAT: Starting full primary refresh. [2025-11-04T17:00:17.066007] [INFO ] [REMOTE ] AAT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:17.067147] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-01. [2025-11-04T17:00:17.069754] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-11-04T17:00:17.082524] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-05. [2025-11-04T17:00:17.201776] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T17:00:17.213159] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-07. [2025-11-04T17:00:17.893542] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T17:00:17.905957] [INFO ] [REMOTE ] A: Fetching primary data for 2025-09. [2025-11-04T17:00:17.964927] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T17:00:17.970454] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-02. [2025-11-04T17:00:18.564593] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T17:00:18.595469] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-08. [2025-11-04T17:00:18.610727] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-11-04T17:00:18.622344] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-06. [2025-11-04T17:00:18.702461] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-11-04T17:00:18.709463] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-03. [2025-11-04T17:00:19.635101] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T17:00:19.642286] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-04. [2025-11-04T17:00:19.735031] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-11-04T17:00:19.745928] [INFO ] [REMOTE ] A: Fetching primary data for 2025-10. [2025-11-04T17:00:19.791813] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T17:00:19.803589] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-07. [2025-11-04T17:00:20.519020] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-11-04T17:00:20.532013] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-09. [2025-11-04T17:00:20.792963] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T17:00:20.800324] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-05. [2025-11-04T17:00:21.245624] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:00:21.257082] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-08. [2025-11-04T17:00:21.485025] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-11-04T17:00:21.498246] [INFO ] [REMOTE ] A: Fetching primary data for 2025-11. [2025-11-04T17:00:21.845836] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T17:00:21.852824] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-06. [2025-11-04T17:00:21.885524] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T17:00:21.897847] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-10. [2025-11-04T17:00:22.634522] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T17:00:22.639199] [INFO ] [REMOTE ] A: Full fetch complete (73531 records). [2025-11-04T17:00:22.639364] [DEBUG] [VALIDATOR ] A: Validating primary data. [2025-11-04T17:00:23.017864] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T17:00:23.085363] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-07. [2025-11-04T17:00:23.127394] [DEBUG] [VALIDATOR ] A: All expected trading days are present. [2025-11-04T17:00:23.127638] [DEBUG] [VALIDATOR ] A: Primary data validation passed. [2025-11-04T17:00:23.128967] [INFO ] [MAIN ] A: Fetched primary data is valid. [2025-11-04T17:00:23.129084] [DEBUG] [LOCAL ] A: Saving primary data. [2025-11-04T17:00:23.158901] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-11-04T17:00:23.159078] [DEBUG] [LOCAL ] A: Trimmed 73531 records to 50451 (keeping 200 days of data) [2025-11-04T17:00:23.171698] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-09. [2025-11-04T17:00:23.489938] [DEBUG] [LOCAL ] A: Primary data saved (50451 records). [2025-11-04T17:00:23.490100] [INFO ] [MAIN ] A: Full history saved. [2025-11-04T17:00:23.490817] [INFO ] [MAIN ] Update Progress: 3/2367 symbols completed. A took 20.2s. Estimated remaining: 4h 25m 14s. Rate: 8.9 symbols/min [2025-11-04T17:00:23.690547] [INFO ] [MAIN ] AAUC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:23.690756] [DEBUG] [LOCAL ] AAUC: No existing data file found in primary storage. [2025-11-04T17:00:23.690910] [WARN ] [MAIN ] AAUC: No existing data found, performing full refresh instead. [2025-11-04T17:00:23.691040] [INFO ] [MAIN ] AAUC: Wiping existing primary data before full refresh. [2025-11-04T17:00:23.691115] [INFO ] [LOCAL ] AAUC: Starting deletion of primary data. [2025-11-04T17:00:23.691248] [DEBUG] [LOCAL ] AAUC: No primary data file found to delete. [2025-11-04T17:00:23.691311] [INFO ] [MAIN ] AAUC: Starting full primary refresh. [2025-11-04T17:00:24.068909] [INFO ] [REMOTE ] AAUC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:24.083338] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-01. [2025-11-04T17:00:24.085793] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-10 (2.19 seconds). [2025-11-04T17:00:24.098338] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-11. [2025-11-04T17:00:24.538594] [DEBUG] [REMOTE ] AAUC: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:00:24.539906] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T17:00:24.540235] [WARN ] [REMOTE ] AAUC: No data returned for month 2025-01 [2025-11-04T17:00:24.547303] [FATAL] [REMOTE ] AAUC: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:00:24.547494] [FATAL] [MAIN ] AAUC: Failed to fetch primary data. [2025-11-04T17:00:24.547835] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-08. [2025-11-04T17:00:24.548028] [WARN ] [MAIN ] AAUC: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:00:24.548917] [INFO ] [LOCAL ] AAUC: Starting deletion of primary data. [2025-11-04T17:00:24.548987] [DEBUG] [LOCAL ] AAUC: No primary data file found to delete. [2025-11-04T17:00:24.549025] [INFO ] [LOCAL ] AAUC: Starting deletion of derivative data. [2025-11-04T17:00:24.549063] [DEBUG] [LOCAL ] AAUC: No derivative data file found to delete. [2025-11-04T17:00:24.549112] [INFO ] [MAIN ] Update Progress: 4/2367 symbols completed. AAUC took 1.1s. Estimated remaining: 3h 29m 16s. Rate: 11.3 symbols/min [2025-11-04T17:00:24.734051] [INFO ] [MAIN ] AB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:24.764237] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T17:00:24.780963] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-10. [2025-11-04T17:00:24.792007] [DEBUG] [LOCAL ] AB: Loaded 23774 existing records from primary storage. [2025-11-04T17:00:24.792313] [INFO ] [MAIN ] AB: Loaded 23774 existing records. [2025-11-04T17:00:24.982533] [WARN ] [MAIN ] AB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:24.982772] [INFO ] [MAIN ] AB: Wiping existing primary data before full refresh. [2025-11-04T17:00:24.982835] [INFO ] [LOCAL ] AB: Starting deletion of primary data. [2025-11-04T17:00:24.983099] [DEBUG] [LOCAL ] AB: Deleted primary data file. [2025-11-04T17:00:24.983157] [INFO ] [MAIN ] AB: Starting full primary refresh. [2025-11-04T17:00:25.336256] [INFO ] [REMOTE ] AB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:25.336555] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-01. [2025-11-04T17:00:25.657373] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-11 (1.56 seconds). [2025-11-04T17:00:25.657219] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T17:00:25.665735] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-09. [2025-11-04T17:00:25.666860] [INFO ] [REMOTE ] AA: Full fetch complete (77742 records). [2025-11-04T17:00:25.666968] [DEBUG] [VALIDATOR ] AA: Validating primary data. [2025-11-04T17:00:26.061930] [DEBUG] [VALIDATOR ] AA: All expected trading days are present. [2025-11-04T17:00:26.062036] [DEBUG] [VALIDATOR ] AA: Primary data validation passed. [2025-11-04T17:00:26.062781] [INFO ] [MAIN ] AA: Fetched primary data is valid. [2025-11-04T17:00:26.062891] [DEBUG] [LOCAL ] AA: Saving primary data. [2025-11-04T17:00:26.064517] [DEBUG] [LOCAL ] AA: Trimmed 77742 records to 53622 (keeping 200 days of data) [2025-11-04T17:00:26.307084] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-11-04T17:00:26.321142] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-11. [2025-11-04T17:00:26.432118] [DEBUG] [LOCAL ] AA: Primary data saved (53622 records). [2025-11-04T17:00:26.432251] [INFO ] [MAIN ] AA: Full history saved. [2025-11-04T17:00:26.432614] [INFO ] [MAIN ] Update Progress: 5/2367 symbols completed. AA took 23.1s. Estimated remaining: 3h 2m 10s. Rate: 13.0 symbols/min [2025-11-04T17:00:26.622396] [INFO ] [MAIN ] ABBV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:26.636087] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T17:00:26.643727] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-02. [2025-11-04T17:00:26.692983] [DEBUG] [LOCAL ] ABBV: Loaded 53674 existing records from primary storage. [2025-11-04T17:00:26.693282] [INFO ] [MAIN ] ABBV: Loaded 53674 existing records. [2025-11-04T17:00:26.873593] [WARN ] [MAIN ] ABBV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:26.892551] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T17:00:26.892007] [INFO ] [MAIN ] ABBV: Wiping existing primary data before full refresh. [2025-11-04T17:00:26.892728] [INFO ] [LOCAL ] ABBV: Starting deletion of primary data. [2025-11-04T17:00:26.898889] [DEBUG] [LOCAL ] ABBV: Deleted primary data file. [2025-11-04T17:00:26.898946] [INFO ] [MAIN ] ABBV: Starting full primary refresh. [2025-11-04T17:00:26.943045] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-10. [2025-11-04T17:00:27.254334] [INFO ] [REMOTE ] ABBV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:27.254676] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-01. [2025-11-04T17:00:27.557045] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T17:00:27.559670] [INFO ] [REMOTE ] AAP: Full fetch complete (75061 records). [2025-11-04T17:00:27.559774] [DEBUG] [VALIDATOR ] AAP: Validating primary data. [2025-11-04T17:00:28.003013] [DEBUG] [VALIDATOR ] AAP: All expected trading days are present. [2025-11-04T17:00:28.003446] [DEBUG] [VALIDATOR ] AAP: Primary data validation passed. [2025-11-04T17:00:28.004773] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T17:00:28.008506] [INFO ] [MAIN ] AAP: Fetched primary data is valid. [2025-11-04T17:00:28.008689] [DEBUG] [LOCAL ] AAP: Saving primary data. [2025-11-04T17:00:28.015452] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-03. [2025-11-04T17:00:28.035183] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-11-04T17:00:28.041874] [DEBUG] [LOCAL ] AAP: Trimmed 75061 records to 51586 (keeping 200 days of data) [2025-11-04T17:00:28.104055] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-11. [2025-11-04T17:00:28.347726] [DEBUG] [LOCAL ] AAP: Primary data saved (51586 records). [2025-11-04T17:00:28.347860] [INFO ] [MAIN ] AAP: Full history saved. [2025-11-04T17:00:28.347932] [INFO ] [MAIN ] Update Progress: 6/2367 symbols completed. AAP took 18.5s. Estimated remaining: 2h 44m 18s. Rate: 14.4 symbols/min [2025-11-04T17:00:28.521743] [INFO ] [MAIN ] ABCB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:28.548712] [DEBUG] [LOCAL ] ABCB: Loaded 30043 existing records from primary storage. [2025-11-04T17:00:28.548970] [INFO ] [MAIN ] ABCB: Loaded 30043 existing records. [2025-11-04T17:00:28.736776] [WARN ] [MAIN ] ABCB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:28.737030] [INFO ] [MAIN ] ABCB: Wiping existing primary data before full refresh. [2025-11-04T17:00:28.737123] [INFO ] [LOCAL ] ABCB: Starting deletion of primary data. [2025-11-04T17:00:28.737397] [DEBUG] [LOCAL ] ABCB: Deleted primary data file. [2025-11-04T17:00:28.737469] [INFO ] [MAIN ] ABCB: Starting full primary refresh. [2025-11-04T17:00:29.118202] [INFO ] [REMOTE ] ABCB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:29.118935] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-01. [2025-11-04T17:00:29.147004] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-11-04T17:00:29.157518] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-02. [2025-11-04T17:00:29.196669] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T17:00:29.198202] [INFO ] [REMOTE ] AAT: Full fetch complete (43362 records). [2025-11-04T17:00:29.198276] [DEBUG] [VALIDATOR ] AAT: Validating primary data. [2025-11-04T17:00:29.576046] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T17:00:29.629264] [DEBUG] [VALIDATOR ] AAT: All expected trading days are present. [2025-11-04T17:00:29.629773] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-04. [2025-11-04T17:00:29.629389] [DEBUG] [VALIDATOR ] AAT: Primary data validation passed. [2025-11-04T17:00:29.631120] [INFO ] [MAIN ] AAT: Fetched primary data is valid. [2025-11-04T17:00:29.631192] [DEBUG] [LOCAL ] AAT: Saving primary data. [2025-11-04T17:00:29.632283] [DEBUG] [LOCAL ] AAT: Trimmed 43362 records to 30038 (keeping 200 days of data) [2025-11-04T17:00:29.806214] [DEBUG] [LOCAL ] AAT: Primary data saved (30038 records). [2025-11-04T17:00:29.806339] [INFO ] [MAIN ] AAT: Full history saved. [2025-11-04T17:00:29.806415] [INFO ] [MAIN ] Update Progress: 7/2367 symbols completed. AAT took 13.6s. Estimated remaining: 2h 28m 58s. Rate: 15.8 symbols/min [2025-11-04T17:00:29.973755] [INFO ] [MAIN ] ABEV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:29.973900] [DEBUG] [LOCAL ] ABEV: No existing data file found in primary storage. [2025-11-04T17:00:29.973936] [WARN ] [MAIN ] ABEV: No existing data found, performing full refresh instead. [2025-11-04T17:00:29.974043] [INFO ] [MAIN ] ABEV: Wiping existing primary data before full refresh. [2025-11-04T17:00:29.974084] [INFO ] [LOCAL ] ABEV: Starting deletion of primary data. [2025-11-04T17:00:29.974124] [DEBUG] [LOCAL ] ABEV: No primary data file found to delete. [2025-11-04T17:00:29.974156] [INFO ] [MAIN ] ABEV: Starting full primary refresh. [2025-11-04T17:00:30.332053] [INFO ] [REMOTE ] ABEV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:30.332538] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-01. [2025-11-04T17:00:30.353489] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T17:00:30.362139] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-02. [2025-11-04T17:00:30.522397] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T17:00:30.535061] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-03. [2025-11-04T17:00:30.778128] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T17:00:30.786254] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-05. [2025-11-04T17:00:31.281687] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-11-04T17:00:31.287873] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-03. [2025-11-04T17:00:31.851597] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T17:00:31.857849] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-06. [2025-11-04T17:00:31.918442] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-11-04T17:00:31.929994] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-02. [2025-11-04T17:00:32.270596] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T17:00:32.281955] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-04. [2025-11-04T17:00:32.319299] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-11-04T17:00:32.331474] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-04. [2025-11-04T17:00:32.787468] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T17:00:32.792109] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-07. [2025-11-04T17:00:33.424938] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T17:00:33.433488] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-05. [2025-11-04T17:00:33.464747] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-11-04T17:00:33.476497] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-03. [2025-11-04T17:00:33.562081] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T17:00:33.568446] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-08. [2025-11-04T17:00:33.957297] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-11-04T17:00:33.970991] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-05. [2025-11-04T17:00:34.395132] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-11-04T17:00:34.402794] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-09. [2025-11-04T17:00:34.473706] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-11-04T17:00:34.482137] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-06. [2025-11-04T17:00:35.221618] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-11-04T17:00:35.233838] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-04. [2025-11-04T17:00:35.318680] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T17:00:35.337390] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-10. [2025-11-04T17:00:35.339176] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T17:00:35.344970] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-07. [2025-11-04T17:00:36.009235] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-11-04T17:00:36.020624] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-06. [2025-11-04T17:00:36.218790] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-11-04T17:00:36.224969] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-11. [2025-11-04T17:00:36.289156] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-11-04T17:00:36.297148] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-08. [2025-11-04T17:00:36.989242] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-11 (0.76 seconds). [2025-11-04T17:00:36.990381] [INFO ] [REMOTE ] AB: Full fetch complete (43338 records). [2025-11-04T17:00:36.990454] [DEBUG] [VALIDATOR ] AB: Validating primary data. [2025-11-04T17:00:36.991099] [FATAL] [VALIDATOR ] AB: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:00:36.991180] [FATAL] [MAIN ] AB: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:00:36.991223] [WARN ] [MAIN ] AB: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:00:36.991261] [INFO ] [LOCAL ] AB: Starting deletion of primary data. [2025-11-04T17:00:36.991314] [DEBUG] [LOCAL ] AB: No primary data file found to delete. [2025-11-04T17:00:36.991350] [INFO ] [LOCAL ] AB: Starting deletion of derivative data. [2025-11-04T17:00:36.991464] [DEBUG] [LOCAL ] AB: Deleted derivative data file. [2025-11-04T17:00:36.991530] [INFO ] [MAIN ] Update Progress: 8/2367 symbols completed. AB took 12.4s. Estimated remaining: 2h 45m 36s. Rate: 14.2 symbols/min [2025-11-04T17:00:37.196375] [INFO ] [MAIN ] ABG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:37.199377] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-11-04T17:00:37.211406] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-05. [2025-11-04T17:00:37.227210] [DEBUG] [LOCAL ] ABG: Loaded 17221 existing records from primary storage. [2025-11-04T17:00:37.228267] [INFO ] [MAIN ] ABG: Loaded 17221 existing records. [2025-11-04T17:00:37.422564] [WARN ] [MAIN ] ABG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:37.442670] [INFO ] [MAIN ] ABG: Wiping existing primary data before full refresh. [2025-11-04T17:00:37.443619] [INFO ] [LOCAL ] ABG: Starting deletion of primary data. [2025-11-04T17:00:37.444595] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-11-04T17:00:37.451834] [DEBUG] [LOCAL ] ABG: Deleted primary data file. [2025-11-04T17:00:37.452454] [INFO ] [MAIN ] ABG: Starting full primary refresh. [2025-11-04T17:00:37.452914] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-09. [2025-11-04T17:00:37.857780] [INFO ] [REMOTE ] ABG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:37.858973] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-01. [2025-11-04T17:00:37.861212] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-11-04T17:00:37.872913] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-07. [2025-11-04T17:00:38.414176] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T17:00:38.421308] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-10. [2025-11-04T17:00:38.642586] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T17:00:38.653737] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-06. [2025-11-04T17:00:38.738234] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T17:00:38.742555] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-02. [2025-11-04T17:00:39.300721] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-11-04T17:00:39.308489] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-11. [2025-11-04T17:00:39.509570] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-11-04T17:00:39.509955] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-11-04T17:00:39.525806] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-03. [2025-11-04T17:00:39.527326] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-08. [2025-11-04T17:00:40.112440] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T17:00:40.128108] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-07. [2025-11-04T17:00:40.306214] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T17:00:40.307870] [INFO ] [REMOTE ] ABCB: Full fetch complete (44988 records). [2025-11-04T17:00:40.307962] [DEBUG] [VALIDATOR ] ABCB: Validating primary data. [2025-11-04T17:00:40.685270] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-11-04T17:00:40.720408] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-04. [2025-11-04T17:00:40.720561] [DEBUG] [VALIDATOR ] ABCB: All expected trading days are present. [2025-11-04T17:00:40.720846] [DEBUG] [VALIDATOR ] ABCB: Primary data validation passed. [2025-11-04T17:00:40.721923] [INFO ] [MAIN ] ABCB: Fetched primary data is valid. [2025-11-04T17:00:40.721988] [DEBUG] [LOCAL ] ABCB: Saving primary data. [2025-11-04T17:00:40.723000] [DEBUG] [LOCAL ] ABCB: Trimmed 44988 records to 29966 (keeping 200 days of data) [2025-11-04T17:00:40.902868] [DEBUG] [LOCAL ] ABCB: Primary data saved (29966 records). [2025-11-04T17:00:40.903093] [INFO ] [MAIN ] ABCB: Full history saved. [2025-11-04T17:00:40.905221] [INFO ] [MAIN ] Update Progress: 9/2367 symbols completed. ABCB took 12.6s. Estimated remaining: 2h 44m 14s. Rate: 14.4 symbols/min [2025-11-04T17:00:41.085542] [INFO ] [MAIN ] ABM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:41.121071] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-11-04T17:00:41.133195] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-09. [2025-11-04T17:00:41.154072] [DEBUG] [LOCAL ] ABM: Loaded 31608 existing records from primary storage. [2025-11-04T17:00:41.154319] [INFO ] [MAIN ] ABM: Loaded 31608 existing records. [2025-11-04T17:00:41.329016] [WARN ] [MAIN ] ABM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:41.329235] [INFO ] [MAIN ] ABM: Wiping existing primary data before full refresh. [2025-11-04T17:00:41.329282] [INFO ] [LOCAL ] ABM: Starting deletion of primary data. [2025-11-04T17:00:41.329615] [DEBUG] [LOCAL ] ABM: Deleted primary data file. [2025-11-04T17:00:41.329710] [INFO ] [MAIN ] ABM: Starting full primary refresh. [2025-11-04T17:00:41.687578] [INFO ] [REMOTE ] ABM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:41.688005] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-01. [2025-11-04T17:00:41.707712] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-11-04T17:00:41.742324] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-05. [2025-11-04T17:00:41.748624] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-11-04T17:00:41.760622] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-08. [2025-11-04T17:00:42.472493] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T17:00:42.476944] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-06. [2025-11-04T17:00:42.503512] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-11-04T17:00:42.509293] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-02. [2025-11-04T17:00:42.734555] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-11-04T17:00:42.746020] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-10. [2025-11-04T17:00:43.302406] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T17:00:43.306588] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-07. [2025-11-04T17:00:43.335662] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-11-04T17:00:43.347393] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-09. [2025-11-04T17:00:43.421832] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T17:00:43.426845] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-03. [2025-11-04T17:00:43.970961] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-11-04T17:00:43.975743] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-08. [2025-11-04T17:00:44.393171] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-11-04T17:00:44.400630] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-04. [2025-11-04T17:00:44.452817] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-11-04T17:00:44.465609] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-11. [2025-11-04T17:00:44.813516] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-11-04T17:00:44.818796] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-09. [2025-11-04T17:00:44.975133] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-11-04T17:00:44.987476] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-10. [2025-11-04T17:00:45.501599] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T17:00:45.507316] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-10. [2025-11-04T17:00:45.613498] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T17:00:45.621208] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-05. [2025-11-04T17:00:46.114617] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-11 (1.65 seconds). [2025-11-04T17:00:46.116731] [INFO ] [REMOTE ] ABBV: Full fetch complete (77838 records). [2025-11-04T17:00:46.116872] [DEBUG] [VALIDATOR ] ABBV: Validating primary data. [2025-11-04T17:00:46.466829] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-11-04T17:00:46.512336] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-11. [2025-11-04T17:00:46.575117] [DEBUG] [VALIDATOR ] ABBV: All expected trading days are present. [2025-11-04T17:00:46.575664] [DEBUG] [VALIDATOR ] ABBV: Primary data validation passed. [2025-11-04T17:00:46.576756] [INFO ] [MAIN ] ABBV: Fetched primary data is valid. [2025-11-04T17:00:46.576872] [DEBUG] [LOCAL ] ABBV: Saving primary data. [2025-11-04T17:00:46.606582] [DEBUG] [LOCAL ] ABBV: Trimmed 77838 records to 53674 (keeping 200 days of data) [2025-11-04T17:00:46.608164] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T17:00:46.618119] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-06. [2025-11-04T17:00:46.874348] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-11-04T17:00:47.000718] [DEBUG] [LOCAL ] ABBV: Primary data saved (53674 records). [2025-11-04T17:00:47.000876] [INFO ] [MAIN ] ABBV: Full history saved. [2025-11-04T17:00:47.002943] [INFO ] [MAIN ] Update Progress: 10/2367 symbols completed. ABBV took 20.6s. Estimated remaining: 2h 51m 42s. Rate: 13.7 symbols/min [2025-11-04T17:00:47.175471] [INFO ] [MAIN ] ABR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:47.175665] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T17:00:47.177762] [INFO ] [REMOTE ] ABG: Full fetch complete (26849 records). [2025-11-04T17:00:47.177857] [DEBUG] [VALIDATOR ] ABG: Validating primary data. [2025-11-04T17:00:47.367342] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-11. [2025-11-04T17:00:47.557103] [DEBUG] [LOCAL ] ABR: Loaded 51778 existing records from primary storage. [2025-11-04T17:00:47.570324] [INFO ] [MAIN ] ABR: Loaded 51778 existing records. [2025-11-04T17:00:47.662178] [DEBUG] [VALIDATOR ] ABG: All expected trading days are present. [2025-11-04T17:00:47.682223] [DEBUG] [VALIDATOR ] ABG: Primary data validation passed. [2025-11-04T17:00:47.690793] [INFO ] [MAIN ] ABG: Fetched primary data is valid. [2025-11-04T17:00:47.690991] [DEBUG] [LOCAL ] ABG: Saving primary data. [2025-11-04T17:00:47.732638] [DEBUG] [LOCAL ] ABG: Trimmed 26849 records to 17240 (keeping 200 days of data) [2025-11-04T17:00:47.867020] [WARN ] [MAIN ] ABR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:47.918383] [INFO ] [MAIN ] ABR: Wiping existing primary data before full refresh. [2025-11-04T17:00:47.918530] [INFO ] [LOCAL ] ABR: Starting deletion of primary data. [2025-11-04T17:00:47.918307] [DEBUG] [LOCAL ] ABG: Primary data saved (17240 records). [2025-11-04T17:00:47.918763] [INFO ] [MAIN ] ABG: Full history saved. [2025-11-04T17:00:47.918888] [DEBUG] [LOCAL ] ABR: Deleted primary data file. [2025-11-04T17:00:47.919044] [INFO ] [MAIN ] ABR: Starting full primary refresh. [2025-11-04T17:00:47.919230] [INFO ] [MAIN ] Update Progress: 11/2367 symbols completed. ABG took 10.9s. Estimated remaining: 2h 39m 18s. Rate: 14.8 symbols/min [2025-11-04T17:00:48.301242] [INFO ] [MAIN ] ABT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:48.348140] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-11-04T17:00:48.398481] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-07. [2025-11-04T17:00:48.517056] [INFO ] [REMOTE ] ABR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:48.554280] [DEBUG] [LOCAL ] ABT: Loaded 53625 existing records from primary storage. [2025-11-04T17:00:48.554685] [INFO ] [MAIN ] ABT: Loaded 53625 existing records. [2025-11-04T17:00:48.733300] [WARN ] [MAIN ] ABT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:48.733532] [INFO ] [MAIN ] ABT: Wiping existing primary data before full refresh. [2025-11-04T17:00:48.733582] [INFO ] [LOCAL ] ABT: Starting deletion of primary data. [2025-11-04T17:00:48.733818] [DEBUG] [LOCAL ] ABT: Deleted primary data file. [2025-11-04T17:00:48.733875] [INFO ] [MAIN ] ABT: Starting full primary refresh. [2025-11-04T17:00:48.967593] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-11 (1.60 seconds). [2025-11-04T17:00:48.967198] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-01. [2025-11-04T17:00:49.005603] [INFO ] [REMOTE ] ABEV: Full fetch complete (77321 records). [2025-11-04T17:00:49.024352] [DEBUG] [VALIDATOR ] ABEV: Validating primary data. [2025-11-04T17:00:49.248288] [INFO ] [REMOTE ] ABT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:49.527589] [DEBUG] [VALIDATOR ] ABEV: All expected trading days are present. [2025-11-04T17:00:49.527716] [DEBUG] [VALIDATOR ] ABEV: Primary data validation passed. [2025-11-04T17:00:49.528523] [INFO ] [MAIN ] ABEV: Fetched primary data is valid. [2025-11-04T17:00:49.528638] [DEBUG] [LOCAL ] ABEV: Saving primary data. [2025-11-04T17:00:49.548709] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-11-04T17:00:49.548070] [DEBUG] [LOCAL ] ABEV: Trimmed 77321 records to 53281 (keeping 200 days of data) [2025-11-04T17:00:49.768153] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-01. [2025-11-04T17:00:49.858601] [DEBUG] [LOCAL ] ABEV: Primary data saved (53281 records). [2025-11-04T17:00:49.858792] [INFO ] [MAIN ] ABEV: Full history saved. [2025-11-04T17:00:49.858888] [INFO ] [MAIN ] Update Progress: 12/2367 symbols completed. ABEV took 20.1s. Estimated remaining: 2h 32m 18s. Rate: 15.5 symbols/min [2025-11-04T17:00:50.025594] [INFO ] [MAIN ] ACA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:00:50.046539] [DEBUG] [LOCAL ] ACA: Loaded 21855 existing records from primary storage. [2025-11-04T17:00:50.046805] [INFO ] [MAIN ] ACA: Loaded 21855 existing records. [2025-11-04T17:00:50.221541] [WARN ] [MAIN ] ACA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:00:50.221808] [INFO ] [MAIN ] ACA: Wiping existing primary data before full refresh. [2025-11-04T17:00:50.221862] [INFO ] [LOCAL ] ACA: Starting deletion of primary data. [2025-11-04T17:00:50.222089] [DEBUG] [LOCAL ] ACA: Deleted primary data file. [2025-11-04T17:00:50.222147] [INFO ] [MAIN ] ACA: Starting full primary refresh. [2025-11-04T17:00:50.575188] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-08. [2025-11-04T17:00:50.580135] [INFO ] [REMOTE ] ACA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:00:50.630767] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-11-04T17:00:51.366183] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-11-04T17:00:51.376143] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-02. [2025-11-04T17:00:51.704432] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T17:00:52.167099] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-01. [2025-11-04T17:00:52.924703] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-11-04T17:00:52.967081] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-02. [2025-11-04T17:00:53.112773] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T17:00:53.767112] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-03. [2025-11-04T17:00:54.487696] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-11-04T17:00:54.567053] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-03. [2025-11-04T17:00:55.350529] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-11-04T17:00:55.366995] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-04. [2025-11-04T17:00:56.031176] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T17:00:56.166996] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-04. [2025-11-04T17:00:56.927972] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T17:00:56.967060] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-05. [2025-11-04T17:00:57.754631] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-11-04T17:00:57.767009] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-05. [2025-11-04T17:00:58.567006] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-09. [2025-11-04T17:00:58.688838] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-11-04T17:00:59.370552] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-06. [2025-11-04T17:00:59.373563] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-11-04T17:00:59.739931] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T17:01:00.167171] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-10. [2025-11-04T17:01:00.967014] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-02. [2025-11-04T17:01:01.050723] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-11-04T17:01:01.228575] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-11-04T17:01:01.767079] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-11. [2025-11-04T17:01:02.160962] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T17:01:02.567077] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-03. [2025-11-04T17:01:02.944491] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T17:01:02.946040] [INFO ] [REMOTE ] ABM: Full fetch complete (45074 records). [2025-11-04T17:01:02.946133] [DEBUG] [VALIDATOR ] ABM: Validating primary data. [2025-11-04T17:01:02.947856] [FATAL] [VALIDATOR ] ABM: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:01:02.947958] [FATAL] [MAIN ] ABM: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:01:02.948015] [WARN ] [MAIN ] ABM: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:01:02.948083] [INFO ] [LOCAL ] ABM: Starting deletion of primary data. [2025-11-04T17:01:02.948157] [DEBUG] [LOCAL ] ABM: No primary data file found to delete. [2025-11-04T17:01:02.948211] [INFO ] [LOCAL ] ABM: Starting deletion of derivative data. [2025-11-04T17:01:02.948373] [DEBUG] [LOCAL ] ABM: Deleted derivative data file. [2025-11-04T17:01:02.948892] [INFO ] [MAIN ] Update Progress: 13/2367 symbols completed. ABM took 22.0s. Estimated remaining: 3h 0m 2s. Rate: 13.1 symbols/min [2025-11-04T17:01:03.128939] [INFO ] [MAIN ] ACCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:01:03.163116] [DEBUG] [LOCAL ] ACCO: Loaded 36114 existing records from primary storage. [2025-11-04T17:01:03.163404] [INFO ] [MAIN ] ACCO: Loaded 36114 existing records. [2025-11-04T17:01:03.343132] [WARN ] [MAIN ] ACCO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:01:03.343400] [INFO ] [MAIN ] ACCO: Wiping existing primary data before full refresh. [2025-11-04T17:01:03.343453] [INFO ] [LOCAL ] ACCO: Starting deletion of primary data. [2025-11-04T17:01:03.343628] [DEBUG] [LOCAL ] ACCO: Deleted primary data file. [2025-11-04T17:01:03.343733] [INFO ] [MAIN ] ACCO: Starting full primary refresh. [2025-11-04T17:01:03.373636] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-07. [2025-11-04T17:01:03.745205] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-11-04T17:01:03.743575] [INFO ] [REMOTE ] ACCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:01:04.167105] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-04. [2025-11-04T17:01:04.967017] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-01. [2025-11-04T17:01:05.214604] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-11-04T17:01:05.248199] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-11-04T17:01:05.767056] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-08. [2025-11-04T17:01:06.266328] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T17:01:06.567066] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-05. [2025-11-04T17:01:07.127879] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-11-04T17:01:07.347176] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-11-04T17:01:07.367001] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-06. [2025-11-04T17:01:08.167035] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-02. [2025-11-04T17:01:08.277226] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T17:01:08.967088] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-07. [2025-11-04T17:01:09.380750] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T17:01:09.767032] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-09. [2025-11-04T17:01:09.871092] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T17:01:10.567040] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-06. [2025-11-04T17:01:11.367044] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-08. [2025-11-04T17:01:11.494480] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-11-04T17:01:12.069157] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-11-04T17:01:12.167009] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-07. [2025-11-04T17:01:12.967031] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-03. [2025-11-04T17:01:13.767019] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-10. [2025-11-04T17:01:13.819178] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-11-04T17:01:14.192400] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T17:01:14.567056] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-08. [2025-11-04T17:01:15.114172] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T17:01:15.367055] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-11. [2025-11-04T17:01:15.912732] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-11-04T17:01:16.167028] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-04. [2025-11-04T17:01:16.402946] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T17:01:16.404514] [INFO ] [REMOTE ] ABR: Full fetch complete (75003 records). [2025-11-04T17:01:16.404626] [DEBUG] [VALIDATOR ] ABR: Validating primary data. [2025-11-04T17:01:16.405458] [FATAL] [VALIDATOR ] ABR: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:01:16.405553] [FATAL] [MAIN ] ABR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:01:16.405595] [WARN ] [MAIN ] ABR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:01:16.405634] [INFO ] [LOCAL ] ABR: Starting deletion of primary data. [2025-11-04T17:01:16.405744] [DEBUG] [LOCAL ] ABR: No primary data file found to delete. [2025-11-04T17:01:16.405791] [INFO ] [LOCAL ] ABR: Starting deletion of derivative data. [2025-11-04T17:01:16.405978] [DEBUG] [LOCAL ] ABR: Deleted derivative data file. [2025-11-04T17:01:16.406071] [INFO ] [MAIN ] Update Progress: 14/2367 symbols completed. ABR took 29.4s. Estimated remaining: 3h 24m 48s. Rate: 11.5 symbols/min [2025-11-04T17:01:16.584234] [INFO ] [MAIN ] ACEL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:01:16.610234] [DEBUG] [LOCAL ] ACEL: Loaded 25228 existing records from primary storage. [2025-11-04T17:01:16.610630] [INFO ] [MAIN ] ACEL: Loaded 25228 existing records. [2025-11-04T17:01:16.790556] [WARN ] [MAIN ] ACEL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:01:16.790883] [INFO ] [MAIN ] ACEL: Wiping existing primary data before full refresh. [2025-11-04T17:01:16.790943] [INFO ] [LOCAL ] ACEL: Starting deletion of primary data. [2025-11-04T17:01:16.791483] [DEBUG] [LOCAL ] ACEL: Deleted primary data file. [2025-11-04T17:01:16.791540] [INFO ] [MAIN ] ACEL: Starting full primary refresh. [2025-11-04T17:01:16.969398] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-09. [2025-11-04T17:01:17.156064] [INFO ] [REMOTE ] ACEL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:01:17.262540] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T17:01:17.767081] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-05. [2025-11-04T17:01:18.567054] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-01. [2025-11-04T17:01:18.659947] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-11-04T17:01:18.863190] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-11-04T17:01:19.350821] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-11-04T17:01:19.366998] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-02. [2025-11-04T17:01:19.405131] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-08 (8.04 seconds). [2025-11-04T17:01:20.167088] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-10. [2025-11-04T17:01:20.274238] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T17:01:20.967074] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-03. [2025-11-04T17:01:21.766706] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-11-04T17:01:21.776951] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-06. [2025-11-04T17:01:21.980401] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-11-04T17:01:22.567018] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-11. [2025-11-04T17:01:22.793224] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T17:01:23.367048] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-07. [2025-11-04T17:01:24.158807] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-11 (1.59 seconds). [2025-11-04T17:01:24.160369] [INFO ] [REMOTE ] ABT: Full fetch complete (77387 records). [2025-11-04T17:01:24.160479] [DEBUG] [VALIDATOR ] ABT: Validating primary data. [2025-11-04T17:01:24.161313] [FATAL] [VALIDATOR ] ABT: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:01:24.161399] [FATAL] [MAIN ] ABT: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:01:24.161441] [WARN ] [MAIN ] ABT: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:01:24.161509] [INFO ] [LOCAL ] ABT: Starting deletion of primary data. [2025-11-04T17:01:24.161571] [DEBUG] [LOCAL ] ABT: No primary data file found to delete. [2025-11-04T17:01:24.161605] [INFO ] [LOCAL ] ABT: Starting deletion of derivative data. [2025-11-04T17:01:24.161766] [DEBUG] [LOCAL ] ABT: Deleted derivative data file. [2025-11-04T17:01:24.163440] [INFO ] [MAIN ] Update Progress: 15/2367 symbols completed. ABT took 36.2s. Estimated remaining: 3h 31m 20s. Rate: 11.1 symbols/min [2025-11-04T17:01:24.175119] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-04. [2025-11-04T17:01:24.347124] [INFO ] [MAIN ] ACHR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:01:24.398551] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T17:01:24.422424] [DEBUG] [LOCAL ] ACHR: Loaded 53818 existing records from primary storage. [2025-11-04T17:01:24.422776] [INFO ] [MAIN ] ACHR: Loaded 53818 existing records. [2025-11-04T17:01:24.598127] [WARN ] [MAIN ] ACHR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:01:24.598346] [INFO ] [MAIN ] ACHR: Wiping existing primary data before full refresh. [2025-11-04T17:01:24.598392] [INFO ] [LOCAL ] ACHR: Starting deletion of primary data. [2025-11-04T17:01:24.598569] [DEBUG] [LOCAL ] ACHR: Deleted primary data file. [2025-11-04T17:01:24.598606] [INFO ] [MAIN ] ACHR: Starting full primary refresh. [2025-11-04T17:01:24.973522] [INFO ] [REMOTE ] ACHR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:01:24.969505] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-09. [2025-11-04T17:01:25.712563] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-11-04T17:01:25.767012] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-10. [2025-11-04T17:01:26.473134] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-11-04T17:01:26.566999] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-01. [2025-11-04T17:01:27.367037] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-08. [2025-11-04T17:01:28.167040] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-11. [2025-11-04T17:01:28.568077] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T17:01:28.950921] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T17:01:28.952124] [INFO ] [REMOTE ] ACA: Full fetch complete (34038 records). [2025-11-04T17:01:28.952197] [DEBUG] [VALIDATOR ] ACA: Validating primary data. [2025-11-04T17:01:28.984185] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-09. [2025-11-04T17:01:29.327972] [DEBUG] [VALIDATOR ] ACA: All expected trading days are present. [2025-11-04T17:01:29.328046] [DEBUG] [VALIDATOR ] ACA: Primary data validation passed. [2025-11-04T17:01:29.328414] [INFO ] [MAIN ] ACA: Fetched primary data is valid. [2025-11-04T17:01:29.328471] [DEBUG] [LOCAL ] ACA: Saving primary data. [2025-11-04T17:01:29.329204] [DEBUG] [LOCAL ] ACA: Trimmed 34038 records to 21757 (keeping 200 days of data) [2025-11-04T17:01:29.458237] [DEBUG] [LOCAL ] ACA: Primary data saved (21757 records). [2025-11-04T17:01:29.458529] [INFO ] [MAIN ] ACA: Full history saved. [2025-11-04T17:01:29.458602] [INFO ] [MAIN ] Update Progress: 16/2367 symbols completed. ACA took 39.6s. Estimated remaining: 3h 31m 1s. Rate: 11.1 symbols/min [2025-11-04T17:01:29.629270] [INFO ] [MAIN ] ACI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:01:29.680805] [DEBUG] [LOCAL ] ACI: Loaded 53278 existing records from primary storage. [2025-11-04T17:01:29.681105] [INFO ] [MAIN ] ACI: Loaded 53278 existing records. [2025-11-04T17:01:29.861676] [WARN ] [MAIN ] ACI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:01:29.861879] [INFO ] [MAIN ] ACI: Wiping existing primary data before full refresh. [2025-11-04T17:01:29.861944] [INFO ] [LOCAL ] ACI: Starting deletion of primary data. [2025-11-04T17:01:29.862180] [DEBUG] [LOCAL ] ACI: Deleted primary data file. [2025-11-04T17:01:29.862236] [INFO ] [MAIN ] ACI: Starting full primary refresh. [2025-11-04T17:01:30.235846] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T17:01:30.288026] [INFO ] [REMOTE ] ACI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:01:30.288970] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-01. [2025-11-04T17:01:30.567081] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-10. [2025-11-04T17:01:31.584370] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T17:01:31.595323] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-02. [2025-11-04T17:01:31.728527] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T17:01:32.167110] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-11. [2025-11-04T17:01:32.932101] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-11-04T17:01:32.967003] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-03. [2025-11-04T17:01:33.091330] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T17:01:33.093841] [INFO ] [REMOTE ] ACCO: Full fetch complete (53778 records). [2025-11-04T17:01:33.093985] [DEBUG] [VALIDATOR ] ACCO: Validating primary data. [2025-11-04T17:01:33.541134] [DEBUG] [VALIDATOR ] ACCO: All expected trading days are present. [2025-11-04T17:01:33.541214] [DEBUG] [VALIDATOR ] ACCO: Primary data validation passed. [2025-11-04T17:01:33.541912] [INFO ] [MAIN ] ACCO: Fetched primary data is valid. [2025-11-04T17:01:33.541975] [DEBUG] [LOCAL ] ACCO: Saving primary data. [2025-11-04T17:01:33.543326] [DEBUG] [LOCAL ] ACCO: Trimmed 53778 records to 36146 (keeping 200 days of data) [2025-11-04T17:01:33.750722] [DEBUG] [LOCAL ] ACCO: Primary data saved (36146 records). [2025-11-04T17:01:33.750868] [INFO ] [MAIN ] ACCO: Full history saved. [2025-11-04T17:01:33.751721] [INFO ] [MAIN ] Update Progress: 17/2367 symbols completed. ACCO took 30.8s. Estimated remaining: 3h 28m 24s. Rate: 11.3 symbols/min [2025-11-04T17:01:33.874292] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-04 (9.70 seconds). [2025-11-04T17:01:33.897610] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-05. [2025-11-04T17:01:33.952396] [INFO ] [MAIN ] ACM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:01:33.952625] [DEBUG] [LOCAL ] ACM: No existing data file found in primary storage. [2025-11-04T17:01:33.952704] [WARN ] [MAIN ] ACM: No existing data found, performing full refresh instead. [2025-11-04T17:01:33.952832] [INFO ] [MAIN ] ACM: Wiping existing primary data before full refresh. [2025-11-04T17:01:33.952875] [INFO ] [LOCAL ] ACM: Starting deletion of primary data. [2025-11-04T17:01:33.952915] [DEBUG] [LOCAL ] ACM: No primary data file found to delete. [2025-11-04T17:01:33.952946] [INFO ] [MAIN ] ACM: Starting full primary refresh. [2025-11-04T17:01:34.311814] [INFO ] [REMOTE ] ACM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:01:34.386594] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T17:01:34.567049] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-04. [2025-11-04T17:01:34.858185] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T17:01:35.367084] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-01. [2025-11-04T17:01:35.828475] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T17:01:36.167085] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-06. [2025-11-04T17:01:36.647503] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T17:01:36.908827] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-11-04T17:01:36.966991] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-07. [2025-11-04T17:01:37.767091] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-02. [2025-11-04T17:01:37.888004] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T17:01:38.567064] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-08. [2025-11-04T17:01:39.265075] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-11-04T17:01:39.367008] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-03. [2025-11-04T17:01:39.430851] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-11-04T17:01:40.167116] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-05. [2025-11-04T17:01:40.532883] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-11-04T17:01:40.967050] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-09. [2025-11-04T17:01:41.555192] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-11-04T17:01:41.767037] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-06. [2025-11-04T17:01:42.567082] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-04. [2025-11-04T17:01:43.154028] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T17:01:43.367020] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-07. [2025-11-04T17:01:43.890187] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T17:01:44.167170] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-05. [2025-11-04T17:01:44.886551] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-11-04T17:01:44.967002] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-08. [2025-11-04T17:01:45.316005] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T17:01:45.767068] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-06. [2025-11-04T17:01:46.444352] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T17:01:46.567000] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-09. [2025-11-04T17:01:46.840099] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T17:01:47.372160] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-07. [2025-11-04T17:01:47.389286] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-01 (20.82 seconds). [2025-11-04T17:01:47.962829] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T17:01:48.167069] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-02. [2025-11-04T17:01:48.802150] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-11-04T17:01:48.967068] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-08. [2025-11-04T17:01:49.767092] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-10. [2025-11-04T17:01:50.188895] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T17:01:50.311533] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-02 (2.14 seconds). [2025-11-04T17:01:50.567111] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-09. [2025-11-04T17:01:51.361170] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T17:01:51.372981] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-03. [2025-11-04T17:01:51.844453] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T17:01:52.167078] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-11. [2025-11-04T17:01:52.967083] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-10. [2025-11-04T17:01:53.530611] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-11 (1.36 seconds). [2025-11-04T17:01:53.532608] [INFO ] [REMOTE ] ACI: Full fetch complete (77228 records). [2025-11-04T17:01:53.532739] [DEBUG] [VALIDATOR ] ACI: Validating primary data. [2025-11-04T17:01:53.966003] [DEBUG] [VALIDATOR ] ACI: All expected trading days are present. [2025-11-04T17:01:53.966151] [DEBUG] [VALIDATOR ] ACI: Primary data validation passed. [2025-11-04T17:01:53.967007] [INFO ] [MAIN ] ACI: Fetched primary data is valid. [2025-11-04T17:01:53.967091] [DEBUG] [LOCAL ] ACI: Saving primary data. [2025-11-04T17:01:53.969394] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-03 (2.60 seconds). [2025-11-04T17:01:53.980912] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-04. [2025-11-04T17:01:53.981892] [DEBUG] [LOCAL ] ACI: Trimmed 77228 records to 53276 (keeping 200 days of data) [2025-11-04T17:01:54.296790] [DEBUG] [LOCAL ] ACI: Primary data saved (53276 records). [2025-11-04T17:01:54.296962] [INFO ] [MAIN ] ACI: Full history saved. [2025-11-04T17:01:54.297066] [INFO ] [MAIN ] Update Progress: 18/2367 symbols completed. ACI took 24.8s. Estimated remaining: 4h 1m 26s. Rate: 9.7 symbols/min [2025-11-04T17:01:54.464427] [INFO ] [MAIN ] ACN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:01:54.512464] [DEBUG] [LOCAL ] ACN: Loaded 53426 existing records from primary storage. [2025-11-04T17:01:54.512791] [INFO ] [MAIN ] ACN: Loaded 53426 existing records. [2025-11-04T17:01:54.691510] [WARN ] [MAIN ] ACN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:01:54.691763] [INFO ] [MAIN ] ACN: Wiping existing primary data before full refresh. [2025-11-04T17:01:54.691815] [INFO ] [LOCAL ] ACN: Starting deletion of primary data. [2025-11-04T17:01:54.692019] [DEBUG] [LOCAL ] ACN: Deleted primary data file. [2025-11-04T17:01:54.692075] [INFO ] [MAIN ] ACN: Starting full primary refresh. [2025-11-04T17:01:55.044084] [INFO ] [REMOTE ] ACN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:01:55.044390] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-01. [2025-11-04T17:01:56.021228] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-11-04T17:01:56.034361] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-05. [2025-11-04T17:01:56.496820] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T17:01:56.507254] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-02. [2025-11-04T17:01:57.988506] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-11-04T17:01:57.999688] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-03. [2025-11-04T17:01:58.348382] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-09 (17.38 seconds). [2025-11-04T17:01:58.354036] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-10. [2025-11-04T17:01:58.398116] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-05 (2.36 seconds). [2025-11-04T17:01:58.567055] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-06. [2025-11-04T17:01:59.286595] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-11-04T17:01:59.367064] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-11. [2025-11-04T17:01:59.724918] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-11-04T17:02:00.154083] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-11 (0.79 seconds). [2025-11-04T17:02:00.155465] [INFO ] [REMOTE ] ACEL: Full fetch complete (38790 records). [2025-11-04T17:02:00.155543] [DEBUG] [VALIDATOR ] ACEL: Validating primary data. [2025-11-04T17:02:00.188723] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-04. [2025-11-04T17:02:00.541361] [DEBUG] [VALIDATOR ] ACEL: All expected trading days are present. [2025-11-04T17:02:00.541492] [DEBUG] [VALIDATOR ] ACEL: Primary data validation passed. [2025-11-04T17:02:00.543009] [INFO ] [MAIN ] ACEL: Fetched primary data is valid. [2025-11-04T17:02:00.543091] [DEBUG] [LOCAL ] ACEL: Saving primary data. [2025-11-04T17:02:00.544194] [DEBUG] [LOCAL ] ACEL: Trimmed 38790 records to 25210 (keeping 200 days of data) [2025-11-04T17:02:00.684961] [DEBUG] [LOCAL ] ACEL: Primary data saved (25210 records). [2025-11-04T17:02:00.685089] [INFO ] [MAIN ] ACEL: Full history saved. [2025-11-04T17:02:00.685169] [INFO ] [MAIN ] Update Progress: 19/2367 symbols completed. ACEL took 44.3s. Estimated remaining: 4h 1m 47s. Rate: 9.7 symbols/min [2025-11-04T17:02:00.856992] [INFO ] [MAIN ] ACP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:02:00.857214] [DEBUG] [LOCAL ] ACP: No existing data file found in primary storage. [2025-11-04T17:02:00.857258] [WARN ] [MAIN ] ACP: No existing data found, performing full refresh instead. [2025-11-04T17:02:00.857363] [INFO ] [MAIN ] ACP: Wiping existing primary data before full refresh. [2025-11-04T17:02:00.857404] [INFO ] [LOCAL ] ACP: Starting deletion of primary data. [2025-11-04T17:02:00.857443] [DEBUG] [LOCAL ] ACP: No primary data file found to delete. [2025-11-04T17:02:00.857474] [INFO ] [MAIN ] ACP: Starting full primary refresh. [2025-11-04T17:02:01.226184] [INFO ] [REMOTE ] ACP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:02:01.226594] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-01. [2025-11-04T17:02:01.265924] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-06 (2.70 seconds). [2025-11-04T17:02:01.767092] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-07. [2025-11-04T17:02:01.937693] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-11-04T17:02:02.568743] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-05. [2025-11-04T17:02:02.570708] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T17:02:03.367088] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-02. [2025-11-04T17:02:04.237194] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T17:02:04.244578] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-03. [2025-11-04T17:02:04.280837] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-07 (2.51 seconds). [2025-11-04T17:02:04.321776] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-11-04T17:02:04.967051] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-06. [2025-11-04T17:02:05.476607] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T17:02:05.767044] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-04. [2025-11-04T17:02:06.082193] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-10 (13.12 seconds). [2025-11-04T17:02:06.424639] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T17:02:06.567135] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-07. [2025-11-04T17:02:06.775142] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-11-04T17:02:07.367076] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-05. [2025-11-04T17:02:08.167093] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-11. [2025-11-04T17:02:08.302573] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T17:02:08.573884] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-11-04T17:02:08.967056] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-08. [2025-11-04T17:02:09.376503] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-11 (1.21 seconds). [2025-11-04T17:02:09.378261] [INFO ] [REMOTE ] ACM: Full fetch complete (62743 records). [2025-11-04T17:02:09.378349] [DEBUG] [VALIDATOR ] ACM: Validating primary data. [2025-11-04T17:02:09.767990] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-08. [2025-11-04T17:02:09.784214] [DEBUG] [VALIDATOR ] ACM: All expected trading days are present. [2025-11-04T17:02:09.784281] [DEBUG] [VALIDATOR ] ACM: Primary data validation passed. [2025-11-04T17:02:09.784886] [INFO ] [MAIN ] ACM: Fetched primary data is valid. [2025-11-04T17:02:09.784966] [DEBUG] [LOCAL ] ACM: Saving primary data. [2025-11-04T17:02:09.786279] [DEBUG] [LOCAL ] ACM: Trimmed 62743 records to 42075 (keeping 200 days of data) [2025-11-04T17:02:10.062954] [DEBUG] [LOCAL ] ACM: Primary data saved (42075 records). [2025-11-04T17:02:10.063090] [INFO ] [MAIN ] ACM: Full history saved. [2025-11-04T17:02:10.063179] [INFO ] [MAIN ] Update Progress: 20/2367 symbols completed. ACM took 36.3s. Estimated remaining: 4h 7m 56s. Rate: 9.5 symbols/min [2025-11-04T17:02:10.237259] [INFO ] [MAIN ] ACR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:02:10.244590] [DEBUG] [LOCAL ] ACR: Loaded 5241 existing records from primary storage. [2025-11-04T17:02:10.244803] [INFO ] [MAIN ] ACR: Loaded 5241 existing records. [2025-11-04T17:02:10.430375] [WARN ] [MAIN ] ACR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:02:10.430616] [INFO ] [MAIN ] ACR: Wiping existing primary data before full refresh. [2025-11-04T17:02:10.430713] [INFO ] [LOCAL ] ACR: Starting deletion of primary data. [2025-11-04T17:02:10.431632] [DEBUG] [LOCAL ] ACR: Deleted primary data file. [2025-11-04T17:02:10.431739] [INFO ] [MAIN ] ACR: Starting full primary refresh. [2025-11-04T17:02:10.569921] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-06. [2025-11-04T17:02:10.825552] [INFO ] [REMOTE ] ACR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:02:10.857912] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-11-04T17:02:11.367092] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-09. [2025-11-04T17:02:11.664848] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T17:02:12.172116] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-07. [2025-11-04T17:02:12.191576] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-08 (2.42 seconds). [2025-11-04T17:02:12.967068] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-09. [2025-11-04T17:02:13.250279] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-11-04T17:02:13.369494] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T17:02:13.767085] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-08. [2025-11-04T17:02:14.567096] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-10. [2025-11-04T17:02:14.749129] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-11-04T17:02:15.314801] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-09 (2.35 seconds). [2025-11-04T17:02:15.367011] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-10. [2025-11-04T17:02:16.172166] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-09. [2025-11-04T17:02:16.186707] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-11-04T17:02:16.967054] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-11. [2025-11-04T17:02:17.084885] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-11-04T17:02:17.380358] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T17:02:17.382359] [INFO ] [REMOTE ] ACN: Full fetch complete (77436 records). [2025-11-04T17:02:17.382472] [DEBUG] [VALIDATOR ] ACN: Validating primary data. [2025-11-04T17:02:17.772554] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-01. [2025-11-04T17:02:17.776822] [DEBUG] [VALIDATOR ] ACN: All expected trading days are present. [2025-11-04T17:02:17.777810] [DEBUG] [VALIDATOR ] ACN: Primary data validation passed. [2025-11-04T17:02:17.778574] [INFO ] [MAIN ] ACN: Fetched primary data is valid. [2025-11-04T17:02:17.778761] [DEBUG] [LOCAL ] ACN: Saving primary data. [2025-11-04T17:02:17.780301] [DEBUG] [LOCAL ] ACN: Trimmed 77436 records to 53427 (keeping 200 days of data) [2025-11-04T17:02:18.063449] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-10 (2.70 seconds). [2025-11-04T17:02:18.176493] [DEBUG] [LOCAL ] ACN: Primary data saved (53427 records). [2025-11-04T17:02:18.176627] [INFO ] [MAIN ] ACN: Full history saved. [2025-11-04T17:02:18.178105] [INFO ] [MAIN ] Update Progress: 21/2367 symbols completed. ACN took 23.9s. Estimated remaining: 4h 11m 8s. Rate: 9.3 symbols/min [2025-11-04T17:02:18.372986] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-11-04T17:02:18.375976] [INFO ] [MAIN ] ACRE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:02:18.419693] [DEBUG] [LOCAL ] ACRE: Loaded 32126 existing records from primary storage. [2025-11-04T17:02:18.420070] [INFO ] [MAIN ] ACRE: Loaded 32126 existing records. [2025-11-04T17:02:18.572190] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-11. [2025-11-04T17:02:18.630779] [WARN ] [MAIN ] ACRE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:02:18.631232] [INFO ] [MAIN ] ACRE: Wiping existing primary data before full refresh. [2025-11-04T17:02:18.631284] [INFO ] [LOCAL ] ACRE: Starting deletion of primary data. [2025-11-04T17:02:18.631601] [DEBUG] [LOCAL ] ACRE: Deleted primary data file. [2025-11-04T17:02:18.631690] [INFO ] [MAIN ] ACRE: Starting full primary refresh. [2025-11-04T17:02:19.038721] [INFO ] [REMOTE ] ACRE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:02:19.367081] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-02. [2025-11-04T17:02:19.811549] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-11-04T17:02:20.167048] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-10. [2025-11-04T17:02:20.283445] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-11 (1.71 seconds). [2025-11-04T17:02:20.285276] [INFO ] [REMOTE ] ACHR: Full fetch complete (77608 records). [2025-11-04T17:02:20.285417] [DEBUG] [VALIDATOR ] ACHR: Validating primary data. [2025-11-04T17:02:20.286390] [FATAL] [VALIDATOR ] ACHR: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:02:20.286515] [FATAL] [MAIN ] ACHR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:02:20.286596] [WARN ] [MAIN ] ACHR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:02:20.286710] [INFO ] [LOCAL ] ACHR: Starting deletion of primary data. [2025-11-04T17:02:20.286786] [DEBUG] [LOCAL ] ACHR: No primary data file found to delete. [2025-11-04T17:02:20.286825] [INFO ] [LOCAL ] ACHR: Starting deletion of derivative data. [2025-11-04T17:02:20.286984] [DEBUG] [LOCAL ] ACHR: Deleted derivative data file. [2025-11-04T17:02:20.287714] [INFO ] [MAIN ] Update Progress: 22/2367 symbols completed. ACHR took 56.1s. Estimated remaining: 4h 3m 22s. Rate: 9.6 symbols/min [2025-11-04T17:02:20.455946] [INFO ] [MAIN ] ACV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:02:20.468490] [DEBUG] [LOCAL ] ACV: Loaded 7376 existing records from primary storage. [2025-11-04T17:02:20.468793] [INFO ] [MAIN ] ACV: Loaded 7376 existing records. [2025-11-04T17:02:20.652284] [WARN ] [MAIN ] ACV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:02:20.652524] [INFO ] [MAIN ] ACV: Wiping existing primary data before full refresh. [2025-11-04T17:02:20.652584] [INFO ] [LOCAL ] ACV: Starting deletion of primary data. [2025-11-04T17:02:20.652760] [DEBUG] [LOCAL ] ACV: Deleted primary data file. [2025-11-04T17:02:20.652818] [INFO ] [MAIN ] ACV: Starting full primary refresh. [2025-11-04T17:02:20.972135] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-03. [2025-11-04T17:02:21.008821] [INFO ] [REMOTE ] ACV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:02:21.311862] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T17:02:21.542939] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T17:02:21.767061] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-01. [2025-11-04T17:02:22.567039] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-01. [2025-11-04T17:02:22.944791] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T17:02:23.099812] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T17:02:23.367035] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-02. [2025-11-04T17:02:24.167071] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-02. [2025-11-04T17:02:24.458178] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T17:02:24.695878] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-11-04T17:02:24.967065] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-03. [2025-11-04T17:02:25.382347] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T17:02:25.767069] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-04. [2025-11-04T17:02:26.310410] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-11-04T17:02:26.567038] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-03. [2025-11-04T17:02:27.367045] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-05. [2025-11-04T17:02:27.877377] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-11-04T17:02:28.167029] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-06. [2025-11-04T17:02:28.596247] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-11-04T17:02:28.967046] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-04. [2025-11-04T17:02:29.606173] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-11-04T17:02:29.767092] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-07. [2025-11-04T17:02:30.144316] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-11-04T17:02:30.567077] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-08. [2025-11-04T17:02:30.672834] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-03 (4.11 seconds). [2025-11-04T17:02:31.072092] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-11-04T17:02:31.367055] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-09. [2025-11-04T17:02:31.938844] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-11-04T17:02:32.167087] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-11. [2025-11-04T17:02:32.967074] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-10. [2025-11-04T17:02:33.146379] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-11 (0.98 seconds). [2025-11-04T17:02:33.148004] [INFO ] [REMOTE ] ACP: Full fetch complete (45875 records). [2025-11-04T17:02:33.148083] [DEBUG] [VALIDATOR ] ACP: Validating primary data. [2025-11-04T17:02:33.525173] [DEBUG] [VALIDATOR ] ACP: All expected trading days are present. [2025-11-04T17:02:33.525278] [DEBUG] [VALIDATOR ] ACP: Primary data validation passed. [2025-11-04T17:02:33.525810] [INFO ] [MAIN ] ACP: Fetched primary data is valid. [2025-11-04T17:02:33.525872] [DEBUG] [LOCAL ] ACP: Saving primary data. [2025-11-04T17:02:33.526916] [DEBUG] [LOCAL ] ACP: Trimmed 45875 records to 30717 (keeping 200 days of data) [2025-11-04T17:02:33.643063] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-11-04T17:02:33.716422] [DEBUG] [LOCAL ] ACP: Primary data saved (30717 records). [2025-11-04T17:02:33.716547] [INFO ] [MAIN ] ACP: Full history saved. [2025-11-04T17:02:33.716615] [INFO ] [MAIN ] Update Progress: 23/2367 symbols completed. ACP took 33.0s. Estimated remaining: 4h 15m 30s. Rate: 9.2 symbols/min [2025-11-04T17:02:33.772154] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-04. [2025-11-04T17:02:33.893452] [INFO ] [MAIN ] ACVA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:02:33.893716] [DEBUG] [LOCAL ] ACVA: No existing data file found in primary storage. [2025-11-04T17:02:33.893768] [WARN ] [MAIN ] ACVA: No existing data found, performing full refresh instead. [2025-11-04T17:02:33.893877] [INFO ] [MAIN ] ACVA: Wiping existing primary data before full refresh. [2025-11-04T17:02:33.893919] [INFO ] [LOCAL ] ACVA: Starting deletion of primary data. [2025-11-04T17:02:33.893961] [DEBUG] [LOCAL ] ACVA: No primary data file found to delete. [2025-11-04T17:02:33.893993] [INFO ] [MAIN ] ACVA: Starting full primary refresh. [2025-11-04T17:02:34.250896] [INFO ] [REMOTE ] ACVA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:02:34.567083] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-01. [2025-11-04T17:02:35.019495] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T17:02:35.367071] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-05. [2025-11-04T17:02:35.873063] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-11-04T17:02:36.167021] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-06. [2025-11-04T17:02:36.628789] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-11-04T17:02:36.967025] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-07. [2025-11-04T17:02:37.393424] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-11-04T17:02:37.459926] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-01 (2.89 seconds). [2025-11-04T17:02:37.767099] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-08. [2025-11-04T17:02:38.156602] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-11-04T17:02:38.567048] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-02. [2025-11-04T17:02:39.367047] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-05. [2025-11-04T17:02:39.829969] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T17:02:40.167195] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-03. [2025-11-04T17:02:40.437636] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T17:02:40.967048] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-11. [2025-11-04T17:02:41.532385] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T17:02:41.536323] [INFO ] [REMOTE ] ACV: Full fetch complete (10776 records). [2025-11-04T17:02:41.538352] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T17:02:41.536819] [DEBUG] [VALIDATOR ] ACV: Validating primary data. [2025-11-04T17:02:41.772478] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-09. [2025-11-04T17:02:41.992025] [DEBUG] [VALIDATOR ] ACV: All expected trading days are present. [2025-11-04T17:02:41.992111] [DEBUG] [VALIDATOR ] ACV: Primary data validation passed. [2025-11-04T17:02:41.992309] [INFO ] [MAIN ] ACV: Fetched primary data is valid. [2025-11-04T17:02:41.992350] [DEBUG] [LOCAL ] ACV: Saving primary data. [2025-11-04T17:02:41.993027] [DEBUG] [LOCAL ] ACV: Trimmed 10776 records to 7394 (keeping 200 days of data) [2025-11-04T17:02:42.043269] [DEBUG] [LOCAL ] ACV: Primary data saved (7394 records). [2025-11-04T17:02:42.043409] [INFO ] [MAIN ] ACV: Full history saved. [2025-11-04T17:02:42.043494] [INFO ] [MAIN ] Update Progress: 24/2367 symbols completed. ACV took 21.8s. Estimated remaining: 4h 18m 18s. Rate: 9.1 symbols/min [2025-11-04T17:02:42.231706] [INFO ] [MAIN ] AD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:02:42.231877] [DEBUG] [LOCAL ] AD: No existing data file found in primary storage. [2025-11-04T17:02:42.231913] [WARN ] [MAIN ] AD: No existing data found, performing full refresh instead. [2025-11-04T17:02:42.232030] [INFO ] [MAIN ] AD: Wiping existing primary data before full refresh. [2025-11-04T17:02:42.232070] [INFO ] [LOCAL ] AD: Starting deletion of primary data. [2025-11-04T17:02:42.232108] [DEBUG] [LOCAL ] AD: No primary data file found to delete. [2025-11-04T17:02:42.232138] [INFO ] [MAIN ] AD: Starting full primary refresh. [2025-11-04T17:02:42.370606] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-11-04T17:02:42.572138] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-06. [2025-11-04T17:02:42.649610] [INFO ] [REMOTE ] AD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:02:43.367041] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-01. [2025-11-04T17:02:43.484039] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T17:02:43.531984] [DEBUG] [REMOTE ] AD: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:02:43.532313] [WARN ] [REMOTE ] AD: No data returned for month 2025-01 [2025-11-04T17:02:43.532414] [FATAL] [REMOTE ] AD: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:02:43.532502] [FATAL] [MAIN ] AD: Failed to fetch primary data. [2025-11-04T17:02:43.532575] [WARN ] [MAIN ] AD: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:02:43.532681] [INFO ] [LOCAL ] AD: Starting deletion of primary data. [2025-11-04T17:02:43.532770] [DEBUG] [LOCAL ] AD: No primary data file found to delete. [2025-11-04T17:02:43.532822] [INFO ] [LOCAL ] AD: Starting deletion of derivative data. [2025-11-04T17:02:43.532892] [DEBUG] [LOCAL ] AD: No derivative data file found to delete. [2025-11-04T17:02:43.532949] [INFO ] [MAIN ] Update Progress: 25/2367 symbols completed. AD took 1.5s. Estimated remaining: 4h 10m 11s. Rate: 9.4 symbols/min [2025-11-04T17:02:43.718284] [INFO ] [MAIN ] ADC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:02:43.760681] [DEBUG] [LOCAL ] ADC: Loaded 46248 existing records from primary storage. [2025-11-04T17:02:43.761025] [INFO ] [MAIN ] ADC: Loaded 46248 existing records. [2025-11-04T17:02:43.935741] [WARN ] [MAIN ] ADC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:02:43.935957] [INFO ] [MAIN ] ADC: Wiping existing primary data before full refresh. [2025-11-04T17:02:43.936004] [INFO ] [LOCAL ] ADC: Starting deletion of primary data. [2025-11-04T17:02:43.936223] [DEBUG] [LOCAL ] ADC: Deleted primary data file. [2025-11-04T17:02:43.936279] [INFO ] [MAIN ] ADC: Starting full primary refresh. [2025-11-04T17:02:44.177550] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-07. [2025-11-04T17:02:44.300447] [INFO ] [REMOTE ] ADC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:02:44.967104] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-10. [2025-11-04T17:02:45.392001] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-11-04T17:02:45.767055] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-11. [2025-11-04T17:02:45.826581] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-11-04T17:02:46.277438] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-11 (0.51 seconds). [2025-11-04T17:02:46.278442] [INFO ] [REMOTE ] ACR: Full fetch complete (8630 records). [2025-11-04T17:02:46.278528] [DEBUG] [VALIDATOR ] ACR: Validating primary data. [2025-11-04T17:02:46.575216] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-04. [2025-11-04T17:02:46.663815] [DEBUG] [VALIDATOR ] ACR: All expected trading days are present. [2025-11-04T17:02:46.663900] [DEBUG] [VALIDATOR ] ACR: Primary data validation passed. [2025-11-04T17:02:46.664266] [INFO ] [MAIN ] ACR: Fetched primary data is valid. [2025-11-04T17:02:46.664303] [DEBUG] [LOCAL ] ACR: Saving primary data. [2025-11-04T17:02:46.664778] [DEBUG] [LOCAL ] ACR: Trimmed 8630 records to 5228 (keeping 200 days of data) [2025-11-04T17:02:46.711127] [DEBUG] [LOCAL ] ACR: Primary data saved (5228 records). [2025-11-04T17:02:46.711291] [INFO ] [MAIN ] ACR: Full history saved. [2025-11-04T17:02:46.711393] [INFO ] [MAIN ] Update Progress: 26/2367 symbols completed. ACR took 36.6s. Estimated remaining: 4h 5m 14s. Rate: 9.5 symbols/min [2025-11-04T17:02:46.891237] [INFO ] [MAIN ] ADCT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:02:46.926209] [DEBUG] [LOCAL ] ADCT: Loaded 36204 existing records from primary storage. [2025-11-04T17:02:46.926491] [INFO ] [MAIN ] ADCT: Loaded 36204 existing records. [2025-11-04T17:02:47.110683] [WARN ] [MAIN ] ADCT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:02:47.110912] [INFO ] [MAIN ] ADCT: Wiping existing primary data before full refresh. [2025-11-04T17:02:47.110959] [INFO ] [LOCAL ] ADCT: Starting deletion of primary data. [2025-11-04T17:02:47.111131] [DEBUG] [LOCAL ] ADCT: Deleted primary data file. [2025-11-04T17:02:47.111168] [INFO ] [MAIN ] ADCT: Starting full primary refresh. [2025-11-04T17:02:47.372229] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-08. [2025-11-04T17:02:47.470815] [INFO ] [REMOTE ] ADCT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:02:48.009176] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T17:02:48.167105] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-01. [2025-11-04T17:02:48.264319] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T17:02:48.967053] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-05. [2025-11-04T17:02:49.365230] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T17:02:49.767084] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-09. [2025-11-04T17:02:50.248879] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T17:02:50.567065] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-02. [2025-11-04T17:02:50.745873] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T17:02:51.367097] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-06. [2025-11-04T17:02:51.637486] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T17:02:52.167082] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-10. [2025-11-04T17:02:52.856730] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-11-04T17:02:52.967035] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-03. [2025-11-04T17:02:53.150207] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-11-04T17:02:53.767045] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-07. [2025-11-04T17:02:54.567029] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-01. [2025-11-04T17:02:55.014543] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T17:02:55.367089] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-08. [2025-11-04T17:02:55.666357] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-03 (2.70 seconds). [2025-11-04T17:02:55.770694] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T17:02:56.167090] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-11. [2025-11-04T17:02:56.589914] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T17:02:56.967046] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-04. [2025-11-04T17:02:57.150795] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-11 (0.98 seconds). [2025-11-04T17:02:57.151992] [INFO ] [REMOTE ] ACRE: Full fetch complete (48457 records). [2025-11-04T17:02:57.152067] [DEBUG] [VALIDATOR ] ACRE: Validating primary data. [2025-11-04T17:02:57.152692] [FATAL] [VALIDATOR ] ACRE: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:02:57.152794] [FATAL] [MAIN ] ACRE: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:02:57.152870] [WARN ] [MAIN ] ACRE: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:02:57.152961] [INFO ] [LOCAL ] ACRE: Starting deletion of primary data. [2025-11-04T17:02:57.153042] [DEBUG] [LOCAL ] ACRE: No primary data file found to delete. [2025-11-04T17:02:57.153079] [INFO ] [LOCAL ] ACRE: Starting deletion of derivative data. [2025-11-04T17:02:57.153205] [DEBUG] [LOCAL ] ACRE: Deleted derivative data file. [2025-11-04T17:02:57.153294] [INFO ] [MAIN ] Update Progress: 27/2367 symbols completed. ACRE took 39.0s. Estimated remaining: 4h 11m 8s. Rate: 9.3 symbols/min [2025-11-04T17:02:57.318342] [INFO ] [MAIN ] ADM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:02:57.318527] [DEBUG] [LOCAL ] ADM: No existing data file found in primary storage. [2025-11-04T17:02:57.318564] [WARN ] [MAIN ] ADM: No existing data found, performing full refresh instead. [2025-11-04T17:02:57.318729] [INFO ] [MAIN ] ADM: Wiping existing primary data before full refresh. [2025-11-04T17:02:57.318812] [INFO ] [LOCAL ] ADM: Starting deletion of primary data. [2025-11-04T17:02:57.318859] [DEBUG] [LOCAL ] ADM: No primary data file found to delete. [2025-11-04T17:02:57.318890] [INFO ] [MAIN ] ADM: Starting full primary refresh. [2025-11-04T17:02:57.674887] [INFO ] [REMOTE ] ADM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:02:57.767022] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-01. [2025-11-04T17:02:58.252765] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T17:02:58.567119] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-02. [2025-11-04T17:02:59.054404] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T17:02:59.367060] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-05. [2025-11-04T17:02:59.622434] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T17:03:00.167142] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-03. [2025-11-04T17:03:00.967054] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-02. [2025-11-04T17:03:01.129543] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T17:03:01.225895] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-11-04T17:03:01.767043] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-04. [2025-11-04T17:03:02.474278] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-11-04T17:03:02.567076] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-03. [2025-11-04T17:03:02.710602] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-11-04T17:03:03.367041] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-09. [2025-11-04T17:03:04.025281] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T17:03:04.167011] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-04. [2025-11-04T17:03:04.793734] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T17:03:04.967100] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-10. [2025-11-04T17:03:05.566025] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T17:03:05.767065] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-05. [2025-11-04T17:03:06.368230] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T17:03:06.567010] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-11. [2025-11-04T17:03:07.367057] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-05. [2025-11-04T17:03:07.418084] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-11-04T17:03:07.685837] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T17:03:07.687854] [INFO ] [REMOTE ] ACVA: Full fetch complete (71383 records). [2025-11-04T17:03:07.688011] [DEBUG] [VALIDATOR ] ACVA: Validating primary data. [2025-11-04T17:03:08.132382] [DEBUG] [VALIDATOR ] ACVA: All expected trading days are present. [2025-11-04T17:03:08.132494] [DEBUG] [VALIDATOR ] ACVA: Primary data validation passed. [2025-11-04T17:03:08.133372] [INFO ] [MAIN ] ACVA: Fetched primary data is valid. [2025-11-04T17:03:08.133441] [DEBUG] [LOCAL ] ACVA: Saving primary data. [2025-11-04T17:03:08.134829] [DEBUG] [LOCAL ] ACVA: Trimmed 71383 records to 49074 (keeping 200 days of data) [2025-11-04T17:03:08.189802] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-06. [2025-11-04T17:03:08.472716] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T17:03:08.494449] [DEBUG] [LOCAL ] ACVA: Primary data saved (49074 records). [2025-11-04T17:03:08.495044] [INFO ] [MAIN ] ACVA: Full history saved. [2025-11-04T17:03:08.495140] [INFO ] [MAIN ] Update Progress: 28/2367 symbols completed. ACVA took 34.8s. Estimated remaining: 4h 17m 51s. Rate: 9.1 symbols/min [2025-11-04T17:03:08.706293] [INFO ] [MAIN ] ADNT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:03:08.751607] [DEBUG] [LOCAL ] ADNT: Loaded 45653 existing records from primary storage. [2025-11-04T17:03:08.751978] [INFO ] [MAIN ] ADNT: Loaded 45653 existing records. [2025-11-04T17:03:08.971292] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-06. [2025-11-04T17:03:08.975540] [WARN ] [MAIN ] ADNT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:03:08.977153] [INFO ] [MAIN ] ADNT: Wiping existing primary data before full refresh. [2025-11-04T17:03:08.977216] [INFO ] [LOCAL ] ADNT: Starting deletion of primary data. [2025-11-04T17:03:08.977427] [DEBUG] [LOCAL ] ADNT: Deleted primary data file. [2025-11-04T17:03:08.977485] [INFO ] [MAIN ] ADNT: Starting full primary refresh. [2025-11-04T17:03:09.421872] [INFO ] [REMOTE ] ADNT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:03:09.571289] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-11-04T17:03:09.767055] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-01. [2025-11-04T17:03:10.361948] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T17:03:10.567046] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-06. [2025-11-04T17:03:11.097245] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T17:03:11.367048] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-02. [2025-11-04T17:03:11.756095] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-11-04T17:03:12.167039] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-07. [2025-11-04T17:03:12.481547] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T17:03:12.967044] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-07. [2025-11-04T17:03:13.698575] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T17:03:13.767041] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-03. [2025-11-04T17:03:14.245002] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T17:03:14.567032] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-08. [2025-11-04T17:03:15.014820] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-11-04T17:03:15.367040] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-04. [2025-11-04T17:03:16.041607] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-11-04T17:03:16.167051] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-09. [2025-11-04T17:03:16.627245] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T17:03:16.967063] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-08. [2025-11-04T17:03:17.373162] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T17:03:17.767047] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-10. [2025-11-04T17:03:18.567047] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-07. [2025-11-04T17:03:18.716412] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-11-04T17:03:18.962713] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T17:03:19.367042] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-05. [2025-11-04T17:03:19.668800] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T17:03:20.167053] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-09. [2025-11-04T17:03:20.714821] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-11-04T17:03:20.967039] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-06. [2025-11-04T17:03:21.650423] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-11-04T17:03:21.767037] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-10. [2025-11-04T17:03:22.190525] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-11-04T17:03:22.567037] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-08. [2025-11-04T17:03:23.367161] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-11. [2025-11-04T17:03:23.503684] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-11-04T17:03:23.571871] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-11-04T17:03:24.167042] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-11. [2025-11-04T17:03:24.967074] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-07. [2025-11-04T17:03:25.246491] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-11 (1.88 seconds). [2025-11-04T17:03:25.248386] [INFO ] [REMOTE ] ADC: Full fetch complete (66974 records). [2025-11-04T17:03:25.248475] [DEBUG] [VALIDATOR ] ADC: Validating primary data. [2025-11-04T17:03:25.548908] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-11 (1.38 seconds). [2025-11-04T17:03:25.576681] [INFO ] [REMOTE ] ADM: Full fetch complete (77088 records). [2025-11-04T17:03:25.609441] [DEBUG] [VALIDATOR ] ADM: Validating primary data. [2025-11-04T17:03:25.661406] [FATAL] [VALIDATOR ] ADM: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:03:25.661710] [FATAL] [MAIN ] ADM: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:03:25.661620] [DEBUG] [VALIDATOR ] ADC: All expected trading days are present. [2025-11-04T17:03:25.661845] [DEBUG] [VALIDATOR ] ADC: Primary data validation passed. [2025-11-04T17:03:25.662591] [INFO ] [MAIN ] ADC: Fetched primary data is valid. [2025-11-04T17:03:25.662702] [DEBUG] [LOCAL ] ADC: Saving primary data. [2025-11-04T17:03:25.661897] [WARN ] [MAIN ] ADM: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:03:25.662920] [INFO ] [LOCAL ] ADM: Starting deletion of primary data. [2025-11-04T17:03:25.663008] [DEBUG] [LOCAL ] ADM: No primary data file found to delete. [2025-11-04T17:03:25.663067] [INFO ] [LOCAL ] ADM: Starting deletion of derivative data. [2025-11-04T17:03:25.663112] [DEBUG] [LOCAL ] ADM: No derivative data file found to delete. [2025-11-04T17:03:25.663183] [INFO ] [MAIN ] Update Progress: 29/2367 symbols completed. ADM took 28.5s. Estimated remaining: 4h 31m 55s. Rate: 8.6 symbols/min [2025-11-04T17:03:25.749075] [DEBUG] [LOCAL ] ADC: Trimmed 66974 records to 46237 (keeping 200 days of data) [2025-11-04T17:03:25.767597] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-09. [2025-11-04T17:03:25.906456] [INFO ] [MAIN ] ADT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:03:26.003302] [DEBUG] [LOCAL ] ADT: Loaded 53420 existing records from primary storage. [2025-11-04T17:03:26.066623] [INFO ] [MAIN ] ADT: Loaded 53420 existing records. [2025-11-04T17:03:26.253593] [DEBUG] [LOCAL ] ADC: Primary data saved (46237 records). [2025-11-04T17:03:26.261799] [INFO ] [MAIN ] ADC: Full history saved. [2025-11-04T17:03:26.273423] [INFO ] [MAIN ] Update Progress: 30/2367 symbols completed. ADC took 42.7s. Estimated remaining: 4h 23m 32s. Rate: 8.9 symbols/min [2025-11-04T17:03:26.485050] [WARN ] [MAIN ] ADT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:03:26.504582] [INFO ] [MAIN ] ADT: Wiping existing primary data before full refresh. [2025-11-04T17:03:26.505243] [INFO ] [LOCAL ] ADT: Starting deletion of primary data. [2025-11-04T17:03:26.547031] [DEBUG] [LOCAL ] ADT: Deleted primary data file. [2025-11-04T17:03:26.560959] [INFO ] [MAIN ] ADT: Starting full primary refresh. [2025-11-04T17:03:26.634088] [INFO ] [MAIN ] ADX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:03:26.658676] [DEBUG] [LOCAL ] ADX: No existing data file found in primary storage. [2025-11-04T17:03:26.664011] [WARN ] [MAIN ] ADX: No existing data found, performing full refresh instead. [2025-11-04T17:03:26.686329] [INFO ] [MAIN ] ADX: Wiping existing primary data before full refresh. [2025-11-04T17:03:26.695569] [INFO ] [LOCAL ] ADX: Starting deletion of primary data. [2025-11-04T17:03:26.707358] [DEBUG] [LOCAL ] ADX: No primary data file found to delete. [2025-11-04T17:03:26.734300] [INFO ] [MAIN ] ADX: Starting full primary refresh. [2025-11-04T17:03:26.903177] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-11-04T17:03:27.013807] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-08. [2025-11-04T17:03:27.285921] [INFO ] [REMOTE ] ADT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:03:27.368359] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-01. [2025-11-04T17:03:27.371375] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-11-04T17:03:27.458939] [INFO ] [REMOTE ] ADX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:03:28.167047] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-10. [2025-11-04T17:03:28.981256] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-01. [2025-11-04T17:03:28.984855] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-11-04T17:03:29.460566] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T17:03:29.641859] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-01 (2.27 seconds). [2025-11-04T17:03:29.767073] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-02. [2025-11-04T17:03:30.313985] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T17:03:30.567065] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-11. [2025-11-04T17:03:30.954449] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T17:03:31.367057] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-03. [2025-11-04T17:03:31.854281] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-11 (1.29 seconds). [2025-11-04T17:03:31.855465] [INFO ] [REMOTE ] ADCT: Full fetch complete (51306 records). [2025-11-04T17:03:31.855543] [DEBUG] [VALIDATOR ] ADCT: Validating primary data. [2025-11-04T17:03:31.856106] [FATAL] [VALIDATOR ] ADCT: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:03:31.856189] [FATAL] [MAIN ] ADCT: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:03:31.856232] [WARN ] [MAIN ] ADCT: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:03:31.856294] [INFO ] [LOCAL ] ADCT: Starting deletion of primary data. [2025-11-04T17:03:31.856356] [DEBUG] [LOCAL ] ADCT: No primary data file found to delete. [2025-11-04T17:03:31.856392] [INFO ] [LOCAL ] ADCT: Starting deletion of derivative data. [2025-11-04T17:03:31.856519] [DEBUG] [LOCAL ] ADCT: Deleted derivative data file. [2025-11-04T17:03:31.858057] [INFO ] [MAIN ] Update Progress: 31/2367 symbols completed. ADCT took 45.1s. Estimated remaining: 4h 21m 56s. Rate: 8.9 symbols/min [2025-11-04T17:03:32.026677] [INFO ] [MAIN ] AEE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:03:32.026861] [DEBUG] [LOCAL ] AEE: No existing data file found in primary storage. [2025-11-04T17:03:32.026965] [WARN ] [MAIN ] AEE: No existing data found, performing full refresh instead. [2025-11-04T17:03:32.027081] [INFO ] [MAIN ] AEE: Wiping existing primary data before full refresh. [2025-11-04T17:03:32.027123] [INFO ] [LOCAL ] AEE: Starting deletion of primary data. [2025-11-04T17:03:32.027169] [DEBUG] [LOCAL ] AEE: No primary data file found to delete. [2025-11-04T17:03:32.027202] [INFO ] [MAIN ] AEE: Starting full primary refresh. [2025-11-04T17:03:32.172147] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-09. [2025-11-04T17:03:32.388211] [INFO ] [REMOTE ] AEE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:03:32.967015] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-01. [2025-11-04T17:03:33.011317] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-11-04T17:03:33.574575] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T17:03:33.767051] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-10. [2025-11-04T17:03:34.517245] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-11-04T17:03:34.567086] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-02. [2025-11-04T17:03:34.742673] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-11-04T17:03:35.367083] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-02. [2025-11-04T17:03:35.791768] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T17:03:36.167089] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-03. [2025-11-04T17:03:36.256033] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T17:03:36.967036] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-03. [2025-11-04T17:03:37.650323] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-11-04T17:03:37.767024] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-11. [2025-11-04T17:03:37.917837] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-11-04T17:03:38.567080] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-04. [2025-11-04T17:03:38.716601] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-11 (0.95 seconds). [2025-11-04T17:03:38.718849] [INFO ] [REMOTE ] ADNT: Full fetch complete (67447 records). [2025-11-04T17:03:38.718973] [DEBUG] [VALIDATOR ] ADNT: Validating primary data. [2025-11-04T17:03:39.184078] [DEBUG] [VALIDATOR ] ADNT: All expected trading days are present. [2025-11-04T17:03:39.184368] [DEBUG] [VALIDATOR ] ADNT: Primary data validation passed. [2025-11-04T17:03:39.185174] [INFO ] [MAIN ] ADNT: Fetched primary data is valid. [2025-11-04T17:03:39.185265] [DEBUG] [LOCAL ] ADNT: Saving primary data. [2025-11-04T17:03:39.186600] [DEBUG] [LOCAL ] ADNT: Trimmed 67447 records to 45654 (keeping 200 days of data) [2025-11-04T17:03:39.371484] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-04. [2025-11-04T17:03:39.483850] [DEBUG] [LOCAL ] ADNT: Primary data saved (45654 records). [2025-11-04T17:03:39.483979] [INFO ] [MAIN ] ADNT: Full history saved. [2025-11-04T17:03:39.484902] [INFO ] [MAIN ] Update Progress: 32/2367 symbols completed. ADNT took 31.0s. Estimated remaining: 4h 22m 55s. Rate: 8.9 symbols/min [2025-11-04T17:03:39.693965] [INFO ] [MAIN ] AEFC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:03:39.704573] [DEBUG] [LOCAL ] AEFC: Loaded 7498 existing records from primary storage. [2025-11-04T17:03:39.704864] [INFO ] [MAIN ] AEFC: Loaded 7498 existing records. [2025-11-04T17:03:39.881849] [WARN ] [MAIN ] AEFC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:03:39.882089] [INFO ] [MAIN ] AEFC: Wiping existing primary data before full refresh. [2025-11-04T17:03:39.882140] [INFO ] [LOCAL ] AEFC: Starting deletion of primary data. [2025-11-04T17:03:39.882319] [DEBUG] [LOCAL ] AEFC: Deleted primary data file. [2025-11-04T17:03:39.882376] [INFO ] [MAIN ] AEFC: Starting full primary refresh. [2025-11-04T17:03:40.168065] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-04. [2025-11-04T17:03:40.305757] [INFO ] [REMOTE ] AEFC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:03:40.551975] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-11-04T17:03:40.580713] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T17:03:40.967083] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-05. [2025-11-04T17:03:41.708383] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-11-04T17:03:41.767027] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-05. [2025-11-04T17:03:42.426957] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T17:03:42.567070] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-06. [2025-11-04T17:03:43.054280] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T17:03:43.367051] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-01. [2025-11-04T17:03:43.990091] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-11-04T17:03:44.172188] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-02. [2025-11-04T17:03:44.196263] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-11-04T17:03:44.703012] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-11-04T17:03:44.967036] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-06. [2025-11-04T17:03:45.767065] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-05. [2025-11-04T17:03:46.283897] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T17:03:46.567041] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-03. [2025-11-04T17:03:46.752130] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T17:03:47.016220] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-11-04T17:03:47.367041] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-06. [2025-11-04T17:03:48.167063] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-04. [2025-11-04T17:03:48.322979] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-11-04T17:03:48.698029] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T17:03:48.967037] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-05. [2025-11-04T17:03:49.546534] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-11-04T17:03:49.767065] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-07. [2025-11-04T17:03:50.567092] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-06. [2025-11-04T17:03:50.886382] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-11-04T17:03:51.025208] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-11-04T17:03:51.367100] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-07. [2025-11-04T17:03:52.167146] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-07. [2025-11-04T17:03:52.640079] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-11-04T17:03:52.784824] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-11-04T17:03:52.967084] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-08. [2025-11-04T17:03:53.767095] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-08. [2025-11-04T17:03:54.360331] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T17:03:54.567030] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-08. [2025-11-04T17:03:54.687264] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-11-04T17:03:55.004535] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T17:03:55.367065] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-09. [2025-11-04T17:03:56.167043] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-07. [2025-11-04T17:03:56.891022] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-11-04T17:03:56.967036] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-10. [2025-11-04T17:03:57.643742] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T17:03:57.767058] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-08. [2025-11-04T17:03:58.545866] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-11-04T17:03:58.567000] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-11. [2025-11-04T17:03:59.168452] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T17:03:59.367008] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-09. [2025-11-04T17:03:59.883741] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-11 (1.32 seconds). [2025-11-04T17:03:59.885782] [INFO ] [REMOTE ] ADT: Full fetch complete (77382 records). [2025-11-04T17:03:59.885924] [DEBUG] [VALIDATOR ] ADT: Validating primary data. [2025-11-04T17:04:00.176502] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-09. [2025-11-04T17:04:00.282579] [DEBUG] [VALIDATOR ] ADT: All expected trading days are present. [2025-11-04T17:04:00.282679] [DEBUG] [VALIDATOR ] ADT: Primary data validation passed. [2025-11-04T17:04:00.283375] [INFO ] [MAIN ] ADT: Fetched primary data is valid. [2025-11-04T17:04:00.283418] [DEBUG] [LOCAL ] ADT: Saving primary data. [2025-11-04T17:04:00.284766] [DEBUG] [LOCAL ] ADT: Trimmed 77382 records to 53422 (keeping 200 days of data) [2025-11-04T17:04:00.588785] [DEBUG] [LOCAL ] ADT: Primary data saved (53422 records). [2025-11-04T17:04:00.588914] [INFO ] [MAIN ] ADT: Full history saved. [2025-11-04T17:04:00.588994] [INFO ] [MAIN ] Update Progress: 33/2367 symbols completed. ADT took 34.9s. Estimated remaining: 4h 39m 43s. Rate: 8.3 symbols/min [2025-11-04T17:04:00.764066] [INFO ] [MAIN ] AEG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:04:00.816337] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-11-04T17:04:00.840852] [DEBUG] [LOCAL ] AEG: Loaded 49336 existing records from primary storage. [2025-11-04T17:04:00.841499] [INFO ] [MAIN ] AEG: Loaded 49336 existing records. [2025-11-04T17:04:00.977556] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-10. [2025-11-04T17:04:01.017297] [WARN ] [MAIN ] AEG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:04:01.017465] [INFO ] [MAIN ] AEG: Wiping existing primary data before full refresh. [2025-11-04T17:04:01.017512] [INFO ] [LOCAL ] AEG: Starting deletion of primary data. [2025-11-04T17:04:01.017705] [DEBUG] [LOCAL ] AEG: Deleted primary data file. [2025-11-04T17:04:01.017762] [INFO ] [MAIN ] AEG: Starting full primary refresh. [2025-11-04T17:04:01.363066] [INFO ] [REMOTE ] AEG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:04:01.591718] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T17:04:01.767040] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-01. [2025-11-04T17:04:02.567013] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-10. [2025-11-04T17:04:02.647947] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-11-04T17:04:03.100081] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T17:04:03.367063] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-02. [2025-11-04T17:04:03.856635] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T17:04:04.167098] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-11. [2025-11-04T17:04:04.527019] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T17:04:04.967046] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-11. [2025-11-04T17:04:05.017807] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-11 (0.85 seconds). [2025-11-04T17:04:05.018950] [INFO ] [REMOTE ] ADX: Full fetch complete (41284 records). [2025-11-04T17:04:05.019035] [DEBUG] [VALIDATOR ] ADX: Validating primary data. [2025-11-04T17:04:05.019827] [FATAL] [VALIDATOR ] ADX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:04:05.019919] [FATAL] [MAIN ] ADX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:04:05.019961] [WARN ] [MAIN ] ADX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:04:05.020024] [INFO ] [LOCAL ] ADX: Starting deletion of primary data. [2025-11-04T17:04:05.020109] [DEBUG] [LOCAL ] ADX: No primary data file found to delete. [2025-11-04T17:04:05.020166] [INFO ] [LOCAL ] ADX: Starting deletion of derivative data. [2025-11-04T17:04:05.020210] [DEBUG] [LOCAL ] ADX: No derivative data file found to delete. [2025-11-04T17:04:05.020264] [INFO ] [MAIN ] Update Progress: 34/2367 symbols completed. ADX took 38.7s. Estimated remaining: 4h 36m 27s. Rate: 8.4 symbols/min [2025-11-04T17:04:05.190751] [INFO ] [MAIN ] AEM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:04:05.190942] [DEBUG] [LOCAL ] AEM: No existing data file found in primary storage. [2025-11-04T17:04:05.190981] [WARN ] [MAIN ] AEM: No existing data found, performing full refresh instead. [2025-11-04T17:04:05.191092] [INFO ] [MAIN ] AEM: Wiping existing primary data before full refresh. [2025-11-04T17:04:05.191133] [INFO ] [LOCAL ] AEM: Starting deletion of primary data. [2025-11-04T17:04:05.191175] [DEBUG] [LOCAL ] AEM: No primary data file found to delete. [2025-11-04T17:04:05.191208] [INFO ] [MAIN ] AEM: Starting full primary refresh. [2025-11-04T17:04:05.305763] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T17:04:05.346841] [INFO ] [REMOTE ] AEE: Full fetch complete (72607 records). [2025-11-04T17:04:05.366538] [DEBUG] [VALIDATOR ] AEE: Validating primary data. [2025-11-04T17:04:05.459951] [FATAL] [VALIDATOR ] AEE: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:04:05.489722] [FATAL] [MAIN ] AEE: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:04:05.492493] [WARN ] [MAIN ] AEE: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:04:05.498335] [INFO ] [LOCAL ] AEE: Starting deletion of primary data. [2025-11-04T17:04:05.508818] [DEBUG] [LOCAL ] AEE: No primary data file found to delete. [2025-11-04T17:04:05.514025] [INFO ] [LOCAL ] AEE: Starting deletion of derivative data. [2025-11-04T17:04:05.519256] [DEBUG] [LOCAL ] AEE: No derivative data file found to delete. [2025-11-04T17:04:05.548297] [INFO ] [MAIN ] Update Progress: 35/2367 symbols completed. AEE took 33.7s. Estimated remaining: 4h 29m 1s. Rate: 8.7 symbols/min [2025-11-04T17:04:05.631847] [INFO ] [REMOTE ] AEM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:04:05.769689] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-03. [2025-11-04T17:04:05.790019] [INFO ] [MAIN ] AEO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:04:05.845890] [DEBUG] [LOCAL ] AEO: Loaded 53564 existing records from primary storage. [2025-11-04T17:04:05.846213] [INFO ] [MAIN ] AEO: Loaded 53564 existing records. [2025-11-04T17:04:06.067781] [WARN ] [MAIN ] AEO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:04:06.067994] [INFO ] [MAIN ] AEO: Wiping existing primary data before full refresh. [2025-11-04T17:04:06.068045] [INFO ] [LOCAL ] AEO: Starting deletion of primary data. [2025-11-04T17:04:06.068270] [DEBUG] [LOCAL ] AEO: Deleted primary data file. [2025-11-04T17:04:06.068311] [INFO ] [MAIN ] AEO: Starting full primary refresh. [2025-11-04T17:04:06.496301] [INFO ] [REMOTE ] AEO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:04:06.567012] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-01. [2025-11-04T17:04:07.147262] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T17:04:07.367038] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-04. [2025-11-04T17:04:08.041716] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T17:04:08.167008] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-02. [2025-11-04T17:04:08.733597] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-11-04T17:04:08.967093] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-05. [2025-11-04T17:04:09.627305] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-11-04T17:04:09.767127] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-01. [2025-11-04T17:04:10.478570] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-11-04T17:04:10.567065] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-03. [2025-11-04T17:04:11.376142] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-06. [2025-11-04T17:04:11.395821] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-11-04T17:04:12.083183] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T17:04:12.167033] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-02. [2025-11-04T17:04:12.885308] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-11-04T17:04:12.967054] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-04. [2025-11-04T17:04:13.682360] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-11-04T17:04:13.767045] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-07. [2025-11-04T17:04:14.419770] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T17:04:14.567038] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-09. [2025-11-04T17:04:15.047567] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T17:04:15.047909] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-11-04T17:04:15.367015] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-05. [2025-11-04T17:04:16.167050] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-10. [2025-11-04T17:04:16.654686] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-11-04T17:04:16.909715] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T17:04:16.967011] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-06. [2025-11-04T17:04:17.767104] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-08. [2025-11-04T17:04:18.311564] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T17:04:18.567046] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-07. [2025-11-04T17:04:19.253621] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T17:04:19.367082] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-09. [2025-11-04T17:04:20.055269] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-11-04T17:04:20.167074] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-08. [2025-11-04T17:04:20.721049] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T17:04:20.967069] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-10. [2025-11-04T17:04:21.767061] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-11. [2025-11-04T17:04:22.089547] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-11-04T17:04:22.320487] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T17:04:22.321595] [INFO ] [REMOTE ] AEFC: Full fetch complete (11645 records). [2025-11-04T17:04:22.321689] [DEBUG] [VALIDATOR ] AEFC: Validating primary data. [2025-11-04T17:04:22.569945] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-09. [2025-11-04T17:04:22.711884] [DEBUG] [VALIDATOR ] AEFC: All expected trading days are present. [2025-11-04T17:04:22.712073] [DEBUG] [VALIDATOR ] AEFC: Primary data validation passed. [2025-11-04T17:04:22.712307] [INFO ] [MAIN ] AEFC: Fetched primary data is valid. [2025-11-04T17:04:22.712348] [DEBUG] [LOCAL ] AEFC: Saving primary data. [2025-11-04T17:04:22.712861] [DEBUG] [LOCAL ] AEFC: Trimmed 11645 records to 7547 (keeping 200 days of data) [2025-11-04T17:04:22.782893] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-11-04T17:04:22.799733] [DEBUG] [LOCAL ] AEFC: Primary data saved (7547 records). [2025-11-04T17:04:22.799878] [INFO ] [MAIN ] AEFC: Full history saved. [2025-11-04T17:04:22.799981] [INFO ] [MAIN ] Update Progress: 36/2367 symbols completed. AEFC took 43.3s. Estimated remaining: 4h 40m 3s. Rate: 8.3 symbols/min [2025-11-04T17:04:22.968716] [INFO ] [MAIN ] AER: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:04:23.026157] [DEBUG] [LOCAL ] AER: Loaded 47595 existing records from primary storage. [2025-11-04T17:04:23.026484] [INFO ] [MAIN ] AER: Loaded 47595 existing records. [2025-11-04T17:04:23.201926] [WARN ] [MAIN ] AER: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:04:23.202160] [INFO ] [MAIN ] AER: Wiping existing primary data before full refresh. [2025-11-04T17:04:23.202210] [INFO ] [LOCAL ] AER: Starting deletion of primary data. [2025-11-04T17:04:23.202407] [DEBUG] [LOCAL ] AER: Deleted primary data file. [2025-11-04T17:04:23.202463] [INFO ] [MAIN ] AER: Starting full primary refresh. [2025-11-04T17:04:23.372718] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-03. [2025-11-04T17:04:23.557221] [INFO ] [REMOTE ] AER: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:04:24.167104] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-01. [2025-11-04T17:04:24.255753] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-11-04T17:04:24.967062] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-11. [2025-11-04T17:04:25.123795] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-11-04T17:04:25.364398] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T17:04:25.767109] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-02. [2025-11-04T17:04:26.254523] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-11 (1.29 seconds). [2025-11-04T17:04:26.255987] [INFO ] [REMOTE ] AEG: Full fetch complete (70367 records). [2025-11-04T17:04:26.256134] [DEBUG] [VALIDATOR ] AEG: Validating primary data. [2025-11-04T17:04:26.256958] [FATAL] [VALIDATOR ] AEG: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:04:26.257063] [FATAL] [MAIN ] AEG: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:04:26.257107] [WARN ] [MAIN ] AEG: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:04:26.257143] [INFO ] [LOCAL ] AEG: Starting deletion of primary data. [2025-11-04T17:04:26.257197] [DEBUG] [LOCAL ] AEG: No primary data file found to delete. [2025-11-04T17:04:26.257230] [INFO ] [LOCAL ] AEG: Starting deletion of derivative data. [2025-11-04T17:04:26.257357] [DEBUG] [LOCAL ] AEG: Deleted derivative data file. [2025-11-04T17:04:26.258509] [INFO ] [MAIN ] Update Progress: 37/2367 symbols completed. AEG took 25.7s. Estimated remaining: 4h 35m 60s. Rate: 8.4 symbols/min [2025-11-04T17:04:26.567090] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-10. [2025-11-04T17:04:26.934096] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T17:04:27.367156] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-03. [2025-11-04T17:04:28.167068] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-04. [2025-11-04T17:04:28.397963] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-11-04T17:04:28.725181] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T17:04:28.967049] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-11. [2025-11-04T17:04:29.767002] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-04. [2025-11-04T17:04:29.844024] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-11-04T17:04:30.244217] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-11 (1.28 seconds). [2025-11-04T17:04:30.246203] [INFO ] [REMOTE ] AEO: Full fetch complete (77656 records). [2025-11-04T17:04:30.246323] [DEBUG] [VALIDATOR ] AEO: Validating primary data. [2025-11-04T17:04:30.580805] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-05. [2025-11-04T17:04:30.638810] [DEBUG] [VALIDATOR ] AEO: All expected trading days are present. [2025-11-04T17:04:30.638891] [DEBUG] [VALIDATOR ] AEO: Primary data validation passed. [2025-11-04T17:04:30.639587] [INFO ] [MAIN ] AEO: Fetched primary data is valid. [2025-11-04T17:04:30.639630] [DEBUG] [LOCAL ] AEO: Saving primary data. [2025-11-04T17:04:30.641188] [DEBUG] [LOCAL ] AEO: Trimmed 77656 records to 53564 (keeping 200 days of data) [2025-11-04T17:04:30.964569] [DEBUG] [LOCAL ] AEO: Primary data saved (53564 records). [2025-11-04T17:04:30.964813] [INFO ] [MAIN ] AEO: Full history saved. [2025-11-04T17:04:30.965825] [INFO ] [MAIN ] Update Progress: 38/2367 symbols completed. AEO took 25.4s. Estimated remaining: 4h 33m 25s. Rate: 8.5 symbols/min [2025-11-04T17:04:30.997073] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-11-04T17:04:31.367050] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-05. [2025-11-04T17:04:32.423437] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-11-04T17:04:32.434480] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-06. [2025-11-04T17:04:32.505549] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T17:04:32.967057] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-06. [2025-11-04T17:04:33.741803] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T17:04:33.767046] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-07. [2025-11-04T17:04:34.263732] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T17:04:34.567041] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-07. [2025-11-04T17:04:35.482135] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-11-04T17:04:35.493937] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-08. [2025-11-04T17:04:36.049978] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T17:04:36.167155] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-08. [2025-11-04T17:04:36.955610] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-11-04T17:04:36.966996] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-09. [2025-11-04T17:04:37.392829] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T17:04:37.767096] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-09. [2025-11-04T17:04:38.804385] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-11-04T17:04:38.816070] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-10. [2025-11-04T17:04:38.983242] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T17:04:39.367093] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-10. [2025-11-04T17:04:40.477360] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-11-04T17:04:40.490078] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-11. [2025-11-04T17:04:40.562842] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T17:04:40.967075] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-11. [2025-11-04T17:04:40.981266] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-11 (0.49 seconds). [2025-11-04T17:04:40.983435] [INFO ] [REMOTE ] AEM: Full fetch complete (77550 records). [2025-11-04T17:04:40.983565] [DEBUG] [VALIDATOR ] AEM: Validating primary data. [2025-11-04T17:04:41.301013] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-11 (0.33 seconds). [2025-11-04T17:04:41.329039] [INFO ] [REMOTE ] AER: Full fetch complete (70047 records). [2025-11-04T17:04:41.358633] [DEBUG] [VALIDATOR ] AER: Validating primary data. [2025-11-04T17:04:41.474314] [DEBUG] [VALIDATOR ] AEM: All expected trading days are present. [2025-11-04T17:04:41.479542] [DEBUG] [VALIDATOR ] AEM: Primary data validation passed. [2025-11-04T17:04:41.490985] [INFO ] [MAIN ] AEM: Fetched primary data is valid. [2025-11-04T17:04:41.501280] [DEBUG] [LOCAL ] AEM: Saving primary data. [2025-11-04T17:04:41.576292] [DEBUG] [LOCAL ] AEM: Trimmed 77550 records to 53462 (keeping 200 days of data) [2025-11-04T17:04:42.034326] [DEBUG] [VALIDATOR ] AER: All expected trading days are present. [2025-11-04T17:04:42.048281] [DEBUG] [VALIDATOR ] AER: Primary data validation passed. [2025-11-04T17:04:42.049116] [INFO ] [MAIN ] AER: Fetched primary data is valid. [2025-11-04T17:04:42.081612] [DEBUG] [LOCAL ] AER: Saving primary data. [2025-11-04T17:04:42.131241] [DEBUG] [LOCAL ] AEM: Primary data saved (53462 records). [2025-11-04T17:04:42.125385] [DEBUG] [LOCAL ] AER: Trimmed 70047 records to 47593 (keeping 200 days of data) [2025-11-04T17:04:42.131424] [INFO ] [MAIN ] AEM: Full history saved. [2025-11-04T17:04:42.131564] [INFO ] [MAIN ] Update Progress: 39/2367 symbols completed. AEM took 37.1s. Estimated remaining: 4h 37m 24s. Rate: 8.4 symbols/min [2025-11-04T17:04:42.422181] [DEBUG] [LOCAL ] AER: Primary data saved (47593 records). [2025-11-04T17:04:42.422302] [INFO ] [MAIN ] AER: Full history saved. [2025-11-04T17:04:42.423669] [INFO ] [MAIN ] Update Progress: 40/2367 symbols completed. AER took 19.6s. Estimated remaining: 4h 30m 38s. Rate: 8.6 symbols/min [2025-11-04T17:04:43.080340] [INFO ] [MAIN ] AFB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:04:43.105167] [INFO ] [MAIN ] AES: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:04:43.154299] [INFO ] [MAIN ] AEXA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:04:43.157228] [INFO ] [MAIN ] AESI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:04:43.157376] [DEBUG] [LOCAL ] AESI: No existing data file found in primary storage. [2025-11-04T17:04:43.157449] [WARN ] [MAIN ] AESI: No existing data found, performing full refresh instead. [2025-11-04T17:04:43.157681] [INFO ] [MAIN ] AESI: Wiping existing primary data before full refresh. [2025-11-04T17:04:43.157501] [DEBUG] [LOCAL ] AEXA: No existing data file found in primary storage. [2025-11-04T17:04:43.157749] [INFO ] [LOCAL ] AESI: Starting deletion of primary data. [2025-11-04T17:04:43.158761] [DEBUG] [LOCAL ] AESI: No primary data file found to delete. [2025-11-04T17:04:43.158818] [INFO ] [MAIN ] AESI: Starting full primary refresh. [2025-11-04T17:04:43.158638] [WARN ] [MAIN ] AEXA: No existing data found, performing full refresh instead. [2025-11-04T17:04:43.193914] [INFO ] [MAIN ] AEXA: Wiping existing primary data before full refresh. [2025-11-04T17:04:43.225870] [INFO ] [LOCAL ] AEXA: Starting deletion of primary data. [2025-11-04T17:04:43.236689] [DEBUG] [LOCAL ] AEXA: No primary data file found to delete. [2025-11-04T17:04:43.256513] [INFO ] [MAIN ] AEXA: Starting full primary refresh. [2025-11-04T17:04:43.304573] [DEBUG] [LOCAL ] AFB: Loaded 10898 existing records from primary storage. [2025-11-04T17:04:43.329007] [INFO ] [MAIN ] AFB: Loaded 10898 existing records. [2025-11-04T17:04:43.899761] [DEBUG] [LOCAL ] AES: Loaded 53663 existing records from primary storage. [2025-11-04T17:04:43.952208] [INFO ] [MAIN ] AES: Loaded 53663 existing records. [2025-11-04T17:04:44.008398] [WARN ] [MAIN ] AFB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:04:44.068526] [INFO ] [MAIN ] AFB: Wiping existing primary data before full refresh. [2025-11-04T17:04:44.078195] [INFO ] [LOCAL ] AFB: Starting deletion of primary data. [2025-11-04T17:04:44.131468] [DEBUG] [LOCAL ] AFB: Deleted primary data file. [2025-11-04T17:04:44.142823] [INFO ] [REMOTE ] AESI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:04:44.250898] [INFO ] [MAIN ] AFB: Starting full primary refresh. [2025-11-04T17:04:44.272583] [INFO ] [REMOTE ] AEXA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:04:44.290858] [INFO ] [REMOTE ] AEXA: Fetching primary data for 2025-01. [2025-11-04T17:04:44.296366] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-01. [2025-11-04T17:04:44.524278] [WARN ] [MAIN ] AES: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:04:44.555697] [INFO ] [MAIN ] AES: Wiping existing primary data before full refresh. [2025-11-04T17:04:44.556088] [DEBUG] [REMOTE ] AEXA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:04:44.589133] [WARN ] [REMOTE ] AEXA: No data returned for month 2025-01 [2025-11-04T17:04:44.589207] [INFO ] [LOCAL ] AES: Starting deletion of primary data. [2025-11-04T17:04:44.614152] [DEBUG] [LOCAL ] AES: Deleted primary data file. [2025-11-04T17:04:44.632889] [INFO ] [MAIN ] AES: Starting full primary refresh. [2025-11-04T17:04:44.620750] [FATAL] [REMOTE ] AEXA: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:04:44.647012] [FATAL] [MAIN ] AEXA: Failed to fetch primary data. [2025-11-04T17:04:44.679035] [WARN ] [MAIN ] AEXA: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:04:44.679158] [INFO ] [LOCAL ] AEXA: Starting deletion of primary data. [2025-11-04T17:04:44.732179] [DEBUG] [LOCAL ] AEXA: No primary data file found to delete. [2025-11-04T17:04:44.751200] [INFO ] [LOCAL ] AEXA: Starting deletion of derivative data. [2025-11-04T17:04:44.867265] [DEBUG] [LOCAL ] AEXA: No derivative data file found to delete. [2025-11-04T17:04:44.867429] [INFO ] [MAIN ] Update Progress: 41/2367 symbols completed. AEXA took 2.4s. Estimated remaining: 4h 26m 14s. Rate: 8.7 symbols/min [2025-11-04T17:04:44.870698] [INFO ] [REMOTE ] AFB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:04:44.871011] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-01. [2025-11-04T17:04:45.244442] [INFO ] [MAIN ] AFG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:04:45.260418] [DEBUG] [LOCAL ] AFG: No existing data file found in primary storage. [2025-11-04T17:04:45.270833] [WARN ] [MAIN ] AFG: No existing data found, performing full refresh instead. [2025-11-04T17:04:45.286427] [INFO ] [MAIN ] AFG: Wiping existing primary data before full refresh. [2025-11-04T17:04:45.320982] [INFO ] [LOCAL ] AFG: Starting deletion of primary data. [2025-11-04T17:04:45.331173] [DEBUG] [LOCAL ] AFG: No primary data file found to delete. [2025-11-04T17:04:45.351984] [INFO ] [REMOTE ] AES: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:04:45.352108] [INFO ] [MAIN ] AFG: Starting full primary refresh. [2025-11-04T17:04:45.352512] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-01. [2025-11-04T17:04:45.730769] [INFO ] [REMOTE ] AFG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:04:45.731353] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-01. [2025-11-04T17:04:45.757767] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-11-04T17:04:45.766214] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T17:04:45.775407] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-02. [2025-11-04T17:04:46.567067] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-02. [2025-11-04T17:04:46.773523] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-11-04T17:04:46.943622] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-11-04T17:04:46.981836] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T17:04:47.055079] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-11-04T17:04:47.367048] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-02. [2025-11-04T17:04:48.167012] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-03. [2025-11-04T17:04:48.337371] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T17:04:48.967054] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-03. [2025-11-04T17:04:49.619910] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-11-04T17:04:49.767027] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-02. [2025-11-04T17:04:49.843069] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-11-04T17:04:50.567034] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-04. [2025-11-04T17:04:51.256988] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-11-04T17:04:51.278822] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-11-04T17:04:51.367047] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-03. [2025-11-04T17:04:52.167053] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-03. [2025-11-04T17:04:52.967022] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-04. [2025-11-04T17:04:53.027803] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-11-04T17:04:53.217078] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-11-04T17:04:53.767040] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-04. [2025-11-04T17:04:54.223230] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T17:04:54.567049] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-04. [2025-11-04T17:04:54.830244] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-11-04T17:04:55.367080] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-05. [2025-11-04T17:04:56.167087] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-05. [2025-11-04T17:04:56.202798] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-11-04T17:04:56.405188] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-11-04T17:04:56.824046] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T17:04:56.967128] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-05. [2025-11-04T17:04:57.767036] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-05. [2025-11-04T17:04:58.567059] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-06. [2025-11-04T17:04:58.830029] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-11-04T17:04:59.109625] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T17:04:59.110144] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-11-04T17:04:59.367087] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-06. [2025-11-04T17:05:00.167124] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-06. [2025-11-04T17:05:00.967047] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-07. [2025-11-04T17:05:01.123730] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-11-04T17:05:01.553515] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T17:05:01.691364] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-11-04T17:05:01.767077] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-07. [2025-11-04T17:05:02.567055] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-06. [2025-11-04T17:05:03.062518] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:05:03.367071] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-08. [2025-11-04T17:05:03.882428] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-11-04T17:05:04.109225] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-11-04T17:05:04.167033] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-07. [2025-11-04T17:05:04.967027] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-09. [2025-11-04T17:05:05.593254] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T17:05:05.767056] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-10. [2025-11-04T17:05:06.001335] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-11-04T17:05:06.456153] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-11-04T17:05:06.567040] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-08. [2025-11-04T17:05:07.367053] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-08. [2025-11-04T17:05:07.958391] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T17:05:08.167091] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-09. [2025-11-04T17:05:08.643519] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-11-04T17:05:08.967061] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-09. [2025-11-04T17:05:09.650380] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-11-04T17:05:09.767068] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-07. [2025-11-04T17:05:10.164857] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T17:05:10.567069] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-10. [2025-11-04T17:05:10.616335] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T17:05:11.367030] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-08. [2025-11-04T17:05:12.160510] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T17:05:12.171628] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-11. [2025-11-04T17:05:12.344713] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-11-04T17:05:12.870318] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-11 (0.70 seconds). [2025-11-04T17:05:12.871360] [INFO ] [REMOTE ] AFB: Full fetch complete (16362 records). [2025-11-04T17:05:12.871430] [DEBUG] [VALIDATOR ] AFB: Validating primary data. [2025-11-04T17:05:12.972723] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-09. [2025-11-04T17:05:13.253386] [DEBUG] [VALIDATOR ] AFB: All expected trading days are present. [2025-11-04T17:05:13.253468] [DEBUG] [VALIDATOR ] AFB: Primary data validation passed. [2025-11-04T17:05:13.253712] [INFO ] [MAIN ] AFB: Fetched primary data is valid. [2025-11-04T17:05:13.253773] [DEBUG] [LOCAL ] AFB: Saving primary data. [2025-11-04T17:05:13.254337] [DEBUG] [LOCAL ] AFB: Trimmed 16362 records to 10882 (keeping 200 days of data) [2025-11-04T17:05:13.321968] [DEBUG] [LOCAL ] AFB: Primary data saved (10882 records). [2025-11-04T17:05:13.322107] [INFO ] [MAIN ] AFB: Full history saved. [2025-11-04T17:05:13.322190] [INFO ] [MAIN ] Update Progress: 42/2367 symbols completed. AFB took 30.9s. Estimated remaining: 4h 46m 2s. Rate: 8.1 symbols/min [2025-11-04T17:05:13.494700] [INFO ] [MAIN ] AFGB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:05:13.498317] [DEBUG] [LOCAL ] AFGB: Loaded 2070 existing records from primary storage. [2025-11-04T17:05:13.498484] [INFO ] [MAIN ] AFGB: Loaded 2070 existing records. [2025-11-04T17:05:13.682751] [WARN ] [MAIN ] AFGB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:05:13.682961] [INFO ] [MAIN ] AFGB: Wiping existing primary data before full refresh. [2025-11-04T17:05:13.683010] [INFO ] [LOCAL ] AFGB: Starting deletion of primary data. [2025-11-04T17:05:13.683162] [DEBUG] [LOCAL ] AFGB: Deleted primary data file. [2025-11-04T17:05:13.683219] [INFO ] [MAIN ] AFGB: Starting full primary refresh. [2025-11-04T17:05:13.774247] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-11. [2025-11-04T17:05:14.060904] [INFO ] [REMOTE ] AFGB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:05:14.081201] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T17:05:14.567042] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-01. [2025-11-04T17:05:14.965148] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T17:05:14.967204] [INFO ] [REMOTE ] AESI: Full fetch complete (72171 records). [2025-11-04T17:05:14.967318] [DEBUG] [VALIDATOR ] AESI: Validating primary data. [2025-11-04T17:05:15.075145] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-11-04T17:05:15.380748] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-10. [2025-11-04T17:05:15.384511] [DEBUG] [VALIDATOR ] AESI: All expected trading days are present. [2025-11-04T17:05:15.385741] [DEBUG] [VALIDATOR ] AESI: Primary data validation passed. [2025-11-04T17:05:15.386440] [INFO ] [MAIN ] AESI: Fetched primary data is valid. [2025-11-04T17:05:15.386503] [DEBUG] [LOCAL ] AESI: Saving primary data. [2025-11-04T17:05:15.387922] [DEBUG] [LOCAL ] AESI: Trimmed 72171 records to 49866 (keeping 200 days of data) [2025-11-04T17:05:15.685391] [DEBUG] [LOCAL ] AESI: Primary data saved (49866 records). [2025-11-04T17:05:15.685524] [INFO ] [MAIN ] AESI: Full history saved. [2025-11-04T17:05:15.685615] [INFO ] [MAIN ] Update Progress: 43/2367 symbols completed. AESI took 33.2s. Estimated remaining: 4h 41m 24s. Rate: 8.3 symbols/min [2025-11-04T17:05:15.851425] [INFO ] [MAIN ] AFGC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:05:15.856126] [DEBUG] [LOCAL ] AFGC: Loaded 2569 existing records from primary storage. [2025-11-04T17:05:15.856264] [INFO ] [MAIN ] AFGC: Loaded 2569 existing records. [2025-11-04T17:05:16.032450] [WARN ] [MAIN ] AFGC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:05:16.032731] [INFO ] [MAIN ] AFGC: Wiping existing primary data before full refresh. [2025-11-04T17:05:16.032791] [INFO ] [LOCAL ] AFGC: Starting deletion of primary data. [2025-11-04T17:05:16.032906] [DEBUG] [LOCAL ] AFGC: Deleted primary data file. [2025-11-04T17:05:16.032945] [INFO ] [MAIN ] AFGC: Starting full primary refresh. [2025-11-04T17:05:16.170031] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-10. [2025-11-04T17:05:16.396146] [INFO ] [REMOTE ] AFGC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:05:16.416332] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-11-04T17:05:16.967035] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-02. [2025-11-04T17:05:17.213217] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-02 (0.25 seconds). [2025-11-04T17:05:17.767043] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-03. [2025-11-04T17:05:18.034734] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-03 (0.27 seconds). [2025-11-04T17:05:18.097134] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-11-04T17:05:18.567090] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-01. [2025-11-04T17:05:19.044069] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-01 (0.48 seconds). [2025-11-04T17:05:19.367071] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-04. [2025-11-04T17:05:19.749739] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T17:05:20.167097] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-05. [2025-11-04T17:05:20.499476] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-11-04T17:05:20.967052] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-02. [2025-11-04T17:05:21.281986] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-11-04T17:05:21.767032] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-06. [2025-11-04T17:05:22.060017] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-06 (0.29 seconds). [2025-11-04T17:05:22.567066] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-11. [2025-11-04T17:05:23.367065] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-11. [2025-11-04T17:05:24.167108] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-03. [2025-11-04T17:05:24.238936] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-11 (1.67 seconds). [2025-11-04T17:05:24.241573] [INFO ] [REMOTE ] AES: Full fetch complete (77841 records). [2025-11-04T17:05:24.241791] [DEBUG] [VALIDATOR ] AES: Validating primary data. [2025-11-04T17:05:24.520749] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T17:05:24.568201] [INFO ] [REMOTE ] AFG: Full fetch complete (46960 records). [2025-11-04T17:05:24.568410] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T17:05:24.570529] [DEBUG] [VALIDATOR ] AFG: Validating primary data. [2025-11-04T17:05:24.838204] [DEBUG] [VALIDATOR ] AES: All expected trading days are present. [2025-11-04T17:05:24.853631] [DEBUG] [VALIDATOR ] AES: Primary data validation passed. [2025-11-04T17:05:24.866931] [INFO ] [MAIN ] AES: Fetched primary data is valid. [2025-11-04T17:05:24.886050] [DEBUG] [LOCAL ] AES: Saving primary data. [2025-11-04T17:05:24.972245] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-04. [2025-11-04T17:05:25.016556] [DEBUG] [LOCAL ] AES: Trimmed 77841 records to 53663 (keeping 200 days of data) [2025-11-04T17:05:25.222467] [DEBUG] [VALIDATOR ] AFG: All expected trading days are present. [2025-11-04T17:05:25.225320] [DEBUG] [VALIDATOR ] AFG: Primary data validation passed. [2025-11-04T17:05:25.225838] [INFO ] [MAIN ] AFG: Fetched primary data is valid. [2025-11-04T17:05:25.228085] [DEBUG] [LOCAL ] AFG: Saving primary data. [2025-11-04T17:05:25.233469] [DEBUG] [LOCAL ] AFG: Trimmed 46960 records to 31528 (keeping 200 days of data) [2025-11-04T17:05:25.409216] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-11-04T17:05:25.538125] [DEBUG] [LOCAL ] AES: Primary data saved (53663 records). [2025-11-04T17:05:25.540877] [INFO ] [MAIN ] AES: Full history saved. [2025-11-04T17:05:25.541885] [INFO ] [MAIN ] Update Progress: 44/2367 symbols completed. AES took 43.1s. Estimated remaining: 4h 43m 33s. Rate: 8.2 symbols/min [2025-11-04T17:05:25.770456] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-05. [2025-11-04T17:05:25.779896] [INFO ] [MAIN ] AFGD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:05:25.815570] [DEBUG] [LOCAL ] AFGD: Loaded 1669 existing records from primary storage. [2025-11-04T17:05:25.817513] [INFO ] [MAIN ] AFGD: Loaded 1669 existing records. [2025-11-04T17:05:25.950006] [DEBUG] [LOCAL ] AFG: Primary data saved (31528 records). [2025-11-04T17:05:25.967437] [INFO ] [MAIN ] AFG: Full history saved. [2025-11-04T17:05:25.985831] [INFO ] [MAIN ] Update Progress: 45/2367 symbols completed. AFG took 41.1s. Estimated remaining: 4h 37m 31s. Rate: 8.4 symbols/min [2025-11-04T17:05:26.122344] [WARN ] [MAIN ] AFGD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:05:26.129102] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-11-04T17:05:26.125832] [INFO ] [MAIN ] AFGD: Wiping existing primary data before full refresh. [2025-11-04T17:05:26.144627] [INFO ] [LOCAL ] AFGD: Starting deletion of primary data. [2025-11-04T17:05:26.175475] [DEBUG] [LOCAL ] AFGD: Deleted primary data file. [2025-11-04T17:05:26.181637] [INFO ] [MAIN ] AFGD: Starting full primary refresh. [2025-11-04T17:05:26.272497] [INFO ] [MAIN ] AFGE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:05:26.288103] [DEBUG] [LOCAL ] AFGE: No existing data file found in primary storage. [2025-11-04T17:05:26.296141] [WARN ] [MAIN ] AFGE: No existing data found, performing full refresh instead. [2025-11-04T17:05:26.298528] [INFO ] [MAIN ] AFGE: Wiping existing primary data before full refresh. [2025-11-04T17:05:26.298623] [INFO ] [LOCAL ] AFGE: Starting deletion of primary data. [2025-11-04T17:05:26.298731] [DEBUG] [LOCAL ] AFGE: No primary data file found to delete. [2025-11-04T17:05:26.304356] [INFO ] [MAIN ] AFGE: Starting full primary refresh. [2025-11-04T17:05:26.573815] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-06. [2025-11-04T17:05:26.832215] [INFO ] [REMOTE ] AFGD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:05:26.906050] [INFO ] [REMOTE ] AFGE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:05:26.983450] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-11-04T17:05:27.367048] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-01. [2025-11-04T17:05:27.774975] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-01 (0.41 seconds). [2025-11-04T17:05:28.167048] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-07. [2025-11-04T17:05:28.570266] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-11-04T17:05:28.967061] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-02. [2025-11-04T17:05:29.203809] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-02 (0.24 seconds). [2025-11-04T17:05:29.767061] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-03. [2025-11-04T17:05:30.096954] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-11-04T17:05:30.567045] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-04. [2025-11-04T17:05:30.853326] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-04 (0.29 seconds). [2025-11-04T17:05:31.367054] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-08. [2025-11-04T17:05:31.837084] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-11-04T17:05:32.167033] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-09. [2025-11-04T17:05:32.511116] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-09 (0.34 seconds). [2025-11-04T17:05:32.967031] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-07. [2025-11-04T17:05:33.443538] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-11-04T17:05:33.767039] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-01. [2025-11-04T17:05:34.282490] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-11-04T17:05:34.567017] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-02. [2025-11-04T17:05:34.914845] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-11-04T17:05:35.367035] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-10. [2025-11-04T17:05:35.662048] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-10 (0.30 seconds). [2025-11-04T17:05:36.167090] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-11. [2025-11-04T17:05:36.475147] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-11 (0.31 seconds). [2025-11-04T17:05:36.475860] [INFO ] [REMOTE ] AFGC: Full fetch complete (3532 records). [2025-11-04T17:05:36.475934] [DEBUG] [VALIDATOR ] AFGC: Validating primary data. [2025-11-04T17:05:36.476293] [FATAL] [VALIDATOR ] AFGC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:05:36.476376] [FATAL] [MAIN ] AFGC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:05:36.476419] [WARN ] [MAIN ] AFGC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:05:36.476456] [INFO ] [LOCAL ] AFGC: Starting deletion of primary data. [2025-11-04T17:05:36.476507] [DEBUG] [LOCAL ] AFGC: No primary data file found to delete. [2025-11-04T17:05:36.476541] [INFO ] [LOCAL ] AFGC: Starting deletion of derivative data. [2025-11-04T17:05:36.476751] [DEBUG] [LOCAL ] AFGC: Deleted derivative data file. [2025-11-04T17:05:36.476836] [INFO ] [MAIN ] Update Progress: 46/2367 symbols completed. AFGC took 20.8s. Estimated remaining: 4h 40m 11s. Rate: 8.3 symbols/min [2025-11-04T17:05:36.647777] [INFO ] [MAIN ] AFL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:05:36.696850] [DEBUG] [LOCAL ] AFL: Loaded 53131 existing records from primary storage. [2025-11-04T17:05:36.697167] [INFO ] [MAIN ] AFL: Loaded 53131 existing records. [2025-11-04T17:05:36.871386] [WARN ] [MAIN ] AFL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:05:36.871610] [INFO ] [MAIN ] AFL: Wiping existing primary data before full refresh. [2025-11-04T17:05:36.871684] [INFO ] [LOCAL ] AFL: Starting deletion of primary data. [2025-11-04T17:05:36.872608] [DEBUG] [LOCAL ] AFL: Deleted primary data file. [2025-11-04T17:05:36.872709] [INFO ] [MAIN ] AFL: Starting full primary refresh. [2025-11-04T17:05:36.972273] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-03. [2025-11-04T17:05:37.251972] [INFO ] [REMOTE ] AFL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:05:37.315606] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-11-04T17:05:37.767052] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-04. [2025-11-04T17:05:38.127519] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T17:05:38.567059] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-08. [2025-11-04T17:05:38.889377] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-11-04T17:05:39.367082] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-05. [2025-11-04T17:05:39.754368] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-11-04T17:05:40.167051] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-06. [2025-11-04T17:05:40.421590] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-06 (0.25 seconds). [2025-11-04T17:05:40.967051] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-09. [2025-11-04T17:05:41.276227] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-09 (0.31 seconds). [2025-11-04T17:05:41.767089] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-10. [2025-11-04T17:05:42.109101] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-10 (0.34 seconds). [2025-11-04T17:05:42.567069] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-05. [2025-11-04T17:05:43.001442] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T17:05:43.367039] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-01. [2025-11-04T17:05:44.167087] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-06. [2025-11-04T17:05:44.469360] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-06 (0.30 seconds). [2025-11-04T17:05:44.765558] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T17:05:44.967050] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-07. [2025-11-04T17:05:45.405210] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-11-04T17:05:45.767083] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-11. [2025-11-04T17:05:46.306196] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T17:05:46.306857] [INFO ] [REMOTE ] AFGB: Full fetch complete (3345 records). [2025-11-04T17:05:46.306961] [DEBUG] [VALIDATOR ] AFGB: Validating primary data. [2025-11-04T17:05:46.572213] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-07. [2025-11-04T17:05:46.671041] [DEBUG] [VALIDATOR ] AFGB: All expected trading days are present. [2025-11-04T17:05:46.671130] [DEBUG] [VALIDATOR ] AFGB: Primary data validation passed. [2025-11-04T17:05:46.671255] [INFO ] [MAIN ] AFGB: Fetched primary data is valid. [2025-11-04T17:05:46.671295] [DEBUG] [LOCAL ] AFGB: Saving primary data. [2025-11-04T17:05:46.671627] [DEBUG] [LOCAL ] AFGB: Trimmed 3345 records to 2070 (keeping 200 days of data) [2025-11-04T17:05:46.684724] [DEBUG] [LOCAL ] AFGB: Primary data saved (2070 records). [2025-11-04T17:05:46.684789] [INFO ] [MAIN ] AFGB: Full history saved. [2025-11-04T17:05:46.684854] [INFO ] [MAIN ] Update Progress: 47/2367 symbols completed. AFGB took 33.4s. Estimated remaining: 4h 42m 30s. Rate: 8.2 symbols/min [2025-11-04T17:05:46.854154] [INFO ] [MAIN ] AG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:05:46.905773] [DEBUG] [LOCAL ] AG: Loaded 53644 existing records from primary storage. [2025-11-04T17:05:46.906124] [INFO ] [MAIN ] AG: Loaded 53644 existing records. [2025-11-04T17:05:46.979715] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-11-04T17:05:47.091997] [WARN ] [MAIN ] AG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:05:47.092272] [INFO ] [MAIN ] AG: Wiping existing primary data before full refresh. [2025-11-04T17:05:47.092320] [INFO ] [LOCAL ] AG: Starting deletion of primary data. [2025-11-04T17:05:47.092772] [DEBUG] [LOCAL ] AG: Deleted primary data file. [2025-11-04T17:05:47.092835] [INFO ] [MAIN ] AG: Starting full primary refresh. [2025-11-04T17:05:47.374419] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-08. [2025-11-04T17:05:47.457126] [INFO ] [REMOTE ] AG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:05:47.723436] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-11-04T17:05:48.167077] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-09. [2025-11-04T17:05:48.455007] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-09 (0.29 seconds). [2025-11-04T17:05:48.967079] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-08. [2025-11-04T17:05:49.388008] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-11-04T17:05:49.767054] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-01. [2025-11-04T17:05:50.567066] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-09. [2025-11-04T17:05:50.866068] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-09 (0.30 seconds). [2025-11-04T17:05:51.372196] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-10. [2025-11-04T17:05:51.378948] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-11-04T17:05:51.644219] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-10 (0.27 seconds). [2025-11-04T17:05:52.167058] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-11. [2025-11-04T17:05:52.531575] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T17:05:52.532249] [INFO ] [REMOTE ] AFGD: Full fetch complete (2622 records). [2025-11-04T17:05:52.532318] [DEBUG] [VALIDATOR ] AFGD: Validating primary data. [2025-11-04T17:05:52.890610] [DEBUG] [VALIDATOR ] AFGD: All expected trading days are present. [2025-11-04T17:05:52.890728] [DEBUG] [VALIDATOR ] AFGD: Primary data validation passed. [2025-11-04T17:05:52.890854] [INFO ] [MAIN ] AFGD: Fetched primary data is valid. [2025-11-04T17:05:52.890895] [DEBUG] [LOCAL ] AFGD: Saving primary data. [2025-11-04T17:05:52.891227] [DEBUG] [LOCAL ] AFGD: Trimmed 2622 records to 1677 (keeping 200 days of data) [2025-11-04T17:05:52.901385] [DEBUG] [LOCAL ] AFGD: Primary data saved (1677 records). [2025-11-04T17:05:52.901448] [INFO ] [MAIN ] AFGD: Full history saved. [2025-11-04T17:05:52.901516] [INFO ] [MAIN ] Update Progress: 48/2367 symbols completed. AFGD took 27.4s. Estimated remaining: 4h 41m 30s. Rate: 8.2 symbols/min [2025-11-04T17:05:52.972153] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-02. [2025-11-04T17:05:53.093380] [INFO ] [MAIN ] AGCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:05:53.093549] [DEBUG] [LOCAL ] AGCO: No existing data file found in primary storage. [2025-11-04T17:05:53.093586] [WARN ] [MAIN ] AGCO: No existing data found, performing full refresh instead. [2025-11-04T17:05:53.093721] [INFO ] [MAIN ] AGCO: Wiping existing primary data before full refresh. [2025-11-04T17:05:53.093783] [INFO ] [LOCAL ] AGCO: Starting deletion of primary data. [2025-11-04T17:05:53.093827] [DEBUG] [LOCAL ] AGCO: No primary data file found to delete. [2025-11-04T17:05:53.093862] [INFO ] [MAIN ] AGCO: Starting full primary refresh. [2025-11-04T17:05:53.463629] [INFO ] [REMOTE ] AGCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:05:53.767079] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-10. [2025-11-04T17:05:54.035242] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-10 (0.27 seconds). [2025-11-04T17:05:54.410714] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-11-04T17:05:54.567026] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-03. [2025-11-04T17:05:55.367076] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-11. [2025-11-04T17:05:55.852624] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-11 (0.49 seconds). [2025-11-04T17:05:55.853518] [INFO ] [REMOTE ] AFGE: Full fetch complete (3192 records). [2025-11-04T17:05:55.853603] [DEBUG] [VALIDATOR ] AFGE: Validating primary data. [2025-11-04T17:05:56.170300] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-01. [2025-11-04T17:05:56.258065] [DEBUG] [VALIDATOR ] AFGE: All expected trading days are present. [2025-11-04T17:05:56.258200] [DEBUG] [VALIDATOR ] AFGE: Primary data validation passed. [2025-11-04T17:05:56.258360] [INFO ] [MAIN ] AFGE: Fetched primary data is valid. [2025-11-04T17:05:56.258442] [DEBUG] [LOCAL ] AFGE: Saving primary data. [2025-11-04T17:05:56.259124] [DEBUG] [LOCAL ] AFGE: Trimmed 3192 records to 2176 (keeping 200 days of data) [2025-11-04T17:05:56.273866] [DEBUG] [LOCAL ] AFGE: Primary data saved (2176 records). [2025-11-04T17:05:56.274046] [INFO ] [MAIN ] AFGE: Full history saved. [2025-11-04T17:05:56.274223] [INFO ] [MAIN ] Update Progress: 49/2367 symbols completed. AFGE took 30.3s. Estimated remaining: 4h 38m 18s. Rate: 8.3 symbols/min [2025-11-04T17:05:56.497997] [INFO ] [MAIN ] AGD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:05:56.500221] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-11-04T17:05:56.528140] [DEBUG] [LOCAL ] AGD: Loaded 18745 existing records from primary storage. [2025-11-04T17:05:56.528376] [INFO ] [MAIN ] AGD: Loaded 18745 existing records. [2025-11-04T17:05:56.703949] [WARN ] [MAIN ] AGD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:05:56.704139] [INFO ] [MAIN ] AGD: Wiping existing primary data before full refresh. [2025-11-04T17:05:56.704196] [INFO ] [LOCAL ] AGD: Starting deletion of primary data. [2025-11-04T17:05:56.704401] [DEBUG] [LOCAL ] AGD: Deleted primary data file. [2025-11-04T17:05:56.704457] [INFO ] [MAIN ] AGD: Starting full primary refresh. [2025-11-04T17:05:56.976969] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-04. [2025-11-04T17:05:57.059274] [INFO ] [REMOTE ] AGD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:05:57.483768] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T17:05:57.767137] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-01. [2025-11-04T17:05:58.567053] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-02. [2025-11-04T17:05:58.767706] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-11-04T17:05:58.791303] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T17:05:59.367040] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-05. [2025-11-04T17:06:00.021060] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-11-04T17:06:00.167047] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-02. [2025-11-04T17:06:00.967059] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-03. [2025-11-04T17:06:01.543920] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-11-04T17:06:01.603119] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-05 (2.24 seconds). [2025-11-04T17:06:01.767056] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-06. [2025-11-04T17:06:02.493470] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T17:06:02.567011] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-02. [2025-11-04T17:06:03.062285] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-11-04T17:06:03.367047] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-04. [2025-11-04T17:06:03.417167] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-11-04T17:06:04.167054] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-03. [2025-11-04T17:06:04.806358] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T17:06:04.967044] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-05. [2025-11-04T17:06:05.218296] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-11-04T17:06:05.767039] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-07. [2025-11-04T17:06:06.411255] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-11-04T17:06:06.567039] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-06. [2025-11-04T17:06:07.367090] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-03. [2025-11-04T17:06:07.645293] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-11-04T17:06:08.029513] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T17:06:08.030627] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-11-04T17:06:08.167051] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-07. [2025-11-04T17:06:08.967063] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-04. [2025-11-04T17:06:09.499291] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T17:06:09.767025] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-04. [2025-11-04T17:06:09.782479] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T17:06:10.567069] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-08. [2025-11-04T17:06:11.192271] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T17:06:11.367054] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-05. [2025-11-04T17:06:12.173128] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-08. [2025-11-04T17:06:12.198830] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-11-04T17:06:12.219136] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-11-04T17:06:12.967027] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-06. [2025-11-04T17:06:13.681498] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T17:06:13.767028] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-09. [2025-11-04T17:06:13.803824] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T17:06:14.567038] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-05. [2025-11-04T17:06:15.367068] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-09. [2025-11-04T17:06:15.625984] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-11-04T17:06:15.716006] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T17:06:16.167048] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-06. [2025-11-04T17:06:16.626895] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T17:06:16.967062] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-07. [2025-11-04T17:06:17.434881] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T17:06:17.746481] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-11-04T17:06:17.767005] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-08. [2025-11-04T17:06:18.355034] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-11-04T17:06:18.567036] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-10. [2025-11-04T17:06:19.367042] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-07. [2025-11-04T17:06:20.167040] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-09. [2025-11-04T17:06:20.767516] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-11-04T17:06:20.808840] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:06:20.967072] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-11. [2025-11-04T17:06:21.108338] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-11-04T17:06:21.767041] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-10. [2025-11-04T17:06:22.567687] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T17:06:22.567037] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-10. [2025-11-04T17:06:22.643619] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-11 (1.68 seconds). [2025-11-04T17:06:22.646320] [INFO ] [REMOTE ] AG: Full fetch complete (77810 records). [2025-11-04T17:06:22.646466] [DEBUG] [VALIDATOR ] AG: Validating primary data. [2025-11-04T17:06:23.036251] [DEBUG] [VALIDATOR ] AG: All expected trading days are present. [2025-11-04T17:06:23.036366] [DEBUG] [VALIDATOR ] AG: Primary data validation passed. [2025-11-04T17:06:23.037553] [INFO ] [MAIN ] AG: Fetched primary data is valid. [2025-11-04T17:06:23.037621] [DEBUG] [LOCAL ] AG: Saving primary data. [2025-11-04T17:06:23.039251] [DEBUG] [LOCAL ] AG: Trimmed 77810 records to 53644 (keeping 200 days of data) [2025-11-04T17:06:23.357132] [DEBUG] [LOCAL ] AG: Primary data saved (53644 records). [2025-11-04T17:06:23.357276] [INFO ] [MAIN ] AG: Full history saved. [2025-11-04T17:06:23.358336] [INFO ] [MAIN ] Update Progress: 50/2367 symbols completed. AG took 36.7s. Estimated remaining: 4h 53m 32s. Rate: 7.9 symbols/min [2025-11-04T17:06:23.372133] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-08. [2025-11-04T17:06:23.572560] [INFO ] [MAIN ] AGI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:06:23.630591] [DEBUG] [LOCAL ] AGI: Loaded 53426 existing records from primary storage. [2025-11-04T17:06:23.631033] [INFO ] [MAIN ] AGI: Loaded 53426 existing records. [2025-11-04T17:06:23.817114] [WARN ] [MAIN ] AGI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:06:23.817324] [INFO ] [MAIN ] AGI: Wiping existing primary data before full refresh. [2025-11-04T17:06:23.817371] [INFO ] [LOCAL ] AGI: Starting deletion of primary data. [2025-11-04T17:06:23.817557] [DEBUG] [LOCAL ] AGI: Deleted primary data file. [2025-11-04T17:06:23.817595] [INFO ] [MAIN ] AGI: Starting full primary refresh. [2025-11-04T17:06:24.170233] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-11. [2025-11-04T17:06:24.184370] [INFO ] [REMOTE ] AGI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:06:24.218917] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-11-04T17:06:24.538795] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T17:06:24.967087] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-09. [2025-11-04T17:06:25.044939] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T17:06:25.045747] [INFO ] [REMOTE ] AGD: Full fetch complete (25540 records). [2025-11-04T17:06:25.045815] [DEBUG] [VALIDATOR ] AGD: Validating primary data. [2025-11-04T17:06:25.046254] [FATAL] [VALIDATOR ] AGD: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:06:25.046333] [FATAL] [MAIN ] AGD: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:06:25.046376] [WARN ] [MAIN ] AGD: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:06:25.046411] [INFO ] [LOCAL ] AGD: Starting deletion of primary data. [2025-11-04T17:06:25.046461] [DEBUG] [LOCAL ] AGD: No primary data file found to delete. [2025-11-04T17:06:25.046496] [INFO ] [LOCAL ] AGD: Starting deletion of derivative data. [2025-11-04T17:06:25.046677] [DEBUG] [LOCAL ] AGD: Deleted derivative data file. [2025-11-04T17:06:25.046760] [INFO ] [MAIN ] Update Progress: 51/2367 symbols completed. AGD took 28.8s. Estimated remaining: 4h 48m 56s. Rate: 8.0 symbols/min [2025-11-04T17:06:25.217486] [INFO ] [MAIN ] AGL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:06:25.217667] [DEBUG] [LOCAL ] AGL: No existing data file found in primary storage. [2025-11-04T17:06:25.217723] [WARN ] [MAIN ] AGL: No existing data found, performing full refresh instead. [2025-11-04T17:06:25.217831] [INFO ] [MAIN ] AGL: Wiping existing primary data before full refresh. [2025-11-04T17:06:25.217872] [INFO ] [LOCAL ] AGL: Starting deletion of primary data. [2025-11-04T17:06:25.217914] [DEBUG] [LOCAL ] AGL: No primary data file found to delete. [2025-11-04T17:06:25.217954] [INFO ] [MAIN ] AGL: Starting full primary refresh. [2025-11-04T17:06:25.569577] [INFO ] [REMOTE ] AGL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:06:25.767116] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-01. [2025-11-04T17:06:26.067735] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T17:06:26.567081] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-10. [2025-11-04T17:06:27.025010] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T17:06:27.367042] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-01. [2025-11-04T17:06:27.823094] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-11-04T17:06:28.167011] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-02. [2025-11-04T17:06:28.799372] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T17:06:28.967087] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-02. [2025-11-04T17:06:29.437122] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T17:06:29.767045] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-11. [2025-11-04T17:06:30.153144] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-11 (0.39 seconds). [2025-11-04T17:06:30.155293] [INFO ] [REMOTE ] AFL: Full fetch complete (77069 records). [2025-11-04T17:06:30.155476] [DEBUG] [VALIDATOR ] AFL: Validating primary data. [2025-11-04T17:06:30.564863] [DEBUG] [VALIDATOR ] AFL: All expected trading days are present. [2025-11-04T17:06:30.565085] [DEBUG] [VALIDATOR ] AFL: Primary data validation passed. [2025-11-04T17:06:30.565878] [INFO ] [MAIN ] AFL: Fetched primary data is valid. [2025-11-04T17:06:30.566018] [DEBUG] [LOCAL ] AFL: Saving primary data. [2025-11-04T17:06:30.567098] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-11. [2025-11-04T17:06:30.577963] [DEBUG] [LOCAL ] AFL: Trimmed 77069 records to 53141 (keeping 200 days of data) [2025-11-04T17:06:30.598589] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-11-04T17:06:30.924929] [DEBUG] [LOCAL ] AFL: Primary data saved (53141 records). [2025-11-04T17:06:30.925074] [INFO ] [MAIN ] AFL: Full history saved. [2025-11-04T17:06:30.926373] [INFO ] [MAIN ] Update Progress: 52/2367 symbols completed. AFL took 54.4s. Estimated remaining: 4h 47m 37s. Rate: 8.0 symbols/min [2025-11-04T17:06:31.094500] [INFO ] [MAIN ] AGM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:06:31.107515] [DEBUG] [LOCAL ] AGM: Loaded 10494 existing records from primary storage. [2025-11-04T17:06:31.107790] [INFO ] [MAIN ] AGM: Loaded 10494 existing records. [2025-11-04T17:06:31.287915] [WARN ] [MAIN ] AGM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:06:31.288153] [INFO ] [MAIN ] AGM: Wiping existing primary data before full refresh. [2025-11-04T17:06:31.288201] [INFO ] [LOCAL ] AGM: Starting deletion of primary data. [2025-11-04T17:06:31.288355] [DEBUG] [LOCAL ] AGM: Deleted primary data file. [2025-11-04T17:06:31.288397] [INFO ] [MAIN ] AGM: Starting full primary refresh. [2025-11-04T17:06:31.372204] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-03. [2025-11-04T17:06:31.653100] [INFO ] [REMOTE ] AGM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:06:31.653350] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T17:06:31.655812] [INFO ] [REMOTE ] AGCO: Full fetch complete (61414 records). [2025-11-04T17:06:31.655951] [DEBUG] [VALIDATOR ] AGCO: Validating primary data. [2025-11-04T17:06:32.062277] [DEBUG] [VALIDATOR ] AGCO: All expected trading days are present. [2025-11-04T17:06:32.062356] [DEBUG] [VALIDATOR ] AGCO: Primary data validation passed. [2025-11-04T17:06:32.062955] [INFO ] [MAIN ] AGCO: Fetched primary data is valid. [2025-11-04T17:06:32.063015] [DEBUG] [LOCAL ] AGCO: Saving primary data. [2025-11-04T17:06:32.064170] [DEBUG] [LOCAL ] AGCO: Trimmed 61414 records to 41768 (keeping 200 days of data) [2025-11-04T17:06:32.167091] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-01. [2025-11-04T17:06:32.313260] [DEBUG] [LOCAL ] AGCO: Primary data saved (41768 records). [2025-11-04T17:06:32.313376] [INFO ] [MAIN ] AGCO: Full history saved. [2025-11-04T17:06:32.313440] [INFO ] [MAIN ] Update Progress: 53/2367 symbols completed. AGCO took 39.4s. Estimated remaining: 4h 43m 5s. Rate: 8.2 symbols/min [2025-11-04T17:06:32.479194] [INFO ] [MAIN ] AGO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:06:32.504480] [DEBUG] [LOCAL ] AGO: Loaded 27035 existing records from primary storage. [2025-11-04T17:06:32.504746] [INFO ] [MAIN ] AGO: Loaded 27035 existing records. [2025-11-04T17:06:32.680520] [WARN ] [MAIN ] AGO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:06:32.680873] [INFO ] [MAIN ] AGO: Wiping existing primary data before full refresh. [2025-11-04T17:06:32.680924] [INFO ] [LOCAL ] AGO: Starting deletion of primary data. [2025-11-04T17:06:32.681079] [DEBUG] [LOCAL ] AGO: Deleted primary data file. [2025-11-04T17:06:32.681115] [INFO ] [MAIN ] AGO: Starting full primary refresh. [2025-11-04T17:06:32.968399] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-03. [2025-11-04T17:06:33.013568] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-11-04T17:06:33.054871] [INFO ] [REMOTE ] AGO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:06:33.085890] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-11-04T17:06:33.767076] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-04. [2025-11-04T17:06:34.379141] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T17:06:34.567080] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-01. [2025-11-04T17:06:35.367021] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-04. [2025-11-04T17:06:35.547939] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T17:06:35.595808] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-11-04T17:06:36.167067] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-05. [2025-11-04T17:06:36.891109] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-11-04T17:06:36.967109] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-05. [2025-11-04T17:06:37.749738] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-11-04T17:06:37.766995] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-06. [2025-11-04T17:06:38.352586] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-11-04T17:06:38.567096] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-06. [2025-11-04T17:06:39.161704] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T17:06:39.367037] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-07. [2025-11-04T17:06:40.016432] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T17:06:40.167009] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-07. [2025-11-04T17:06:40.812794] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T17:06:40.967021] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-08. [2025-11-04T17:06:41.286935] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-11-04T17:06:41.767084] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-02. [2025-11-04T17:06:42.431023] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-11-04T17:06:42.520307] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T17:06:42.566995] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-03. [2025-11-04T17:06:43.372139] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-08. [2025-11-04T17:06:43.383182] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-11-04T17:06:44.167096] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-02. [2025-11-04T17:06:44.780626] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-11-04T17:06:44.858301] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T17:06:44.967022] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-09. [2025-11-04T17:06:45.767037] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-04. [2025-11-04T17:06:46.207599] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-11-04T17:06:46.567046] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-03. [2025-11-04T17:06:46.913041] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T17:06:47.068576] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-11-04T17:06:47.367088] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-10. [2025-11-04T17:06:48.167102] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-05. [2025-11-04T17:06:48.961312] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T17:06:48.972266] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-09. [2025-11-04T17:06:49.089988] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T17:06:49.767053] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-11. [2025-11-04T17:06:50.538612] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-11-04T17:06:50.567002] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-10. [2025-11-04T17:06:50.959626] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T17:06:50.961631] [INFO ] [REMOTE ] AGL: Full fetch complete (76020 records). [2025-11-04T17:06:50.961748] [DEBUG] [VALIDATOR ] AGL: Validating primary data. [2025-11-04T17:06:51.356752] [DEBUG] [VALIDATOR ] AGL: All expected trading days are present. [2025-11-04T17:06:51.356837] [DEBUG] [VALIDATOR ] AGL: Primary data validation passed. [2025-11-04T17:06:51.357598] [INFO ] [MAIN ] AGL: Fetched primary data is valid. [2025-11-04T17:06:51.357683] [DEBUG] [LOCAL ] AGL: Saving primary data. [2025-11-04T17:06:51.359202] [DEBUG] [LOCAL ] AGL: Trimmed 76020 records to 52579 (keeping 200 days of data) [2025-11-04T17:06:51.402540] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-04. [2025-11-04T17:06:51.647017] [DEBUG] [LOCAL ] AGL: Primary data saved (52579 records). [2025-11-04T17:06:51.647190] [INFO ] [MAIN ] AGL: Full history saved. [2025-11-04T17:06:51.647283] [INFO ] [MAIN ] Update Progress: 54/2367 symbols completed. AGL took 26.6s. Estimated remaining: 4h 51m 31s. Rate: 7.9 symbols/min [2025-11-04T17:06:51.820409] [INFO ] [MAIN ] AGRO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:06:51.849991] [DEBUG] [LOCAL ] AGRO: Loaded 31768 existing records from primary storage. [2025-11-04T17:06:51.850257] [INFO ] [MAIN ] AGRO: Loaded 31768 existing records. [2025-11-04T17:06:52.035432] [WARN ] [MAIN ] AGRO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:06:52.035906] [INFO ] [MAIN ] AGRO: Wiping existing primary data before full refresh. [2025-11-04T17:06:52.036077] [INFO ] [LOCAL ] AGRO: Starting deletion of primary data. [2025-11-04T17:06:52.036337] [DEBUG] [LOCAL ] AGRO: Deleted primary data file. [2025-11-04T17:06:52.040601] [INFO ] [MAIN ] AGRO: Starting full primary refresh. [2025-11-04T17:06:52.047934] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-11-04T17:06:52.167121] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-06. [2025-11-04T17:06:52.442686] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-11-04T17:06:52.447118] [INFO ] [REMOTE ] AGRO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:06:52.967017] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-11. [2025-11-04T17:06:53.061913] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T17:06:53.767047] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-07. [2025-11-04T17:06:54.210723] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T17:06:54.214573] [INFO ] [REMOTE ] AGI: Full fetch complete (77454 records). [2025-11-04T17:06:54.214888] [DEBUG] [VALIDATOR ] AGI: Validating primary data. [2025-11-04T17:06:54.573871] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-05. [2025-11-04T17:06:54.789472] [DEBUG] [VALIDATOR ] AGI: All expected trading days are present. [2025-11-04T17:06:54.789680] [DEBUG] [VALIDATOR ] AGI: Primary data validation passed. [2025-11-04T17:06:54.790612] [INFO ] [MAIN ] AGI: Fetched primary data is valid. [2025-11-04T17:06:54.790720] [DEBUG] [LOCAL ] AGI: Saving primary data. [2025-11-04T17:06:54.808537] [DEBUG] [LOCAL ] AGI: Trimmed 77454 records to 53426 (keeping 200 days of data) [2025-11-04T17:06:54.821066] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-11-04T17:06:55.211033] [DEBUG] [LOCAL ] AGI: Primary data saved (53426 records). [2025-11-04T17:06:55.211213] [INFO ] [MAIN ] AGI: Full history saved. [2025-11-04T17:06:55.211321] [INFO ] [MAIN ] Update Progress: 55/2367 symbols completed. AGI took 31.9s. Estimated remaining: 4h 48m 35s. Rate: 8.0 symbols/min [2025-11-04T17:06:55.370900] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-01. [2025-11-04T17:06:55.403739] [INFO ] [MAIN ] AGX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:06:55.406250] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-11-04T17:06:55.439503] [DEBUG] [LOCAL ] AGX: Loaded 31924 existing records from primary storage. [2025-11-04T17:06:55.439836] [INFO ] [MAIN ] AGX: Loaded 31924 existing records. [2025-11-04T17:06:55.623062] [WARN ] [MAIN ] AGX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:06:55.623267] [INFO ] [MAIN ] AGX: Wiping existing primary data before full refresh. [2025-11-04T17:06:55.623313] [INFO ] [LOCAL ] AGX: Starting deletion of primary data. [2025-11-04T17:06:55.623485] [DEBUG] [LOCAL ] AGX: Deleted primary data file. [2025-11-04T17:06:55.623522] [INFO ] [MAIN ] AGX: Starting full primary refresh. [2025-11-04T17:06:55.968275] [INFO ] [REMOTE ] AGX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:06:56.167065] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-06. [2025-11-04T17:06:56.624583] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T17:06:56.881868] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T17:06:56.967029] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-01. [2025-11-04T17:06:57.767039] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-08. [2025-11-04T17:06:58.336426] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-11-04T17:06:58.567043] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-07. [2025-11-04T17:06:58.872434] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T17:06:59.242294] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-11-04T17:06:59.367045] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-09. [2025-11-04T17:07:00.139692] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-11-04T17:07:00.167000] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-10. [2025-11-04T17:07:00.967033] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-08. [2025-11-04T17:07:01.298956] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-11-04T17:07:01.630625] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-11-04T17:07:01.767046] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-02. [2025-11-04T17:07:02.567040] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-11. [2025-11-04T17:07:02.833514] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T17:07:03.367016] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-02. [2025-11-04T17:07:03.422986] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T17:07:03.424415] [INFO ] [REMOTE ] AGO: Full fetch complete (39652 records). [2025-11-04T17:07:03.424490] [DEBUG] [VALIDATOR ] AGO: Validating primary data. [2025-11-04T17:07:03.810561] [DEBUG] [VALIDATOR ] AGO: All expected trading days are present. [2025-11-04T17:07:03.810737] [DEBUG] [VALIDATOR ] AGO: Primary data validation passed. [2025-11-04T17:07:03.811266] [INFO ] [MAIN ] AGO: Fetched primary data is valid. [2025-11-04T17:07:03.811330] [DEBUG] [LOCAL ] AGO: Saving primary data. [2025-11-04T17:07:03.812259] [DEBUG] [LOCAL ] AGO: Trimmed 39652 records to 27078 (keeping 200 days of data) [2025-11-04T17:07:03.972769] [DEBUG] [LOCAL ] AGO: Primary data saved (27078 records). [2025-11-04T17:07:03.972986] [INFO ] [MAIN ] AGO: Full history saved. [2025-11-04T17:07:03.973096] [INFO ] [MAIN ] Update Progress: 56/2367 symbols completed. AGO took 31.7s. Estimated remaining: 4h 49m 20s. Rate: 8.0 symbols/min [2025-11-04T17:07:04.140521] [INFO ] [MAIN ] AHH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:07:04.173488] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-03. [2025-11-04T17:07:04.176669] [DEBUG] [LOCAL ] AHH: Loaded 39477 existing records from primary storage. [2025-11-04T17:07:04.177041] [INFO ] [MAIN ] AHH: Loaded 39477 existing records. [2025-11-04T17:07:04.357937] [WARN ] [MAIN ] AHH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:07:04.358165] [INFO ] [MAIN ] AHH: Wiping existing primary data before full refresh. [2025-11-04T17:07:04.358239] [INFO ] [LOCAL ] AHH: Starting deletion of primary data. [2025-11-04T17:07:04.358812] [DEBUG] [LOCAL ] AHH: Deleted primary data file. [2025-11-04T17:07:04.358872] [INFO ] [MAIN ] AHH: Starting full primary refresh. [2025-11-04T17:07:04.728907] [INFO ] [REMOTE ] AHH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:07:04.747121] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-11-04T17:07:04.967034] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-01. [2025-11-04T17:07:05.158576] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T17:07:05.767047] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-04. [2025-11-04T17:07:06.437920] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T17:07:06.567000] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-02. [2025-11-04T17:07:07.135120] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-11-04T17:07:07.367044] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-03. [2025-11-04T17:07:07.785304] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T17:07:08.167037] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-09. [2025-11-04T17:07:08.684038] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T17:07:08.725332] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T17:07:08.967036] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-10. [2025-11-04T17:07:09.551604] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-11-04T17:07:09.767030] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-11. [2025-11-04T17:07:10.333732] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T17:07:10.334556] [INFO ] [REMOTE ] AGM: Full fetch complete (13304 records). [2025-11-04T17:07:10.334622] [DEBUG] [VALIDATOR ] AGM: Validating primary data. [2025-11-04T17:07:10.576033] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-03. [2025-11-04T17:07:10.698556] [DEBUG] [VALIDATOR ] AGM: All expected trading days are present. [2025-11-04T17:07:10.698633] [DEBUG] [VALIDATOR ] AGM: Primary data validation passed. [2025-11-04T17:07:10.698891] [INFO ] [MAIN ] AGM: Fetched primary data is valid. [2025-11-04T17:07:10.698937] [DEBUG] [LOCAL ] AGM: Saving primary data. [2025-11-04T17:07:10.699435] [DEBUG] [LOCAL ] AGM: Trimmed 13304 records to 10529 (keeping 200 days of data) [2025-11-04T17:07:10.766141] [DEBUG] [LOCAL ] AGM: Primary data saved (10529 records). [2025-11-04T17:07:10.766281] [INFO ] [MAIN ] AGM: Full history saved. [2025-11-04T17:07:10.766359] [INFO ] [MAIN ] Update Progress: 57/2367 symbols completed. AGM took 39.8s. Estimated remaining: 4h 48m 44s. Rate: 8.0 symbols/min [2025-11-04T17:07:10.933322] [INFO ] [MAIN ] AHL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:07:10.933488] [DEBUG] [LOCAL ] AHL: No existing data file found in primary storage. [2025-11-04T17:07:10.933525] [WARN ] [MAIN ] AHL: No existing data found, performing full refresh instead. [2025-11-04T17:07:10.933631] [INFO ] [MAIN ] AHL: Wiping existing primary data before full refresh. [2025-11-04T17:07:10.933707] [INFO ] [LOCAL ] AHL: Starting deletion of primary data. [2025-11-04T17:07:10.933757] [DEBUG] [LOCAL ] AHL: No primary data file found to delete. [2025-11-04T17:07:10.933790] [INFO ] [MAIN ] AHL: Starting full primary refresh. [2025-11-04T17:07:11.302988] [INFO ] [REMOTE ] AHL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:07:11.366997] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-01. [2025-11-04T17:07:11.524720] [DEBUG] [REMOTE ] AHL: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:07:11.525031] [WARN ] [REMOTE ] AHL: No data returned for month 2025-01 [2025-11-04T17:07:11.525118] [FATAL] [REMOTE ] AHL: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:07:11.525163] [FATAL] [MAIN ] AHL: Failed to fetch primary data. [2025-11-04T17:07:11.525220] [WARN ] [MAIN ] AHL: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:07:11.525291] [INFO ] [LOCAL ] AHL: Starting deletion of primary data. [2025-11-04T17:07:11.525352] [DEBUG] [LOCAL ] AHL: No primary data file found to delete. [2025-11-04T17:07:11.525386] [INFO ] [LOCAL ] AHL: Starting deletion of derivative data. [2025-11-04T17:07:11.525425] [DEBUG] [LOCAL ] AHL: No derivative data file found to delete. [2025-11-04T17:07:11.525469] [INFO ] [MAIN ] Update Progress: 58/2367 symbols completed. AHL took 0.8s. Estimated remaining: 4h 44m 8s. Rate: 8.1 symbols/min [2025-11-04T17:07:11.692001] [INFO ] [MAIN ] AHR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:07:11.735264] [DEBUG] [LOCAL ] AHR: Loaded 48889 existing records from primary storage. [2025-11-04T17:07:11.735561] [INFO ] [MAIN ] AHR: Loaded 48889 existing records. [2025-11-04T17:07:11.917071] [WARN ] [MAIN ] AHR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:07:11.917360] [INFO ] [MAIN ] AHR: Wiping existing primary data before full refresh. [2025-11-04T17:07:11.917441] [INFO ] [LOCAL ] AHR: Starting deletion of primary data. [2025-11-04T17:07:11.917868] [DEBUG] [LOCAL ] AHR: Deleted primary data file. [2025-11-04T17:07:11.918064] [INFO ] [MAIN ] AHR: Starting full primary refresh. [2025-11-04T17:07:12.128978] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T17:07:12.172245] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-05. [2025-11-04T17:07:12.321913] [INFO ] [REMOTE ] AHR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:07:12.967048] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-04. [2025-11-04T17:07:13.498560] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T17:07:13.767097] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-04. [2025-11-04T17:07:13.955815] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-11-04T17:07:14.567065] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-01. [2025-11-04T17:07:14.988055] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T17:07:15.367044] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-05. [2025-11-04T17:07:15.818797] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T17:07:16.167043] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-02. [2025-11-04T17:07:16.738225] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T17:07:16.967093] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-06. [2025-11-04T17:07:17.285446] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-11-04T17:07:17.767154] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-03. [2025-11-04T17:07:18.201080] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-11-04T17:07:18.567118] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-05. [2025-11-04T17:07:19.076789] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T17:07:19.367054] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-06. [2025-11-04T17:07:19.809835] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-11-04T17:07:20.167048] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-06. [2025-11-04T17:07:20.304203] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-11-04T17:07:20.967044] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-07. [2025-11-04T17:07:21.404233] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T17:07:21.767078] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-07. [2025-11-04T17:07:21.983825] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T17:07:22.567151] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-08. [2025-11-04T17:07:22.995474] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T17:07:23.367039] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-07. [2025-11-04T17:07:23.470070] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-11-04T17:07:24.167099] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-08. [2025-11-04T17:07:24.494252] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T17:07:24.967117] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-08. [2025-11-04T17:07:25.298824] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T17:07:25.767058] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-09. [2025-11-04T17:07:25.947585] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-11-04T17:07:26.567070] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-09. [2025-11-04T17:07:26.637584] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-11-04T17:07:27.367114] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-10. [2025-11-04T17:07:27.671758] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T17:07:28.167043] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-10. [2025-11-04T17:07:28.291673] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-11-04T17:07:28.967009] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-04. [2025-11-04T17:07:29.144226] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-11-04T17:07:29.767021] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-09. [2025-11-04T17:07:30.348080] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T17:07:30.567081] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-05. [2025-11-04T17:07:30.852031] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-11-04T17:07:31.367063] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-11. [2025-11-04T17:07:31.937984] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T17:07:32.167063] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-11. [2025-11-04T17:07:32.417975] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T17:07:32.419704] [INFO ] [REMOTE ] AGX: Full fetch complete (48094 records). [2025-11-04T17:07:32.419809] [DEBUG] [VALIDATOR ] AGX: Validating primary data. [2025-11-04T17:07:32.842825] [DEBUG] [VALIDATOR ] AGX: All expected trading days are present. [2025-11-04T17:07:32.842932] [DEBUG] [VALIDATOR ] AGX: Primary data validation passed. [2025-11-04T17:07:32.843516] [INFO ] [MAIN ] AGX: Fetched primary data is valid. [2025-11-04T17:07:32.843559] [DEBUG] [LOCAL ] AGX: Saving primary data. [2025-11-04T17:07:32.845013] [DEBUG] [LOCAL ] AGX: Trimmed 48094 records to 31754 (keeping 200 days of data) [2025-11-04T17:07:32.967081] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-10. [2025-11-04T17:07:33.068844] [DEBUG] [LOCAL ] AGX: Primary data saved (31754 records). [2025-11-04T17:07:33.069083] [INFO ] [MAIN ] AGX: Full history saved. [2025-11-04T17:07:33.069190] [INFO ] [MAIN ] Update Progress: 59/2367 symbols completed. AGX took 37.9s. Estimated remaining: 4h 53m 15s. Rate: 7.9 symbols/min [2025-11-04T17:07:33.146611] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-11 (0.98 seconds). [2025-11-04T17:07:33.167375] [INFO ] [REMOTE ] AGRO: Full fetch complete (48391 records). [2025-11-04T17:07:33.186456] [DEBUG] [VALIDATOR ] AGRO: Validating primary data. [2025-11-04T17:07:33.388634] [INFO ] [MAIN ] AHT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:07:33.489285] [DEBUG] [LOCAL ] AHT: Loaded 6501 existing records from primary storage. [2025-11-04T17:07:33.505557] [INFO ] [MAIN ] AHT: Loaded 6501 existing records. [2025-11-04T17:07:33.770545] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-06. [2025-11-04T17:07:33.925492] [WARN ] [MAIN ] AHT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:07:33.944638] [DEBUG] [VALIDATOR ] AGRO: All expected trading days are present. [2025-11-04T17:07:33.944444] [INFO ] [MAIN ] AHT: Wiping existing primary data before full refresh. [2025-11-04T17:07:33.944785] [DEBUG] [VALIDATOR ] AGRO: Primary data validation passed. [2025-11-04T17:07:33.945519] [INFO ] [MAIN ] AGRO: Fetched primary data is valid. [2025-11-04T17:07:33.945601] [DEBUG] [LOCAL ] AGRO: Saving primary data. [2025-11-04T17:07:33.945706] [INFO ] [LOCAL ] AHT: Starting deletion of primary data. [2025-11-04T17:07:33.946066] [DEBUG] [LOCAL ] AHT: Deleted primary data file. [2025-11-04T17:07:33.946128] [INFO ] [MAIN ] AHT: Starting full primary refresh. [2025-11-04T17:07:34.057229] [DEBUG] [LOCAL ] AGRO: Trimmed 48391 records to 31729 (keeping 200 days of data) [2025-11-04T17:07:34.506497] [DEBUG] [LOCAL ] AGRO: Primary data saved (31729 records). [2025-11-04T17:07:34.542145] [INFO ] [MAIN ] AGRO: Full history saved. [2025-11-04T17:07:34.550039] [INFO ] [MAIN ] Update Progress: 60/2367 symbols completed. AGRO took 42.9s. Estimated remaining: 4h 49m 11s. Rate: 8.0 symbols/min [2025-11-04T17:07:34.549120] [INFO ] [REMOTE ] AHT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:07:34.582143] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-01. [2025-11-04T17:07:34.681864] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-11-04T17:07:34.761100] [INFO ] [MAIN ] AI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:07:34.814074] [DEBUG] [LOCAL ] AI: Loaded 53765 existing records from primary storage. [2025-11-04T17:07:34.814413] [INFO ] [MAIN ] AI: Loaded 53765 existing records. [2025-11-04T17:07:35.002529] [WARN ] [MAIN ] AI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:07:35.002737] [INFO ] [MAIN ] AI: Wiping existing primary data before full refresh. [2025-11-04T17:07:35.002807] [INFO ] [LOCAL ] AI: Starting deletion of primary data. [2025-11-04T17:07:35.003035] [DEBUG] [LOCAL ] AI: Deleted primary data file. [2025-11-04T17:07:35.003072] [INFO ] [MAIN ] AI: Starting full primary refresh. [2025-11-04T17:07:35.288012] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T17:07:35.368995] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-11. [2025-11-04T17:07:35.368876] [INFO ] [REMOTE ] AI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:07:35.408624] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-11-04T17:07:36.167083] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-02. [2025-11-04T17:07:36.405014] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T17:07:36.406946] [INFO ] [REMOTE ] AHH: Full fetch complete (59252 records). [2025-11-04T17:07:36.407051] [DEBUG] [VALIDATOR ] AHH: Validating primary data. [2025-11-04T17:07:36.747419] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-11-04T17:07:36.821847] [DEBUG] [VALIDATOR ] AHH: All expected trading days are present. [2025-11-04T17:07:36.821917] [DEBUG] [VALIDATOR ] AHH: Primary data validation passed. [2025-11-04T17:07:36.822477] [INFO ] [MAIN ] AHH: Fetched primary data is valid. [2025-11-04T17:07:36.822515] [DEBUG] [LOCAL ] AHH: Saving primary data. [2025-11-04T17:07:36.823581] [DEBUG] [LOCAL ] AHH: Trimmed 59252 records to 39490 (keeping 200 days of data) [2025-11-04T17:07:36.979818] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-03. [2025-11-04T17:07:37.046179] [DEBUG] [LOCAL ] AHH: Primary data saved (39490 records). [2025-11-04T17:07:37.046289] [INFO ] [MAIN ] AHH: Full history saved. [2025-11-04T17:07:37.046362] [INFO ] [MAIN ] Update Progress: 61/2367 symbols completed. AHH took 33.1s. Estimated remaining: 4h 45m 53s. Rate: 8.1 symbols/min [2025-11-04T17:07:37.224923] [INFO ] [MAIN ] AIG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:07:37.272436] [DEBUG] [LOCAL ] AIG: Loaded 53448 existing records from primary storage. [2025-11-04T17:07:37.272742] [INFO ] [MAIN ] AIG: Loaded 53448 existing records. [2025-11-04T17:07:37.470733] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-11-04T17:07:37.476587] [WARN ] [MAIN ] AIG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:07:37.478740] [INFO ] [MAIN ] AIG: Wiping existing primary data before full refresh. [2025-11-04T17:07:37.478792] [INFO ] [LOCAL ] AIG: Starting deletion of primary data. [2025-11-04T17:07:37.478960] [DEBUG] [LOCAL ] AIG: Deleted primary data file. [2025-11-04T17:07:37.478998] [INFO ] [MAIN ] AIG: Starting full primary refresh. [2025-11-04T17:07:37.772195] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-04. [2025-11-04T17:07:37.867246] [INFO ] [REMOTE ] AIG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:07:38.359445] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-11-04T17:07:38.567075] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-01. [2025-11-04T17:07:39.367120] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-05. [2025-11-04T17:07:39.809365] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-11-04T17:07:39.900354] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T17:07:40.167056] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-01. [2025-11-04T17:07:40.967080] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-07. [2025-11-04T17:07:41.767098] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-02. [2025-11-04T17:07:42.289726] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-01 (2.12 seconds). [2025-11-04T17:07:42.338997] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T17:07:42.567135] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-08. [2025-11-04T17:07:42.917533] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T17:07:43.367106] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-02. [2025-11-04T17:07:43.699530] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T17:07:44.167050] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-03. [2025-11-04T17:07:44.967045] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-09. [2025-11-04T17:07:45.266180] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-11-04T17:07:45.485439] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T17:07:45.767046] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-03. [2025-11-04T17:07:46.225329] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T17:07:46.567141] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-04. [2025-11-04T17:07:47.367050] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-10. [2025-11-04T17:07:47.419458] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-11-04T17:07:48.103456] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-11-04T17:07:48.167008] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-06. [2025-11-04T17:07:48.617841] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T17:07:48.634988] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-11-04T17:07:48.967041] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-07. [2025-11-04T17:07:49.452330] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-11-04T17:07:49.767147] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-08. [2025-11-04T17:07:50.316536] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-11-04T17:07:50.567064] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-05. [2025-11-04T17:07:51.367089] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-04. [2025-11-04T17:07:51.896457] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T17:07:52.167083] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-09. [2025-11-04T17:07:52.629796] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-11-04T17:07:52.967065] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-06. [2025-11-04T17:07:53.163912] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-11-04T17:07:53.767100] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-10. [2025-11-04T17:07:54.280845] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-11-04T17:07:54.445807] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-11-04T17:07:54.567047] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-11. [2025-11-04T17:07:55.367034] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-05. [2025-11-04T17:07:55.882626] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-11 (1.32 seconds). [2025-11-04T17:07:55.885076] [INFO ] [REMOTE ] AHR: Full fetch complete (70954 records). [2025-11-04T17:07:55.885227] [DEBUG] [VALIDATOR ] AHR: Validating primary data. [2025-11-04T17:07:56.174265] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-07. [2025-11-04T17:07:56.270245] [DEBUG] [VALIDATOR ] AHR: All expected trading days are present. [2025-11-04T17:07:56.270327] [DEBUG] [VALIDATOR ] AHR: Primary data validation passed. [2025-11-04T17:07:56.271064] [INFO ] [MAIN ] AHR: Fetched primary data is valid. [2025-11-04T17:07:56.271127] [DEBUG] [LOCAL ] AHR: Saving primary data. [2025-11-04T17:07:56.272384] [DEBUG] [LOCAL ] AHR: Trimmed 70954 records to 48929 (keeping 200 days of data) [2025-11-04T17:07:56.561799] [DEBUG] [LOCAL ] AHR: Primary data saved (48929 records). [2025-11-04T17:07:56.561972] [INFO ] [MAIN ] AHR: Full history saved. [2025-11-04T17:07:56.563004] [INFO ] [MAIN ] Update Progress: 62/2367 symbols completed. AHR took 45.0s. Estimated remaining: 4h 53m 15s. Rate: 7.9 symbols/min [2025-11-04T17:07:56.730140] [INFO ] [MAIN ] AII: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:07:56.730315] [DEBUG] [LOCAL ] AII: No existing data file found in primary storage. [2025-11-04T17:07:56.730351] [WARN ] [MAIN ] AII: No existing data found, performing full refresh instead. [2025-11-04T17:07:56.730459] [INFO ] [MAIN ] AII: Wiping existing primary data before full refresh. [2025-11-04T17:07:56.730500] [INFO ] [LOCAL ] AII: Starting deletion of primary data. [2025-11-04T17:07:56.730540] [DEBUG] [LOCAL ] AII: No primary data file found to delete. [2025-11-04T17:07:56.730572] [INFO ] [MAIN ] AII: Starting full primary refresh. [2025-11-04T17:07:56.970756] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-11. [2025-11-04T17:07:57.130731] [INFO ] [REMOTE ] AII: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:07:57.170132] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-11-04T17:07:57.518140] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T17:07:57.521258] [INFO ] [REMOTE ] AHT: Full fetch complete (9114 records). [2025-11-04T17:07:57.521142] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T17:07:57.521397] [DEBUG] [VALIDATOR ] AHT: Validating primary data. [2025-11-04T17:07:57.776491] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-06. [2025-11-04T17:07:57.940528] [DEBUG] [VALIDATOR ] AHT: All expected trading days are present. [2025-11-04T17:07:57.940625] [DEBUG] [VALIDATOR ] AHT: Primary data validation passed. [2025-11-04T17:07:57.940937] [INFO ] [MAIN ] AHT: Fetched primary data is valid. [2025-11-04T17:07:57.940988] [DEBUG] [LOCAL ] AHT: Saving primary data. [2025-11-04T17:07:57.941403] [DEBUG] [LOCAL ] AHT: Trimmed 9114 records to 6540 (keeping 200 days of data) [2025-11-04T17:07:57.981988] [DEBUG] [LOCAL ] AHT: Primary data saved (6540 records). [2025-11-04T17:07:57.982113] [INFO ] [MAIN ] AHT: Full history saved. [2025-11-04T17:07:57.982199] [INFO ] [MAIN ] Update Progress: 63/2367 symbols completed. AHT took 24.9s. Estimated remaining: 4h 49m 20s. Rate: 8.0 symbols/min [2025-11-04T17:07:58.205693] [INFO ] [MAIN ] AIN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:07:58.226789] [DEBUG] [LOCAL ] AIN: Loaded 24197 existing records from primary storage. [2025-11-04T17:07:58.227070] [INFO ] [MAIN ] AIN: Loaded 24197 existing records. [2025-11-04T17:07:58.404865] [WARN ] [MAIN ] AIN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:07:58.405073] [INFO ] [MAIN ] AIN: Wiping existing primary data before full refresh. [2025-11-04T17:07:58.405122] [INFO ] [LOCAL ] AIN: Starting deletion of primary data. [2025-11-04T17:07:58.405474] [DEBUG] [LOCAL ] AIN: Deleted primary data file. [2025-11-04T17:07:58.405534] [INFO ] [MAIN ] AIN: Starting full primary refresh. [2025-11-04T17:07:58.586637] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-01. [2025-11-04T17:07:58.770991] [INFO ] [REMOTE ] AIN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:07:58.835385] [DEBUG] [REMOTE ] AII: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:07:58.835723] [WARN ] [REMOTE ] AII: No data returned for month 2025-01 [2025-11-04T17:07:58.835811] [FATAL] [REMOTE ] AII: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:07:58.835856] [FATAL] [MAIN ] AII: Failed to fetch primary data. [2025-11-04T17:07:58.835915] [WARN ] [MAIN ] AII: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:07:58.835986] [INFO ] [LOCAL ] AII: Starting deletion of primary data. [2025-11-04T17:07:58.836049] [DEBUG] [LOCAL ] AII: No primary data file found to delete. [2025-11-04T17:07:58.836085] [INFO ] [LOCAL ] AII: Starting deletion of derivative data. [2025-11-04T17:07:58.836123] [DEBUG] [LOCAL ] AII: No derivative data file found to delete. [2025-11-04T17:07:58.836168] [INFO ] [MAIN ] Update Progress: 64/2367 symbols completed. AII took 2.3s. Estimated remaining: 4h 45m 12s. Rate: 8.1 symbols/min [2025-11-04T17:07:59.003541] [INFO ] [MAIN ] AIO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:07:59.020024] [DEBUG] [LOCAL ] AIO: Loaded 12741 existing records from primary storage. [2025-11-04T17:07:59.020259] [INFO ] [MAIN ] AIO: Loaded 12741 existing records. [2025-11-04T17:07:59.196897] [WARN ] [MAIN ] AIO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:07:59.197133] [INFO ] [MAIN ] AIO: Wiping existing primary data before full refresh. [2025-11-04T17:07:59.197180] [INFO ] [LOCAL ] AIO: Starting deletion of primary data. [2025-11-04T17:07:59.197355] [DEBUG] [LOCAL ] AIO: Deleted primary data file. [2025-11-04T17:07:59.197410] [INFO ] [MAIN ] AIO: Starting full primary refresh. [2025-11-04T17:07:59.372241] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-01. [2025-11-04T17:07:59.564291] [INFO ] [REMOTE ] AIO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:07:59.793980] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-06 (2.02 seconds). [2025-11-04T17:08:00.167036] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-01. [2025-11-04T17:08:00.526745] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T17:08:00.967049] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-07. [2025-11-04T17:08:01.767092] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-08. [2025-11-04T17:08:02.567039] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-02. [2025-11-04T17:08:03.621938] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-11-04T17:08:03.623255] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T17:08:03.638197] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-03. [2025-11-04T17:08:04.167029] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-09. [2025-11-04T17:08:04.183001] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-01 (4.02 seconds). [2025-11-04T17:08:04.686083] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-11-04T17:08:04.967059] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-04. [2025-11-04T17:08:05.716357] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-07 (4.75 seconds). [2025-11-04T17:08:05.756737] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T17:08:05.767636] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-02. [2025-11-04T17:08:05.894799] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-11-04T17:08:06.394017] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-11-04T17:08:06.567031] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-05. [2025-11-04T17:08:07.262186] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-11-04T17:08:07.367067] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-08. [2025-11-04T17:08:08.167081] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-10. [2025-11-04T17:08:08.967053] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-03. [2025-11-04T17:08:09.589926] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-08 (2.22 seconds). [2025-11-04T17:08:09.612910] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-11-04T17:08:09.733851] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-11-04T17:08:09.767037] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-11. [2025-11-04T17:08:10.014283] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-11 (0.25 seconds). [2025-11-04T17:08:10.016334] [INFO ] [REMOTE ] AIG: Full fetch complete (77568 records). [2025-11-04T17:08:10.016470] [DEBUG] [VALIDATOR ] AIG: Validating primary data. [2025-11-04T17:08:10.400622] [DEBUG] [VALIDATOR ] AIG: All expected trading days are present. [2025-11-04T17:08:10.400732] [DEBUG] [VALIDATOR ] AIG: Primary data validation passed. [2025-11-04T17:08:10.401451] [INFO ] [MAIN ] AIG: Fetched primary data is valid. [2025-11-04T17:08:10.401490] [DEBUG] [LOCAL ] AIG: Saving primary data. [2025-11-04T17:08:10.402878] [DEBUG] [LOCAL ] AIG: Trimmed 77568 records to 53451 (keeping 200 days of data) [2025-11-04T17:08:10.577666] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-09. [2025-11-04T17:08:10.721698] [DEBUG] [LOCAL ] AIG: Primary data saved (53451 records). [2025-11-04T17:08:10.721823] [INFO ] [MAIN ] AIG: Full history saved. [2025-11-04T17:08:10.722918] [INFO ] [MAIN ] Update Progress: 65/2367 symbols completed. AIG took 33.7s. Estimated remaining: 4h 47m 42s. Rate: 8.0 symbols/min [2025-11-04T17:08:10.898425] [INFO ] [MAIN ] AIR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:08:10.926148] [DEBUG] [LOCAL ] AIR: Loaded 28450 existing records from primary storage. [2025-11-04T17:08:10.926397] [INFO ] [MAIN ] AIR: Loaded 28450 existing records. [2025-11-04T17:08:11.100760] [WARN ] [MAIN ] AIR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:08:11.100956] [INFO ] [MAIN ] AIR: Wiping existing primary data before full refresh. [2025-11-04T17:08:11.101002] [INFO ] [LOCAL ] AIR: Starting deletion of primary data. [2025-11-04T17:08:11.101205] [DEBUG] [LOCAL ] AIR: Deleted primary data file. [2025-11-04T17:08:11.101260] [INFO ] [MAIN ] AIR: Starting full primary refresh. [2025-11-04T17:08:11.373535] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-04. [2025-11-04T17:08:11.453740] [INFO ] [REMOTE ] AIR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:08:12.122173] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T17:08:12.167006] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-05. [2025-11-04T17:08:12.521882] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-11-04T17:08:12.711297] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-11-04T17:08:12.967123] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-06. [2025-11-04T17:08:13.767047] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-10. [2025-11-04T17:08:13.940255] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-11-04T17:08:14.567109] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-01. [2025-11-04T17:08:15.367108] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-06. [2025-11-04T17:08:15.908779] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T17:08:16.039300] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-10 (2.27 seconds). [2025-11-04T17:08:16.073953] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T17:08:16.167038] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-11. [2025-11-04T17:08:16.967072] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-07. [2025-11-04T17:08:17.636833] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T17:08:17.767118] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-02. [2025-11-04T17:08:18.044260] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-11 (1.88 seconds). [2025-11-04T17:08:18.045786] [INFO ] [REMOTE ] AI: Full fetch complete (77528 records). [2025-11-04T17:08:18.045888] [DEBUG] [VALIDATOR ] AI: Validating primary data. [2025-11-04T17:08:18.046614] [FATAL] [VALIDATOR ] AI: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:08:18.046717] [FATAL] [MAIN ] AI: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:08:18.046759] [WARN ] [MAIN ] AI: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:08:18.046796] [INFO ] [LOCAL ] AI: Starting deletion of primary data. [2025-11-04T17:08:18.046850] [DEBUG] [LOCAL ] AI: No primary data file found to delete. [2025-11-04T17:08:18.046918] [INFO ] [LOCAL ] AI: Starting deletion of derivative data. [2025-11-04T17:08:18.047231] [DEBUG] [LOCAL ] AI: Deleted derivative data file. [2025-11-04T17:08:18.047333] [INFO ] [MAIN ] Update Progress: 66/2367 symbols completed. AI took 43.5s. Estimated remaining: 4h 47m 29s. Rate: 8.0 symbols/min [2025-11-04T17:08:18.220907] [INFO ] [MAIN ] AIT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:08:18.246969] [DEBUG] [LOCAL ] AIT: Loaded 25978 existing records from primary storage. [2025-11-04T17:08:18.247273] [INFO ] [MAIN ] AIT: Loaded 25978 existing records. [2025-11-04T17:08:18.434934] [WARN ] [MAIN ] AIT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:08:18.435153] [INFO ] [MAIN ] AIT: Wiping existing primary data before full refresh. [2025-11-04T17:08:18.435205] [INFO ] [LOCAL ] AIT: Starting deletion of primary data. [2025-11-04T17:08:18.435390] [DEBUG] [LOCAL ] AIT: Deleted primary data file. [2025-11-04T17:08:18.435446] [INFO ] [MAIN ] AIT: Starting full primary refresh. [2025-11-04T17:08:18.593522] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-08. [2025-11-04T17:08:18.645794] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-11-04T17:08:18.816580] [INFO ] [REMOTE ] AIT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:08:19.367193] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T17:08:19.367295] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-07. [2025-11-04T17:08:20.167076] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-09. [2025-11-04T17:08:20.380699] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T17:08:20.710888] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T17:08:20.967023] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-03. [2025-11-04T17:08:21.767068] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-08. [2025-11-04T17:08:21.923915] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T17:08:22.567046] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-10. [2025-11-04T17:08:22.638264] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-11-04T17:08:23.162153] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-11-04T17:08:23.367027] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-01. [2025-11-04T17:08:24.167038] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-09. [2025-11-04T17:08:24.331550] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T17:08:24.967055] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-11. [2025-11-04T17:08:25.032579] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-11-04T17:08:25.564853] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-11 (0.60 seconds). [2025-11-04T17:08:25.566149] [INFO ] [REMOTE ] AIO: Full fetch complete (20118 records). [2025-11-04T17:08:25.566229] [DEBUG] [VALIDATOR ] AIO: Validating primary data. [2025-11-04T17:08:25.772241] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-04. [2025-11-04T17:08:25.944877] [DEBUG] [VALIDATOR ] AIO: All expected trading days are present. [2025-11-04T17:08:25.944970] [DEBUG] [VALIDATOR ] AIO: Primary data validation passed. [2025-11-04T17:08:25.945265] [INFO ] [MAIN ] AIO: Fetched primary data is valid. [2025-11-04T17:08:25.945306] [DEBUG] [LOCAL ] AIO: Saving primary data. [2025-11-04T17:08:25.946086] [DEBUG] [LOCAL ] AIO: Trimmed 20118 records to 12723 (keeping 200 days of data) [2025-11-04T17:08:26.020745] [DEBUG] [LOCAL ] AIO: Primary data saved (12723 records). [2025-11-04T17:08:26.020879] [INFO ] [MAIN ] AIO: Full history saved. [2025-11-04T17:08:26.020954] [INFO ] [MAIN ] Update Progress: 67/2367 symbols completed. AIO took 27.2s. Estimated remaining: 4h 47m 38s. Rate: 8.0 symbols/min [2025-11-04T17:08:26.189194] [INFO ] [MAIN ] AIV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:08:26.224151] [DEBUG] [LOCAL ] AIV: Loaded 41968 existing records from primary storage. [2025-11-04T17:08:26.224423] [INFO ] [MAIN ] AIV: Loaded 41968 existing records. [2025-11-04T17:08:26.400437] [WARN ] [MAIN ] AIV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:08:26.400684] [INFO ] [MAIN ] AIV: Wiping existing primary data before full refresh. [2025-11-04T17:08:26.400742] [INFO ] [LOCAL ] AIV: Starting deletion of primary data. [2025-11-04T17:08:26.401499] [DEBUG] [LOCAL ] AIV: Deleted primary data file. [2025-11-04T17:08:26.401555] [INFO ] [MAIN ] AIV: Starting full primary refresh. [2025-11-04T17:08:26.571082] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-02. [2025-11-04T17:08:26.750940] [INFO ] [REMOTE ] AIV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:08:27.367026] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-10. [2025-11-04T17:08:27.464433] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-11-04T17:08:27.575908] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T17:08:28.167076] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-05. [2025-11-04T17:08:28.203679] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-11-04T17:08:28.943587] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-11-04T17:08:28.966999] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-06. [2025-11-04T17:08:29.767063] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-11. [2025-11-04T17:08:29.780967] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T17:08:30.494478] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T17:08:30.495745] [INFO ] [REMOTE ] AIN: Full fetch complete (34188 records). [2025-11-04T17:08:30.495820] [DEBUG] [VALIDATOR ] AIN: Validating primary data. [2025-11-04T17:08:30.572138] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-03. [2025-11-04T17:08:30.859586] [DEBUG] [VALIDATOR ] AIN: All expected trading days are present. [2025-11-04T17:08:30.859690] [DEBUG] [VALIDATOR ] AIN: Primary data validation passed. [2025-11-04T17:08:30.860098] [INFO ] [MAIN ] AIN: Fetched primary data is valid. [2025-11-04T17:08:30.860139] [DEBUG] [LOCAL ] AIN: Saving primary data. [2025-11-04T17:08:30.860923] [DEBUG] [LOCAL ] AIN: Trimmed 34188 records to 24208 (keeping 200 days of data) [2025-11-04T17:08:31.001053] [DEBUG] [LOCAL ] AIN: Primary data saved (24208 records). [2025-11-04T17:08:31.001179] [INFO ] [MAIN ] AIN: Full history saved. [2025-11-04T17:08:31.001252] [INFO ] [MAIN ] Update Progress: 68/2367 symbols completed. AIN took 33.0s. Estimated remaining: 4h 46m 5s. Rate: 8.0 symbols/min [2025-11-04T17:08:31.176861] [INFO ] [MAIN ] AIZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:08:31.206994] [DEBUG] [LOCAL ] AIZ: Loaded 28599 existing records from primary storage. [2025-11-04T17:08:31.207315] [INFO ] [MAIN ] AIZ: Loaded 28599 existing records. [2025-11-04T17:08:31.369569] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-07. [2025-11-04T17:08:31.393359] [WARN ] [MAIN ] AIZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:08:31.393727] [INFO ] [MAIN ] AIZ: Wiping existing primary data before full refresh. [2025-11-04T17:08:31.393808] [INFO ] [LOCAL ] AIZ: Starting deletion of primary data. [2025-11-04T17:08:31.394387] [DEBUG] [LOCAL ] AIZ: Deleted primary data file. [2025-11-04T17:08:31.394446] [INFO ] [MAIN ] AIZ: Starting full primary refresh. [2025-11-04T17:08:31.757566] [INFO ] [REMOTE ] AIZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:08:31.777140] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T17:08:32.167046] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-04. [2025-11-04T17:08:32.475919] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T17:08:32.967070] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-01. [2025-11-04T17:08:33.103106] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-11-04T17:08:33.767102] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-01. [2025-11-04T17:08:33.914917] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T17:08:34.567160] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-02. [2025-11-04T17:08:35.086062] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T17:08:35.367051] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-02. [2025-11-04T17:08:35.472060] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T17:08:36.167052] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-08. [2025-11-04T17:08:36.453853] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T17:08:36.967063] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-03. [2025-11-04T17:08:37.244036] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-11-04T17:08:37.767075] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-03. [2025-11-04T17:08:38.062016] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T17:08:38.567042] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-04. [2025-11-04T17:08:38.918174] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T17:08:39.367048] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-05. [2025-11-04T17:08:39.944769] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T17:08:40.167048] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-04. [2025-11-04T17:08:40.563251] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T17:08:40.967013] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-06. [2025-11-04T17:08:41.367029] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T17:08:41.767044] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-05. [2025-11-04T17:08:41.861815] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T17:08:42.567043] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-09. [2025-11-04T17:08:42.915983] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T17:08:43.367078] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-07. [2025-11-04T17:08:43.704586] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T17:08:44.167044] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-10. [2025-11-04T17:08:44.363961] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T17:08:44.967033] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-06. [2025-11-04T17:08:45.196360] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-11-04T17:08:45.767045] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-05. [2025-11-04T17:08:45.962838] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T17:08:46.567080] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-07. [2025-11-04T17:08:47.135534] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T17:08:47.367056] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-06. [2025-11-04T17:08:47.609326] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T17:08:48.167051] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-08. [2025-11-04T17:08:48.422373] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-11-04T17:08:48.967099] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-11. [2025-11-04T17:08:49.767086] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-08. [2025-11-04T17:08:49.898499] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T17:08:49.899863] [INFO ] [REMOTE ] AIR: Full fetch complete (40157 records). [2025-11-04T17:08:49.899933] [DEBUG] [VALIDATOR ] AIR: Validating primary data. [2025-11-04T17:08:50.072809] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-11-04T17:08:50.301457] [DEBUG] [VALIDATOR ] AIR: All expected trading days are present. [2025-11-04T17:08:50.301531] [DEBUG] [VALIDATOR ] AIR: Primary data validation passed. [2025-11-04T17:08:50.301966] [INFO ] [MAIN ] AIR: Fetched primary data is valid. [2025-11-04T17:08:50.302039] [DEBUG] [LOCAL ] AIR: Saving primary data. [2025-11-04T17:08:50.302858] [DEBUG] [LOCAL ] AIR: Trimmed 40157 records to 28450 (keeping 200 days of data) [2025-11-04T17:08:50.472736] [DEBUG] [LOCAL ] AIR: Primary data saved (28450 records). [2025-11-04T17:08:50.472898] [INFO ] [MAIN ] AIR: Full history saved. [2025-11-04T17:08:50.472991] [INFO ] [MAIN ] Update Progress: 69/2367 symbols completed. AIR took 39.7s. Estimated remaining: 4h 52m 37s. Rate: 7.9 symbols/min [2025-11-04T17:08:50.572253] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-09. [2025-11-04T17:08:50.645896] [INFO ] [MAIN ] AIZN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:08:50.652486] [DEBUG] [LOCAL ] AIZN: Loaded 3605 existing records from primary storage. [2025-11-04T17:08:50.652688] [INFO ] [MAIN ] AIZN: Loaded 3605 existing records. [2025-11-04T17:08:50.840949] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T17:08:50.844660] [WARN ] [MAIN ] AIZN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:08:50.850002] [INFO ] [MAIN ] AIZN: Wiping existing primary data before full refresh. [2025-11-04T17:08:50.850165] [INFO ] [LOCAL ] AIZN: Starting deletion of primary data. [2025-11-04T17:08:50.850539] [DEBUG] [LOCAL ] AIZN: Deleted primary data file. [2025-11-04T17:08:50.850600] [INFO ] [MAIN ] AIZN: Starting full primary refresh. [2025-11-04T17:08:51.203973] [INFO ] [REMOTE ] AIZN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:08:51.367006] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-01. [2025-11-04T17:08:51.488573] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T17:08:51.897706] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T17:08:52.167047] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-10. [2025-11-04T17:08:52.967066] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-02. [2025-11-04T17:08:53.200981] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-11-04T17:08:53.347790] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T17:08:53.767070] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-07. [2025-11-04T17:08:54.567046] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-03. [2025-11-04T17:08:54.755986] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-11-04T17:08:54.938710] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-11-04T17:08:55.367037] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-08. [2025-11-04T17:08:56.167085] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-04. [2025-11-04T17:08:56.536541] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-11-04T17:08:56.781695] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-11-04T17:08:56.967055] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-11. [2025-11-04T17:08:57.767074] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-05. [2025-11-04T17:08:58.151949] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-11-04T17:08:58.363612] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-11 (1.40 seconds). [2025-11-04T17:08:58.365076] [INFO ] [REMOTE ] AIZ: Full fetch complete (42719 records). [2025-11-04T17:08:58.365191] [DEBUG] [VALIDATOR ] AIZ: Validating primary data. [2025-11-04T17:08:58.573042] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-09. [2025-11-04T17:08:58.767626] [DEBUG] [VALIDATOR ] AIZ: All expected trading days are present. [2025-11-04T17:08:58.767759] [DEBUG] [VALIDATOR ] AIZ: Primary data validation passed. [2025-11-04T17:08:58.768205] [INFO ] [MAIN ] AIZ: Fetched primary data is valid. [2025-11-04T17:08:58.768246] [DEBUG] [LOCAL ] AIZ: Saving primary data. [2025-11-04T17:08:58.769118] [DEBUG] [LOCAL ] AIZ: Trimmed 42719 records to 28608 (keeping 200 days of data) [2025-11-04T17:08:58.945177] [DEBUG] [LOCAL ] AIZ: Primary data saved (28608 records). [2025-11-04T17:08:58.945310] [INFO ] [MAIN ] AIZ: Full history saved. [2025-11-04T17:08:58.945387] [INFO ] [MAIN ] Update Progress: 70/2367 symbols completed. AIZ took 27.9s. Estimated remaining: 4h 52m 57s. Rate: 7.8 symbols/min [2025-11-04T17:08:59.118998] [INFO ] [MAIN ] AJG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:08:59.161437] [DEBUG] [LOCAL ] AJG: Loaded 46324 existing records from primary storage. [2025-11-04T17:08:59.161762] [INFO ] [MAIN ] AJG: Loaded 46324 existing records. [2025-11-04T17:08:59.346977] [WARN ] [MAIN ] AJG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:08:59.347406] [INFO ] [MAIN ] AJG: Wiping existing primary data before full refresh. [2025-11-04T17:08:59.347548] [INFO ] [LOCAL ] AJG: Starting deletion of primary data. [2025-11-04T17:08:59.347796] [DEBUG] [LOCAL ] AJG: Deleted primary data file. [2025-11-04T17:08:59.347854] [INFO ] [MAIN ] AJG: Starting full primary refresh. [2025-11-04T17:08:59.376630] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-09. [2025-11-04T17:08:59.583499] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T17:08:59.739382] [INFO ] [REMOTE ] AJG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:09:00.167034] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-06. [2025-11-04T17:09:00.620894] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-11-04T17:09:00.967048] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-07. [2025-11-04T17:09:01.043437] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-11-04T17:09:01.343024] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-11-04T17:09:01.767040] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-08. [2025-11-04T17:09:02.301516] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-11-04T17:09:02.567088] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-09. [2025-11-04T17:09:02.990572] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-11-04T17:09:03.367055] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-01. [2025-11-04T17:09:04.167045] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-10. [2025-11-04T17:09:04.570663] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-11-04T17:09:04.907621] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-11-04T17:09:04.967001] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-02. [2025-11-04T17:09:05.767019] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-10. [2025-11-04T17:09:06.322237] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T17:09:06.567016] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-11. [2025-11-04T17:09:06.993575] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T17:09:06.994260] [INFO ] [REMOTE ] AIZN: Full fetch complete (5442 records). [2025-11-04T17:09:06.994328] [DEBUG] [VALIDATOR ] AIZN: Validating primary data. [2025-11-04T17:09:07.374579] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-03. [2025-11-04T17:09:07.382506] [DEBUG] [VALIDATOR ] AIZN: All expected trading days are present. [2025-11-04T17:09:07.382683] [DEBUG] [VALIDATOR ] AIZN: Primary data validation passed. [2025-11-04T17:09:07.382886] [INFO ] [MAIN ] AIZN: Fetched primary data is valid. [2025-11-04T17:09:07.383007] [DEBUG] [LOCAL ] AIZN: Saving primary data. [2025-11-04T17:09:07.383771] [DEBUG] [LOCAL ] AIZN: Trimmed 5442 records to 3611 (keeping 200 days of data) [2025-11-04T17:09:07.434548] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-11-04T17:09:07.435554] [DEBUG] [LOCAL ] AIZN: Primary data saved (3611 records). [2025-11-04T17:09:07.446045] [INFO ] [MAIN ] AIZN: Full history saved. [2025-11-04T17:09:07.446376] [INFO ] [MAIN ] Update Progress: 71/2367 symbols completed. AIZN took 17.0s. Estimated remaining: 4h 53m 17s. Rate: 7.8 symbols/min [2025-11-04T17:09:07.614381] [INFO ] [MAIN ] AKA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:09:07.614590] [DEBUG] [LOCAL ] AKA: No existing data file found in primary storage. [2025-11-04T17:09:07.614634] [WARN ] [MAIN ] AKA: No existing data found, performing full refresh instead. [2025-11-04T17:09:07.614793] [INFO ] [MAIN ] AKA: Wiping existing primary data before full refresh. [2025-11-04T17:09:07.614838] [INFO ] [LOCAL ] AKA: Starting deletion of primary data. [2025-11-04T17:09:07.614880] [DEBUG] [LOCAL ] AKA: No primary data file found to delete. [2025-11-04T17:09:07.614914] [INFO ] [MAIN ] AKA: Starting full primary refresh. [2025-11-04T17:09:07.964703] [INFO ] [REMOTE ] AKA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:09:08.167046] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-01. [2025-11-04T17:09:08.590030] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-11-04T17:09:08.754267] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T17:09:08.967064] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-04. [2025-11-04T17:09:09.767035] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-02. [2025-11-04T17:09:10.123380] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-11-04T17:09:10.498308] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-11-04T17:09:10.567016] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-05. [2025-11-04T17:09:11.367047] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-10. [2025-11-04T17:09:11.943606] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-11-04T17:09:12.167104] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-06. [2025-11-04T17:09:12.386186] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T17:09:12.967106] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-11. [2025-11-04T17:09:13.443707] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T17:09:13.767051] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-07. [2025-11-04T17:09:13.774109] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T17:09:13.775611] [INFO ] [REMOTE ] AIT: Full fetch complete (39008 records). [2025-11-04T17:09:13.775713] [DEBUG] [VALIDATOR ] AIT: Validating primary data. [2025-11-04T17:09:14.159959] [DEBUG] [VALIDATOR ] AIT: All expected trading days are present. [2025-11-04T17:09:14.160040] [DEBUG] [VALIDATOR ] AIT: Primary data validation passed. [2025-11-04T17:09:14.160516] [INFO ] [MAIN ] AIT: Fetched primary data is valid. [2025-11-04T17:09:14.160609] [DEBUG] [LOCAL ] AIT: Saving primary data. [2025-11-04T17:09:14.161483] [DEBUG] [LOCAL ] AIT: Trimmed 39008 records to 25994 (keeping 200 days of data) [2025-11-04T17:09:14.315439] [DEBUG] [LOCAL ] AIT: Primary data saved (25994 records). [2025-11-04T17:09:14.315563] [INFO ] [MAIN ] AIT: Full history saved. [2025-11-04T17:09:14.315636] [INFO ] [MAIN ] Update Progress: 72/2367 symbols completed. AIT took 56.3s. Estimated remaining: 4h 52m 44s. Rate: 7.8 symbols/min [2025-11-04T17:09:14.493167] [INFO ] [MAIN ] AKR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:09:14.532294] [DEBUG] [LOCAL ] AKR: Loaded 46136 existing records from primary storage. [2025-11-04T17:09:14.532574] [INFO ] [MAIN ] AKR: Loaded 46136 existing records. [2025-11-04T17:09:14.576765] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-03. [2025-11-04T17:09:14.737988] [WARN ] [MAIN ] AKR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:09:14.738188] [INFO ] [MAIN ] AKR: Wiping existing primary data before full refresh. [2025-11-04T17:09:14.738237] [INFO ] [LOCAL ] AKR: Starting deletion of primary data. [2025-11-04T17:09:14.738574] [DEBUG] [LOCAL ] AKR: Deleted primary data file. [2025-11-04T17:09:14.738616] [INFO ] [MAIN ] AKR: Starting full primary refresh. [2025-11-04T17:09:15.001043] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T17:09:15.103251] [INFO ] [REMOTE ] AKR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:09:15.210085] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:09:15.367082] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-01. [2025-11-04T17:09:16.167063] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-04. [2025-11-04T17:09:16.480270] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-11-04T17:09:16.701950] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T17:09:16.967089] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-05. [2025-11-04T17:09:17.365000] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-11-04T17:09:17.767119] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-06. [2025-11-04T17:09:18.055323] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-06 (0.29 seconds). [2025-11-04T17:09:18.567007] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-02. [2025-11-04T17:09:19.367085] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-08. [2025-11-04T17:09:19.679137] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T17:09:20.167046] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-11. [2025-11-04T17:09:20.683895] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-11-04T17:09:20.967039] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-03. [2025-11-04T17:09:21.321203] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T17:09:21.322920] [INFO ] [REMOTE ] AIV: Full fetch complete (60086 records). [2025-11-04T17:09:21.323027] [DEBUG] [VALIDATOR ] AIV: Validating primary data. [2025-11-04T17:09:21.323824] [FATAL] [VALIDATOR ] AIV: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:09:21.323920] [FATAL] [MAIN ] AIV: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:09:21.323986] [WARN ] [MAIN ] AIV: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:09:21.324058] [INFO ] [LOCAL ] AIV: Starting deletion of primary data. [2025-11-04T17:09:21.324121] [DEBUG] [LOCAL ] AIV: No primary data file found to delete. [2025-11-04T17:09:21.324159] [INFO ] [LOCAL ] AIV: Starting deletion of derivative data. [2025-11-04T17:09:21.324322] [DEBUG] [LOCAL ] AIV: Deleted derivative data file. [2025-11-04T17:09:21.324832] [INFO ] [MAIN ] Update Progress: 73/2367 symbols completed. AIV took 55.3s. Estimated remaining: 4h 52m 16s. Rate: 7.8 symbols/min [2025-11-04T17:09:21.503699] [INFO ] [MAIN ] AL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:09:21.544803] [DEBUG] [LOCAL ] AL: Loaded 44722 existing records from primary storage. [2025-11-04T17:09:21.545090] [INFO ] [MAIN ] AL: Loaded 44722 existing records. [2025-11-04T17:09:21.716168] [WARN ] [MAIN ] AL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:09:21.716388] [INFO ] [MAIN ] AL: Wiping existing primary data before full refresh. [2025-11-04T17:09:21.716433] [INFO ] [LOCAL ] AL: Starting deletion of primary data. [2025-11-04T17:09:21.717093] [DEBUG] [LOCAL ] AL: Deleted primary data file. [2025-11-04T17:09:21.717150] [INFO ] [MAIN ] AL: Starting full primary refresh. [2025-11-04T17:09:21.778767] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-09. [2025-11-04T17:09:22.065625] [INFO ] [REMOTE ] AL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:09:22.210493] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T17:09:22.567092] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-07. [2025-11-04T17:09:22.916044] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-11-04T17:09:23.029102] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T17:09:23.367025] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-10. [2025-11-04T17:09:24.167068] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-08. [2025-11-04T17:09:24.414518] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-08 (0.25 seconds). [2025-11-04T17:09:24.819349] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T17:09:24.967025] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-11. [2025-11-04T17:09:25.767052] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-09. [2025-11-04T17:09:25.956517] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-09 (0.19 seconds). [2025-11-04T17:09:26.117431] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T17:09:26.119396] [INFO ] [REMOTE ] AJG: Full fetch complete (68796 records). [2025-11-04T17:09:26.119479] [DEBUG] [VALIDATOR ] AJG: Validating primary data. [2025-11-04T17:09:26.502081] [DEBUG] [VALIDATOR ] AJG: All expected trading days are present. [2025-11-04T17:09:26.502153] [DEBUG] [VALIDATOR ] AJG: Primary data validation passed. [2025-11-04T17:09:26.502807] [INFO ] [MAIN ] AJG: Fetched primary data is valid. [2025-11-04T17:09:26.502868] [DEBUG] [LOCAL ] AJG: Saving primary data. [2025-11-04T17:09:26.504083] [DEBUG] [LOCAL ] AJG: Trimmed 68796 records to 46367 (keeping 200 days of data) [2025-11-04T17:09:26.592123] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-01. [2025-11-04T17:09:26.785858] [DEBUG] [LOCAL ] AJG: Primary data saved (46367 records). [2025-11-04T17:09:26.785980] [INFO ] [MAIN ] AJG: Full history saved. [2025-11-04T17:09:26.786055] [INFO ] [MAIN ] Update Progress: 74/2367 symbols completed. AJG took 27.8s. Estimated remaining: 4h 51m 1s. Rate: 7.9 symbols/min [2025-11-04T17:09:26.983000] [INFO ] [MAIN ] ALB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:09:27.029453] [DEBUG] [LOCAL ] ALB: Loaded 53339 existing records from primary storage. [2025-11-04T17:09:27.029781] [INFO ] [MAIN ] ALB: Loaded 53339 existing records. [2025-11-04T17:09:27.207181] [WARN ] [MAIN ] ALB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:09:27.207391] [INFO ] [MAIN ] ALB: Wiping existing primary data before full refresh. [2025-11-04T17:09:27.207439] [INFO ] [LOCAL ] ALB: Starting deletion of primary data. [2025-11-04T17:09:27.208354] [DEBUG] [LOCAL ] ALB: Deleted primary data file. [2025-11-04T17:09:27.208412] [INFO ] [MAIN ] ALB: Starting full primary refresh. [2025-11-04T17:09:27.380256] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-04. [2025-11-04T17:09:27.565091] [INFO ] [REMOTE ] ALB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:09:27.786762] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T17:09:28.167018] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-02. [2025-11-04T17:09:28.783392] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T17:09:28.967087] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-10. [2025-11-04T17:09:29.397125] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T17:09:29.468215] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-11-04T17:09:29.767106] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-01. [2025-11-04T17:09:30.567039] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-05. [2025-11-04T17:09:31.367065] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-03. [2025-11-04T17:09:31.526957] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-11-04T17:09:31.842631] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T17:09:32.167076] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-06. [2025-11-04T17:09:32.694816] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T17:09:32.967076] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-02. [2025-11-04T17:09:33.329101] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T17:09:33.767064] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-04. [2025-11-04T17:09:34.236204] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T17:09:34.567050] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-11. [2025-11-04T17:09:35.114735] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T17:09:35.224675] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T17:09:35.225340] [INFO ] [REMOTE ] AKA: Full fetch complete (2790 records). [2025-11-04T17:09:35.225410] [DEBUG] [VALIDATOR ] AKA: Validating primary data. [2025-11-04T17:09:35.372146] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-05. [2025-11-04T17:09:35.436622] [WARN ] [VALIDATOR ] AKA: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-11-04T17:09:35.436796] [FATAL] [MAIN ] AKA: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:09:35.436898] [WARN ] [MAIN ] AKA: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:09:35.436973] [INFO ] [LOCAL ] AKA: Starting deletion of primary data. [2025-11-04T17:09:35.437035] [DEBUG] [LOCAL ] AKA: No primary data file found to delete. [2025-11-04T17:09:35.437072] [INFO ] [LOCAL ] AKA: Starting deletion of derivative data. [2025-11-04T17:09:35.437112] [DEBUG] [LOCAL ] AKA: No derivative data file found to delete. [2025-11-04T17:09:35.437167] [INFO ] [MAIN ] Update Progress: 75/2367 symbols completed. AKA took 28.0s. Estimated remaining: 4h 51m 25s. Rate: 7.9 symbols/min [2025-11-04T17:09:35.626256] [INFO ] [MAIN ] ALC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:09:35.678021] [DEBUG] [LOCAL ] ALC: Loaded 51117 existing records from primary storage. [2025-11-04T17:09:35.678331] [INFO ] [MAIN ] ALC: Loaded 51117 existing records. [2025-11-04T17:09:35.881824] [WARN ] [MAIN ] ALC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:09:35.882065] [INFO ] [MAIN ] ALC: Wiping existing primary data before full refresh. [2025-11-04T17:09:35.882134] [INFO ] [LOCAL ] ALC: Starting deletion of primary data. [2025-11-04T17:09:35.882348] [DEBUG] [LOCAL ] ALC: Deleted primary data file. [2025-11-04T17:09:35.882401] [INFO ] [MAIN ] ALC: Starting full primary refresh. [2025-11-04T17:09:36.168847] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-03. [2025-11-04T17:09:36.248168] [INFO ] [REMOTE ] ALC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:09:36.838993] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-11-04T17:09:36.967015] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-01. [2025-11-04T17:09:37.708175] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-11-04T17:09:37.767025] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-04. [2025-11-04T17:09:38.316551] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T17:09:38.567037] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-02. [2025-11-04T17:09:39.332441] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-11-04T17:09:39.367005] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-06. [2025-11-04T17:09:39.698501] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T17:09:40.167133] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-03. [2025-11-04T17:09:40.636684] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T17:09:40.967038] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-07. [2025-11-04T17:09:41.636775] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-11-04T17:09:41.767033] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-05. [2025-11-04T17:09:42.408273] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:09:42.567131] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-04. [2025-11-04T17:09:43.367016] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-07. [2025-11-04T17:09:43.499862] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-11-04T17:09:43.985690] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T17:09:44.167021] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-05. [2025-11-04T17:09:44.535325] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T17:09:44.967032] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-08. [2025-11-04T17:09:45.497113] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T17:09:45.767017] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-06. [2025-11-04T17:09:46.023686] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-11-04T17:09:46.567074] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-06. [2025-11-04T17:09:47.367056] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-09. [2025-11-04T17:09:47.471427] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-11-04T17:09:47.875745] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T17:09:48.167074] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-08. [2025-11-04T17:09:48.685127] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-11-04T17:09:48.967125] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-07. [2025-11-04T17:09:49.445445] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-11-04T17:09:49.767079] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-09. [2025-11-04T17:09:50.550216] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-11-04T17:09:50.566993] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-08. [2025-11-04T17:09:51.367101] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-10. [2025-11-04T17:09:52.167083] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-07. [2025-11-04T17:09:52.802984] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T17:09:52.967007] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-11. [2025-11-04T17:09:54.171462] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-11-04T17:09:54.183327] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-08. [2025-11-04T17:09:55.950710] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-11-04T17:09:55.965962] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-09. [2025-11-04T17:09:57.764589] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-11-04T17:09:57.777711] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-10. [2025-11-04T17:09:59.734476] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-11-04T17:09:59.748984] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-11. [2025-11-04T17:10:00.087978] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T17:10:00.090122] [INFO ] [REMOTE ] ALB: Full fetch complete (77152 records). [2025-11-04T17:10:00.090217] [DEBUG] [VALIDATOR ] ALB: Validating primary data. [2025-11-04T17:10:00.559980] [DEBUG] [VALIDATOR ] ALB: All expected trading days are present. [2025-11-04T17:10:00.560062] [DEBUG] [VALIDATOR ] ALB: Primary data validation passed. [2025-11-04T17:10:00.560791] [INFO ] [MAIN ] ALB: Fetched primary data is valid. [2025-11-04T17:10:00.560854] [DEBUG] [LOCAL ] ALB: Saving primary data. [2025-11-04T17:10:00.562247] [DEBUG] [LOCAL ] ALB: Trimmed 77152 records to 53339 (keeping 200 days of data) [2025-11-04T17:10:00.875677] [DEBUG] [LOCAL ] ALB: Primary data saved (53339 records). [2025-11-04T17:10:00.875846] [INFO ] [MAIN ] ALB: Full history saved. [2025-11-04T17:10:00.875940] [INFO ] [MAIN ] Update Progress: 76/2367 symbols completed. ALB took 34.1s. Estimated remaining: 5h 0m 14s. Rate: 7.6 symbols/min [2025-11-04T17:10:01.043091] [INFO ] [MAIN ] ALE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:01.073156] [DEBUG] [LOCAL ] ALE: Loaded 31960 existing records from primary storage. [2025-11-04T17:10:01.073418] [INFO ] [MAIN ] ALE: Loaded 31960 existing records. [2025-11-04T17:10:01.249232] [WARN ] [MAIN ] ALE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:10:01.249478] [INFO ] [MAIN ] ALE: Wiping existing primary data before full refresh. [2025-11-04T17:10:01.249524] [INFO ] [LOCAL ] ALE: Starting deletion of primary data. [2025-11-04T17:10:01.249748] [DEBUG] [LOCAL ] ALE: Deleted primary data file. [2025-11-04T17:10:01.249803] [INFO ] [MAIN ] ALE: Starting full primary refresh. [2025-11-04T17:10:01.597565] [INFO ] [REMOTE ] ALE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:01.597892] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-01. [2025-11-04T17:10:02.459078] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T17:10:02.467695] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-02. [2025-11-04T17:10:03.346284] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-11-04T17:10:03.351478] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-03. [2025-11-04T17:10:03.636014] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-09 (13.87 seconds). [2025-11-04T17:10:03.645833] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-10. [2025-11-04T17:10:04.252536] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-11-04T17:10:04.258398] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-04. [2025-11-04T17:10:05.152235] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T17:10:05.164063] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-11. [2025-11-04T17:10:05.472957] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T17:10:05.479697] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-05. [2025-11-04T17:10:06.488028] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-11-04T17:10:06.496161] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-06. [2025-11-04T17:10:06.545407] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-11 (1.38 seconds). [2025-11-04T17:10:06.547476] [INFO ] [REMOTE ] AKR: Full fetch complete (65776 records). [2025-11-04T17:10:06.547575] [DEBUG] [VALIDATOR ] AKR: Validating primary data. [2025-11-04T17:10:06.958072] [DEBUG] [VALIDATOR ] AKR: All expected trading days are present. [2025-11-04T17:10:06.958163] [DEBUG] [VALIDATOR ] AKR: Primary data validation passed. [2025-11-04T17:10:06.958798] [INFO ] [MAIN ] AKR: Fetched primary data is valid. [2025-11-04T17:10:06.958859] [DEBUG] [LOCAL ] AKR: Saving primary data. [2025-11-04T17:10:06.960035] [DEBUG] [LOCAL ] AKR: Trimmed 65776 records to 46191 (keeping 200 days of data) [2025-11-04T17:10:07.230699] [DEBUG] [LOCAL ] AKR: Primary data saved (46191 records). [2025-11-04T17:10:07.230818] [INFO ] [MAIN ] AKR: Full history saved. [2025-11-04T17:10:07.231323] [INFO ] [MAIN ] Update Progress: 77/2367 symbols completed. AKR took 52.9s. Estimated remaining: 4h 59m 21s. Rate: 7.6 symbols/min [2025-11-04T17:10:07.500186] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T17:10:07.505765] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-07. [2025-11-04T17:10:09.923565] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-07 (2.42 seconds). [2025-11-04T17:10:09.931549] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-08. [2025-11-04T17:10:11.162080] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T17:10:11.170155] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-09. [2025-11-04T17:10:12.321049] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T17:10:12.329548] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-10. [2025-11-04T17:10:13.440003] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-11-04T17:10:13.448713] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-11. [2025-11-04T17:10:14.917517] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-11 (1.47 seconds). [2025-11-04T17:10:14.919104] [INFO ] [REMOTE ] ALE: Full fetch complete (44529 records). [2025-11-04T17:10:14.919214] [DEBUG] [VALIDATOR ] ALE: Validating primary data. [2025-11-04T17:10:15.305141] [DEBUG] [VALIDATOR ] ALE: All expected trading days are present. [2025-11-04T17:10:15.305324] [DEBUG] [VALIDATOR ] ALE: Primary data validation passed. [2025-11-04T17:10:15.305850] [INFO ] [MAIN ] ALE: Fetched primary data is valid. [2025-11-04T17:10:15.305913] [DEBUG] [LOCAL ] ALE: Saving primary data. [2025-11-04T17:10:15.306997] [DEBUG] [LOCAL ] ALE: Trimmed 44529 records to 32063 (keeping 200 days of data) [2025-11-04T17:10:15.490844] [DEBUG] [LOCAL ] ALE: Primary data saved (32063 records). [2025-11-04T17:10:15.490963] [INFO ] [MAIN ] ALE: Full history saved. [2025-11-04T17:10:15.491037] [INFO ] [MAIN ] Update Progress: 78/2367 symbols completed. ALE took 14.6s. Estimated remaining: 4h 59m 26s. Rate: 7.6 symbols/min [2025-11-04T17:10:19.486014] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-08 (28.92 seconds). [2025-11-04T17:10:19.497148] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-09. [2025-11-04T17:10:20.908417] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-11-04T17:10:20.919564] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-10. [2025-11-04T17:10:21.133333] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-11 (28.17 seconds). [2025-11-04T17:10:21.135322] [INFO ] [REMOTE ] AL: Full fetch complete (64067 records). [2025-11-04T17:10:21.135461] [DEBUG] [VALIDATOR ] AL: Validating primary data. [2025-11-04T17:10:21.541863] [DEBUG] [VALIDATOR ] AL: All expected trading days are present. [2025-11-04T17:10:21.541941] [DEBUG] [VALIDATOR ] AL: Primary data validation passed. [2025-11-04T17:10:21.542544] [INFO ] [MAIN ] AL: Fetched primary data is valid. [2025-11-04T17:10:21.542584] [DEBUG] [LOCAL ] AL: Saving primary data. [2025-11-04T17:10:21.543766] [DEBUG] [LOCAL ] AL: Trimmed 64067 records to 44761 (keeping 200 days of data) [2025-11-04T17:10:21.812482] [DEBUG] [LOCAL ] AL: Primary data saved (44761 records). [2025-11-04T17:10:21.812611] [INFO ] [MAIN ] AL: Full history saved. [2025-11-04T17:10:21.813372] [INFO ] [MAIN ] Update Progress: 79/2367 symbols completed. AL took 1m 0s. Estimated remaining: 4h 58m 34s. Rate: 7.7 symbols/min [2025-11-04T17:10:22.371528] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T17:10:22.385136] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-11. [2025-11-04T17:10:23.679762] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-11 (1.29 seconds). [2025-11-04T17:10:23.682446] [INFO ] [REMOTE ] ALC: Full fetch complete (74074 records). [2025-11-04T17:10:23.682587] [DEBUG] [VALIDATOR ] ALC: Validating primary data. [2025-11-04T17:10:24.098314] [DEBUG] [VALIDATOR ] ALC: All expected trading days are present. [2025-11-04T17:10:24.098403] [DEBUG] [VALIDATOR ] ALC: Primary data validation passed. [2025-11-04T17:10:24.099157] [INFO ] [MAIN ] ALC: Fetched primary data is valid. [2025-11-04T17:10:24.099222] [DEBUG] [LOCAL ] ALC: Saving primary data. [2025-11-04T17:10:24.100708] [DEBUG] [LOCAL ] ALC: Trimmed 74074 records to 51168 (keeping 200 days of data) [2025-11-04T17:10:24.399908] [DEBUG] [LOCAL ] ALC: Primary data saved (51168 records). [2025-11-04T17:10:24.400041] [INFO ] [MAIN ] ALC: Full history saved. [2025-11-04T17:10:24.401233] [INFO ] [MAIN ] Update Progress: 80/2367 symbols completed. ALC took 49.0s. Estimated remaining: 4h 55m 56s. Rate: 7.7 symbols/min [2025-11-04T17:10:25.061952] [INFO ] [MAIN ] ALG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:25.061166] [INFO ] [MAIN ] ALEX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:25.119481] [INFO ] [MAIN ] ALIT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:25.123179] [INFO ] [MAIN ] ALH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:25.123751] [DEBUG] [LOCAL ] ALIT: No existing data file found in primary storage. [2025-11-04T17:10:25.123921] [DEBUG] [LOCAL ] ALH: No existing data file found in primary storage. [2025-11-04T17:10:25.123833] [WARN ] [MAIN ] ALIT: No existing data found, performing full refresh instead. [2025-11-04T17:10:25.124104] [INFO ] [MAIN ] ALIT: Wiping existing primary data before full refresh. [2025-11-04T17:10:25.124168] [INFO ] [LOCAL ] ALIT: Starting deletion of primary data. [2025-11-04T17:10:25.124310] [DEBUG] [LOCAL ] ALIT: No primary data file found to delete. [2025-11-04T17:10:25.124367] [INFO ] [MAIN ] ALIT: Starting full primary refresh. [2025-11-04T17:10:25.124232] [WARN ] [MAIN ] ALH: No existing data found, performing full refresh instead. [2025-11-04T17:10:25.138440] [INFO ] [MAIN ] ALH: Wiping existing primary data before full refresh. [2025-11-04T17:10:25.161973] [INFO ] [LOCAL ] ALH: Starting deletion of primary data. [2025-11-04T17:10:25.189752] [DEBUG] [LOCAL ] ALH: No primary data file found to delete. [2025-11-04T17:10:25.235301] [INFO ] [MAIN ] ALH: Starting full primary refresh. [2025-11-04T17:10:25.225927] [DEBUG] [LOCAL ] ALG: Loaded 9467 existing records from primary storage. [2025-11-04T17:10:25.254570] [INFO ] [MAIN ] ALG: Loaded 9467 existing records. [2025-11-04T17:10:25.680572] [DEBUG] [LOCAL ] ALEX: Loaded 30367 existing records from primary storage. [2025-11-04T17:10:25.786112] [INFO ] [MAIN ] ALEX: Loaded 30367 existing records. [2025-11-04T17:10:25.965618] [INFO ] [REMOTE ] ALIT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:26.049118] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-01. [2025-11-04T17:10:26.035213] [WARN ] [MAIN ] ALG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:10:26.128559] [INFO ] [MAIN ] ALG: Wiping existing primary data before full refresh. [2025-11-04T17:10:26.152946] [INFO ] [LOCAL ] ALG: Starting deletion of primary data. [2025-11-04T17:10:26.208674] [DEBUG] [LOCAL ] ALG: Deleted primary data file. [2025-11-04T17:10:26.224579] [INFO ] [MAIN ] ALG: Starting full primary refresh. [2025-11-04T17:10:26.244801] [INFO ] [REMOTE ] ALH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:26.261143] [INFO ] [REMOTE ] ALH: Fetching primary data for 2025-01. [2025-11-04T17:10:26.398119] [WARN ] [MAIN ] ALEX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:10:26.414850] [INFO ] [MAIN ] ALEX: Wiping existing primary data before full refresh. [2025-11-04T17:10:26.415168] [INFO ] [LOCAL ] ALEX: Starting deletion of primary data. [2025-11-04T17:10:26.436031] [DEBUG] [LOCAL ] ALEX: Deleted primary data file. [2025-11-04T17:10:26.453061] [INFO ] [MAIN ] ALEX: Starting full primary refresh. [2025-11-04T17:10:26.597163] [DEBUG] [REMOTE ] ALH: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:10:26.629091] [WARN ] [REMOTE ] ALH: No data returned for month 2025-01 [2025-11-04T17:10:26.641512] [FATAL] [REMOTE ] ALH: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:10:26.672351] [FATAL] [MAIN ] ALH: Failed to fetch primary data. [2025-11-04T17:10:26.682713] [WARN ] [MAIN ] ALH: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:10:26.709800] [INFO ] [LOCAL ] ALH: Starting deletion of primary data. [2025-11-04T17:10:26.790758] [DEBUG] [LOCAL ] ALH: No primary data file found to delete. [2025-11-04T17:10:26.816402] [INFO ] [LOCAL ] ALH: Starting deletion of derivative data. [2025-11-04T17:10:26.854482] [DEBUG] [LOCAL ] ALH: No derivative data file found to delete. [2025-11-04T17:10:26.864845] [INFO ] [MAIN ] Update Progress: 81/2367 symbols completed. ALH took 2.5s. Estimated remaining: 4h 53m 19s. Rate: 7.8 symbols/min [2025-11-04T17:10:27.027904] [INFO ] [REMOTE ] ALG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:27.034774] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-01. [2025-11-04T17:10:27.200505] [INFO ] [REMOTE ] ALEX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:27.216292] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-01. [2025-11-04T17:10:27.222622] [INFO ] [MAIN ] ALK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:27.276114] [DEBUG] [LOCAL ] ALK: Loaded 53086 existing records from primary storage. [2025-11-04T17:10:27.276452] [INFO ] [MAIN ] ALK: Loaded 53086 existing records. [2025-11-04T17:10:27.461468] [WARN ] [MAIN ] ALK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:10:27.461811] [INFO ] [MAIN ] ALK: Wiping existing primary data before full refresh. [2025-11-04T17:10:27.461912] [INFO ] [LOCAL ] ALK: Starting deletion of primary data. [2025-11-04T17:10:27.462256] [DEBUG] [LOCAL ] ALK: Deleted primary data file. [2025-11-04T17:10:27.462312] [INFO ] [MAIN ] ALK: Starting full primary refresh. [2025-11-04T17:10:27.596174] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-11-04T17:10:27.615516] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-02. [2025-11-04T17:10:27.880042] [INFO ] [REMOTE ] ALK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:27.880821] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-11-04T17:10:27.881085] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-01. [2025-11-04T17:10:27.884046] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-02. [2025-11-04T17:10:28.363527] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T17:10:28.369444] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-02. [2025-11-04T17:10:28.410537] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-11-04T17:10:28.414458] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-03. [2025-11-04T17:10:28.829387] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T17:10:28.840213] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-03. [2025-11-04T17:10:29.081462] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-11-04T17:10:29.085260] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-04. [2025-11-04T17:10:29.248954] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-11-04T17:10:29.254290] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-03. [2025-11-04T17:10:29.480449] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-11-04T17:10:29.498252] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-02. [2025-11-04T17:10:29.715297] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-11-04T17:10:29.720044] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-05. [2025-11-04T17:10:30.193523] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-11-04T17:10:30.210340] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-04. [2025-11-04T17:10:30.238839] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T17:10:30.248367] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-04. [2025-11-04T17:10:30.257615] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-11-04T17:10:30.260613] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-06. [2025-11-04T17:10:30.940350] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-11-04T17:10:30.956452] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-03. [2025-11-04T17:10:30.967745] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T17:10:30.972263] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-07. [2025-11-04T17:10:31.361423] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-11-04T17:10:31.369591] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-05. [2025-11-04T17:10:31.527434] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-11-04T17:10:31.531691] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-08. [2025-11-04T17:10:31.613286] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T17:10:31.624315] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-05. [2025-11-04T17:10:32.101855] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-11-04T17:10:32.104290] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-09. [2025-11-04T17:10:32.177444] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T17:10:32.184157] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-06. [2025-11-04T17:10:32.455268] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-11-04T17:10:32.467381] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-04. [2025-11-04T17:10:32.594996] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-11-04T17:10:32.597512] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-10. [2025-11-04T17:10:32.921793] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-11-04T17:10:32.928242] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-07. [2025-11-04T17:10:33.040342] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-11-04T17:10:33.051845] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-06. [2025-11-04T17:10:33.087415] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-11-04T17:10:33.090029] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-11. [2025-11-04T17:10:33.590300] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T17:10:33.591191] [INFO ] [REMOTE ] ALG: Full fetch complete (14903 records). [2025-11-04T17:10:33.591281] [DEBUG] [VALIDATOR ] ALG: Validating primary data. [2025-11-04T17:10:33.591691] [FATAL] [VALIDATOR ] ALG: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:10:33.591773] [FATAL] [MAIN ] ALG: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:10:33.591816] [WARN ] [MAIN ] ALG: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:10:33.591984] [INFO ] [LOCAL ] ALG: Starting deletion of primary data. [2025-11-04T17:10:33.592054] [DEBUG] [LOCAL ] ALG: No primary data file found to delete. [2025-11-04T17:10:33.592090] [INFO ] [LOCAL ] ALG: Starting deletion of derivative data. [2025-11-04T17:10:33.592458] [DEBUG] [LOCAL ] ALG: Deleted derivative data file. [2025-11-04T17:10:33.592941] [INFO ] [MAIN ] Update Progress: 82/2367 symbols completed. ALG took 9.2s. Estimated remaining: 4h 52m 44s. Rate: 7.8 symbols/min [2025-11-04T17:10:33.768484] [INFO ] [MAIN ] ALL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:33.813069] [DEBUG] [LOCAL ] ALL: Loaded 49337 existing records from primary storage. [2025-11-04T17:10:33.813389] [INFO ] [MAIN ] ALL: Loaded 49337 existing records. [2025-11-04T17:10:34.008508] [WARN ] [MAIN ] ALL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:10:34.024504] [INFO ] [MAIN ] ALL: Wiping existing primary data before full refresh. [2025-11-04T17:10:34.026948] [INFO ] [LOCAL ] ALL: Starting deletion of primary data. [2025-11-04T17:10:34.027144] [DEBUG] [LOCAL ] ALL: Deleted primary data file. [2025-11-04T17:10:34.027184] [INFO ] [MAIN ] ALL: Starting full primary refresh. [2025-11-04T17:10:34.099991] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T17:10:34.125276] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-08. [2025-11-04T17:10:34.455542] [INFO ] [REMOTE ] ALL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:34.459765] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-01. [2025-11-04T17:10:34.459130] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-11-04T17:10:34.512400] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T17:10:34.567054] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-07. [2025-11-04T17:10:35.062405] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-11-04T17:10:35.367042] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-05. [2025-11-04T17:10:36.167130] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-09. [2025-11-04T17:10:36.801709] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T17:10:36.967040] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-06. [2025-11-04T17:10:37.183407] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T17:10:37.767039] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-10. [2025-11-04T17:10:38.470443] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-11-04T17:10:38.567077] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-07. [2025-11-04T17:10:38.920333] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-11-04T17:10:39.367024] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-11. [2025-11-04T17:10:40.196458] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T17:10:40.210366] [INFO ] [REMOTE ] ALEX: Full fetch complete (44889 records). [2025-11-04T17:10:40.227609] [DEBUG] [VALIDATOR ] ALEX: Validating primary data. [2025-11-04T17:10:40.242291] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-11-04T17:10:40.332442] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-08. [2025-11-04T17:10:40.662572] [DEBUG] [VALIDATOR ] ALEX: All expected trading days are present. [2025-11-04T17:10:40.662679] [DEBUG] [VALIDATOR ] ALEX: Primary data validation passed. [2025-11-04T17:10:40.663166] [INFO ] [MAIN ] ALEX: Fetched primary data is valid. [2025-11-04T17:10:40.663208] [DEBUG] [LOCAL ] ALEX: Saving primary data. [2025-11-04T17:10:40.664187] [DEBUG] [LOCAL ] ALEX: Trimmed 44889 records to 30422 (keeping 200 days of data) [2025-11-04T17:10:40.845995] [DEBUG] [LOCAL ] ALEX: Primary data saved (30422 records). [2025-11-04T17:10:40.846140] [INFO ] [MAIN ] ALEX: Full history saved. [2025-11-04T17:10:40.846238] [INFO ] [MAIN ] Update Progress: 83/2367 symbols completed. ALEX took 16.4s. Estimated remaining: 4h 52m 24s. Rate: 7.8 symbols/min [2025-11-04T17:10:41.026213] [INFO ] [MAIN ] ALLE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:41.026400] [DEBUG] [LOCAL ] ALLE: No existing data file found in primary storage. [2025-11-04T17:10:41.026438] [WARN ] [MAIN ] ALLE: No existing data found, performing full refresh instead. [2025-11-04T17:10:41.026551] [INFO ] [MAIN ] ALLE: Wiping existing primary data before full refresh. [2025-11-04T17:10:41.026592] [INFO ] [LOCAL ] ALLE: Starting deletion of primary data. [2025-11-04T17:10:41.026632] [DEBUG] [LOCAL ] ALLE: No primary data file found to delete. [2025-11-04T17:10:41.026702] [INFO ] [MAIN ] ALLE: Starting full primary refresh. [2025-11-04T17:10:41.396074] [INFO ] [REMOTE ] ALLE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:41.396382] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-01. [2025-11-04T17:10:41.867201] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T17:10:41.878126] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-09. [2025-11-04T17:10:42.910781] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-11-04T17:10:42.919295] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-02. [2025-11-04T17:10:43.377444] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T17:10:43.389004] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-10. [2025-11-04T17:10:44.056978] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-01 (9.60 seconds). [2025-11-04T17:10:44.167043] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-02. [2025-11-04T17:10:44.946820] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-07 (10.38 seconds). [2025-11-04T17:10:44.967002] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-08. [2025-11-04T17:10:45.074838] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T17:10:45.503502] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-11-04T17:10:45.534909] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-02 (2.62 seconds). [2025-11-04T17:10:45.767060] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-03. [2025-11-04T17:10:46.567090] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-03. [2025-11-04T17:10:46.838608] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-11-04T17:10:47.294546] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T17:10:47.367007] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-11. [2025-11-04T17:10:48.005421] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T17:10:48.167056] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-04. [2025-11-04T17:10:48.792436] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-11 (1.43 seconds). [2025-11-04T17:10:48.794757] [INFO ] [REMOTE ] ALK: Full fetch complete (77052 records). [2025-11-04T17:10:48.794895] [DEBUG] [VALIDATOR ] ALK: Validating primary data. [2025-11-04T17:10:48.974740] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-04. [2025-11-04T17:10:49.274416] [DEBUG] [VALIDATOR ] ALK: All expected trading days are present. [2025-11-04T17:10:49.274499] [DEBUG] [VALIDATOR ] ALK: Primary data validation passed. [2025-11-04T17:10:49.275264] [INFO ] [MAIN ] ALK: Fetched primary data is valid. [2025-11-04T17:10:49.275328] [DEBUG] [LOCAL ] ALK: Saving primary data. [2025-11-04T17:10:49.276750] [DEBUG] [LOCAL ] ALK: Trimmed 77052 records to 53083 (keeping 200 days of data) [2025-11-04T17:10:49.580266] [DEBUG] [LOCAL ] ALK: Primary data saved (53083 records). [2025-11-04T17:10:49.580478] [INFO ] [MAIN ] ALK: Full history saved. [2025-11-04T17:10:49.580567] [INFO ] [MAIN ] Update Progress: 84/2367 symbols completed. ALK took 22.7s. Estimated remaining: 4h 52m 45s. Rate: 7.8 symbols/min [2025-11-04T17:10:49.790099] [INFO ] [MAIN ] ALLY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:49.787292] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-09. [2025-11-04T17:10:49.791384] [DEBUG] [LOCAL ] ALLY: No existing data file found in primary storage. [2025-11-04T17:10:49.794728] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-11-04T17:10:49.795024] [WARN ] [MAIN ] ALLY: No existing data found, performing full refresh instead. [2025-11-04T17:10:49.806554] [INFO ] [MAIN ] ALLY: Wiping existing primary data before full refresh. [2025-11-04T17:10:49.807495] [INFO ] [LOCAL ] ALLY: Starting deletion of primary data. [2025-11-04T17:10:49.807949] [DEBUG] [LOCAL ] ALLY: No primary data file found to delete. [2025-11-04T17:10:49.808158] [INFO ] [MAIN ] ALLY: Starting full primary refresh. [2025-11-04T17:10:50.213854] [INFO ] [REMOTE ] ALLY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:50.504094] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-11-04T17:10:50.567007] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-05. [2025-11-04T17:10:51.240963] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-11-04T17:10:51.367011] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-05. [2025-11-04T17:10:51.804410] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-11-04T17:10:52.167033] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-01. [2025-11-04T17:10:52.689718] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T17:10:52.967020] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-10. [2025-11-04T17:10:53.670295] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T17:10:53.767011] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-02. [2025-11-04T17:10:54.367570] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T17:10:54.567021] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-06. [2025-11-04T17:10:54.973213] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T17:10:55.367035] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-11. [2025-11-04T17:10:55.940300] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-11-04T17:10:56.167042] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-03. [2025-11-04T17:10:56.860583] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-11 (1.49 seconds). [2025-11-04T17:10:56.862661] [INFO ] [REMOTE ] ALIT: Full fetch complete (76635 records). [2025-11-04T17:10:56.862759] [DEBUG] [VALIDATOR ] ALIT: Validating primary data. [2025-11-04T17:10:56.972234] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-07. [2025-11-04T17:10:57.259746] [DEBUG] [VALIDATOR ] ALIT: All expected trading days are present. [2025-11-04T17:10:57.259836] [DEBUG] [VALIDATOR ] ALIT: Primary data validation passed. [2025-11-04T17:10:57.260552] [INFO ] [MAIN ] ALIT: Fetched primary data is valid. [2025-11-04T17:10:57.260594] [DEBUG] [LOCAL ] ALIT: Saving primary data. [2025-11-04T17:10:57.262073] [DEBUG] [LOCAL ] ALIT: Trimmed 76635 records to 52751 (keeping 200 days of data) [2025-11-04T17:10:57.563113] [DEBUG] [LOCAL ] ALIT: Primary data saved (52751 records). [2025-11-04T17:10:57.563336] [INFO ] [MAIN ] ALIT: Full history saved. [2025-11-04T17:10:57.563587] [INFO ] [MAIN ] Update Progress: 85/2367 symbols completed. ALIT took 33.1s. Estimated remaining: 4h 52m 45s. Rate: 7.8 symbols/min [2025-11-04T17:10:57.738605] [INFO ] [MAIN ] ALSN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:10:57.771864] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-06. [2025-11-04T17:10:57.775314] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-11-04T17:10:57.820587] [DEBUG] [LOCAL ] ALSN: Loaded 43010 existing records from primary storage. [2025-11-04T17:10:57.820909] [INFO ] [MAIN ] ALSN: Loaded 43010 existing records. [2025-11-04T17:10:58.008672] [WARN ] [MAIN ] ALSN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:10:58.008915] [INFO ] [MAIN ] ALSN: Wiping existing primary data before full refresh. [2025-11-04T17:10:58.008964] [INFO ] [LOCAL ] ALSN: Starting deletion of primary data. [2025-11-04T17:10:58.009283] [DEBUG] [LOCAL ] ALSN: Deleted primary data file. [2025-11-04T17:10:58.009341] [INFO ] [MAIN ] ALSN: Starting full primary refresh. [2025-11-04T17:10:58.389865] [INFO ] [REMOTE ] ALSN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:10:58.423343] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T17:10:58.567047] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-04. [2025-11-04T17:10:59.187896] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-11-04T17:10:59.367056] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-07. [2025-11-04T17:11:00.145148] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-11-04T17:11:00.167002] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-05. [2025-11-04T17:11:00.967088] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-01. [2025-11-04T17:11:01.036310] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-11-04T17:11:01.475619] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T17:11:01.767042] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-06. [2025-11-04T17:11:02.403869] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T17:11:02.567038] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-08. [2025-11-04T17:11:03.036012] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T17:11:03.367033] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-07. [2025-11-04T17:11:03.975004] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-11-04T17:11:04.167040] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-08. [2025-11-04T17:11:04.833329] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-11-04T17:11:04.967062] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-09. [2025-11-04T17:11:05.628029] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-11-04T17:11:05.767051] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-09. [2025-11-04T17:11:06.326678] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T17:11:06.567044] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-10. [2025-11-04T17:11:07.366416] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-11-04T17:11:07.376024] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-08. [2025-11-04T17:11:07.882535] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T17:11:08.167063] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-10. [2025-11-04T17:11:08.843666] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-11-04T17:11:08.967108] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-02. [2025-11-04T17:11:09.630120] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T17:11:09.767095] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-09. [2025-11-04T17:11:10.297843] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T17:11:10.567126] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-03. [2025-11-04T17:11:11.213956] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-11-04T17:11:11.367096] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-11. [2025-11-04T17:11:11.863159] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T17:11:11.865350] [INFO ] [REMOTE ] ALL: Full fetch complete (72205 records). [2025-11-04T17:11:11.865484] [DEBUG] [VALIDATOR ] ALL: Validating primary data. [2025-11-04T17:11:12.175780] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-10. [2025-11-04T17:11:12.259464] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-11-04T17:11:12.349618] [DEBUG] [VALIDATOR ] ALL: All expected trading days are present. [2025-11-04T17:11:12.349774] [DEBUG] [VALIDATOR ] ALL: Primary data validation passed. [2025-11-04T17:11:12.350571] [INFO ] [MAIN ] ALL: Fetched primary data is valid. [2025-11-04T17:11:12.350618] [DEBUG] [LOCAL ] ALL: Saving primary data. [2025-11-04T17:11:12.352146] [DEBUG] [LOCAL ] ALL: Trimmed 72205 records to 49309 (keeping 200 days of data) [2025-11-04T17:11:12.656205] [DEBUG] [LOCAL ] ALL: Primary data saved (49309 records). [2025-11-04T17:11:12.656353] [INFO ] [MAIN ] ALL: Full history saved. [2025-11-04T17:11:12.656440] [INFO ] [MAIN ] Update Progress: 86/2367 symbols completed. ALL took 39.1s. Estimated remaining: 4h 55m 54s. Rate: 7.7 symbols/min [2025-11-04T17:11:12.832587] [INFO ] [MAIN ] ALTG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:11:12.832843] [DEBUG] [LOCAL ] ALTG: No existing data file found in primary storage. [2025-11-04T17:11:12.832896] [WARN ] [MAIN ] ALTG: No existing data found, performing full refresh instead. [2025-11-04T17:11:12.833022] [INFO ] [MAIN ] ALTG: Wiping existing primary data before full refresh. [2025-11-04T17:11:12.833065] [INFO ] [LOCAL ] ALTG: Starting deletion of primary data. [2025-11-04T17:11:12.833110] [DEBUG] [LOCAL ] ALTG: No primary data file found to delete. [2025-11-04T17:11:12.833144] [INFO ] [MAIN ] ALTG: Starting full primary refresh. [2025-11-04T17:11:12.976412] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-11. [2025-11-04T17:11:13.196266] [INFO ] [REMOTE ] ALTG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:11:13.767026] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-04. [2025-11-04T17:11:13.886605] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-11-04T17:11:14.162996] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T17:11:14.164706] [INFO ] [REMOTE ] ALLE: Full fetch complete (64784 records). [2025-11-04T17:11:14.164800] [DEBUG] [VALIDATOR ] ALLE: Validating primary data. [2025-11-04T17:11:14.552745] [DEBUG] [VALIDATOR ] ALLE: All expected trading days are present. [2025-11-04T17:11:14.552832] [DEBUG] [VALIDATOR ] ALLE: Primary data validation passed. [2025-11-04T17:11:14.553496] [INFO ] [MAIN ] ALLE: Fetched primary data is valid. [2025-11-04T17:11:14.553561] [DEBUG] [LOCAL ] ALLE: Saving primary data. [2025-11-04T17:11:14.554900] [DEBUG] [LOCAL ] ALLE: Trimmed 64784 records to 43482 (keeping 200 days of data) [2025-11-04T17:11:14.609882] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-11. [2025-11-04T17:11:14.826547] [DEBUG] [LOCAL ] ALLE: Primary data saved (43482 records). [2025-11-04T17:11:14.826705] [INFO ] [MAIN ] ALLE: Full history saved. [2025-11-04T17:11:14.826786] [INFO ] [MAIN ] Update Progress: 87/2367 symbols completed. ALLE took 34.0s. Estimated remaining: 4h 53m 19s. Rate: 7.8 symbols/min [2025-11-04T17:11:14.998773] [INFO ] [MAIN ] ALUR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:11:15.008742] [DEBUG] [LOCAL ] ALUR: Loaded 8943 existing records from primary storage. [2025-11-04T17:11:15.008965] [INFO ] [MAIN ] ALUR: Loaded 8943 existing records. [2025-11-04T17:11:15.195050] [WARN ] [MAIN ] ALUR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:11:15.195339] [INFO ] [MAIN ] ALUR: Wiping existing primary data before full refresh. [2025-11-04T17:11:15.195474] [INFO ] [LOCAL ] ALUR: Starting deletion of primary data. [2025-11-04T17:11:15.195768] [DEBUG] [LOCAL ] ALUR: Deleted primary data file. [2025-11-04T17:11:15.195891] [INFO ] [MAIN ] ALUR: Starting full primary refresh. [2025-11-04T17:11:15.376559] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-01. [2025-11-04T17:11:15.488083] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-11-04T17:11:15.606073] [INFO ] [REMOTE ] ALUR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:11:16.165924] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-11 (1.56 seconds). [2025-11-04T17:11:16.167529] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-05. [2025-11-04T17:11:16.169130] [INFO ] [REMOTE ] ALLY: Full fetch complete (77234 records). [2025-11-04T17:11:16.169217] [DEBUG] [VALIDATOR ] ALLY: Validating primary data. [2025-11-04T17:11:16.472690] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-11-04T17:11:16.599028] [DEBUG] [VALIDATOR ] ALLY: All expected trading days are present. [2025-11-04T17:11:16.599119] [DEBUG] [VALIDATOR ] ALLY: Primary data validation passed. [2025-11-04T17:11:16.599891] [INFO ] [MAIN ] ALLY: Fetched primary data is valid. [2025-11-04T17:11:16.599954] [DEBUG] [LOCAL ] ALLY: Saving primary data. [2025-11-04T17:11:16.601596] [DEBUG] [LOCAL ] ALLY: Trimmed 77234 records to 53188 (keeping 200 days of data) [2025-11-04T17:11:16.920454] [DEBUG] [LOCAL ] ALLY: Primary data saved (53188 records). [2025-11-04T17:11:16.920599] [INFO ] [MAIN ] ALLY: Full history saved. [2025-11-04T17:11:16.920712] [INFO ] [MAIN ] Update Progress: 88/2367 symbols completed. ALLY took 27.3s. Estimated remaining: 4h 50m 45s. Rate: 7.8 symbols/min [2025-11-04T17:11:16.975966] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-02. [2025-11-04T17:11:17.094698] [INFO ] [MAIN ] ALV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:11:17.133902] [DEBUG] [LOCAL ] ALV: Loaded 41813 existing records from primary storage. [2025-11-04T17:11:17.134260] [INFO ] [MAIN ] ALV: Loaded 41813 existing records. [2025-11-04T17:11:17.318474] [WARN ] [MAIN ] ALV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:11:17.318836] [INFO ] [MAIN ] ALV: Wiping existing primary data before full refresh. [2025-11-04T17:11:17.318924] [INFO ] [LOCAL ] ALV: Starting deletion of primary data. [2025-11-04T17:11:17.319303] [DEBUG] [LOCAL ] ALV: Deleted primary data file. [2025-11-04T17:11:17.319443] [INFO ] [MAIN ] ALV: Starting full primary refresh. [2025-11-04T17:11:17.709304] [INFO ] [REMOTE ] ALV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:11:17.711671] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T17:11:17.767053] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-01. [2025-11-04T17:11:17.852771] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-11-04T17:11:18.567132] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-03. [2025-11-04T17:11:19.147883] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T17:11:19.367077] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-01. [2025-11-04T17:11:19.609458] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T17:11:20.167148] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-02. [2025-11-04T17:11:20.664765] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T17:11:20.967067] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-04. [2025-11-04T17:11:21.359411] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T17:11:21.710910] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-11-04T17:11:21.767110] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-05. [2025-11-04T17:11:22.477523] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-11-04T17:11:22.567102] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-03. [2025-11-04T17:11:23.367055] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-02. [2025-11-04T17:11:23.972625] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T17:11:24.167020] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-06. [2025-11-04T17:11:24.206172] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-11-04T17:11:24.967085] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-06. [2025-11-04T17:11:25.625917] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T17:11:25.761892] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T17:11:25.767078] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-07. [2025-11-04T17:11:26.567112] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-04. [2025-11-04T17:11:27.129504] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T17:11:27.367089] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-03. [2025-11-04T17:11:27.946301] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T17:11:28.167116] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-07. [2025-11-04T17:11:28.375490] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T17:11:28.967044] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-04. [2025-11-04T17:11:29.418190] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T17:11:29.552961] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-11-04T17:11:29.767049] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-08. [2025-11-04T17:11:30.567046] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-05. [2025-11-04T17:11:30.780561] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-11-04T17:11:31.367072] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-08. [2025-11-04T17:11:31.860181] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T17:11:32.167138] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-09. [2025-11-04T17:11:32.543710] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-11-04T17:11:32.967071] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-05. [2025-11-04T17:11:33.004583] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-11-04T17:11:33.767074] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-10. [2025-11-04T17:11:33.843985] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T17:11:34.558256] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T17:11:34.566993] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-11. [2025-11-04T17:11:35.347265] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T17:11:35.348738] [INFO ] [REMOTE ] ALTG: Full fetch complete (33334 records). [2025-11-04T17:11:35.348827] [DEBUG] [VALIDATOR ] ALTG: Validating primary data. [2025-11-04T17:11:35.372189] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-09. [2025-11-04T17:11:35.763594] [DEBUG] [VALIDATOR ] ALTG: All expected trading days are present. [2025-11-04T17:11:35.763728] [DEBUG] [VALIDATOR ] ALTG: Primary data validation passed. [2025-11-04T17:11:35.764315] [INFO ] [MAIN ] ALTG: Fetched primary data is valid. [2025-11-04T17:11:35.764365] [DEBUG] [LOCAL ] ALTG: Saving primary data. [2025-11-04T17:11:35.765564] [DEBUG] [LOCAL ] ALTG: Trimmed 33334 records to 22724 (keeping 200 days of data) [2025-11-04T17:11:35.906375] [DEBUG] [LOCAL ] ALTG: Primary data saved (22724 records). [2025-11-04T17:11:35.906585] [INFO ] [MAIN ] ALTG: Full history saved. [2025-11-04T17:11:35.906743] [INFO ] [MAIN ] Update Progress: 89/2367 symbols completed. ALTG took 23.3s. Estimated remaining: 4h 55m 28s. Rate: 7.7 symbols/min [2025-11-04T17:11:36.091566] [INFO ] [MAIN ] ALX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:11:36.091854] [DEBUG] [LOCAL ] ALX: No existing data file found in primary storage. [2025-11-04T17:11:36.091899] [WARN ] [MAIN ] ALX: No existing data found, performing full refresh instead. [2025-11-04T17:11:36.092027] [INFO ] [MAIN ] ALX: Wiping existing primary data before full refresh. [2025-11-04T17:11:36.092069] [INFO ] [LOCAL ] ALX: Starting deletion of primary data. [2025-11-04T17:11:36.092113] [DEBUG] [LOCAL ] ALX: No primary data file found to delete. [2025-11-04T17:11:36.092148] [INFO ] [MAIN ] ALX: Starting full primary refresh. [2025-11-04T17:11:36.172154] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-06. [2025-11-04T17:11:36.474096] [INFO ] [REMOTE ] ALX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:11:36.967044] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-06. [2025-11-04T17:11:37.004103] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-11-04T17:11:37.272259] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T17:11:37.595729] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-11-04T17:11:37.767101] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-07. [2025-11-04T17:11:38.567033] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-10. [2025-11-04T17:11:39.055767] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-11-04T17:11:39.367110] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-07. [2025-11-04T17:11:40.026530] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T17:11:40.093677] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-11-04T17:11:40.167032] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-08. [2025-11-04T17:11:40.669147] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-11-04T17:11:40.967083] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-01. [2025-11-04T17:11:41.268574] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-01 (0.30 seconds). [2025-11-04T17:11:41.767044] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-02. [2025-11-04T17:11:42.225772] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-11-04T17:11:42.567068] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-11. [2025-11-04T17:11:43.367052] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-09. [2025-11-04T17:11:43.407047] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T17:11:43.408804] [INFO ] [REMOTE ] ALSN: Full fetch complete (62815 records). [2025-11-04T17:11:43.408969] [DEBUG] [VALIDATOR ] ALSN: Validating primary data. [2025-11-04T17:11:43.409964] [FATAL] [VALIDATOR ] ALSN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:11:43.410099] [FATAL] [MAIN ] ALSN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:11:43.410181] [WARN ] [MAIN ] ALSN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:11:43.410264] [INFO ] [LOCAL ] ALSN: Starting deletion of primary data. [2025-11-04T17:11:43.410358] [DEBUG] [LOCAL ] ALSN: No primary data file found to delete. [2025-11-04T17:11:43.410423] [INFO ] [LOCAL ] ALSN: Starting deletion of derivative data. [2025-11-04T17:11:43.410610] [DEBUG] [LOCAL ] ALSN: Deleted derivative data file. [2025-11-04T17:11:43.411702] [INFO ] [MAIN ] Update Progress: 90/2367 symbols completed. ALSN took 45.8s. Estimated remaining: 4h 55m 13s. Rate: 7.7 symbols/min [2025-11-04T17:11:43.600761] [INFO ] [MAIN ] AM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:11:43.658217] [DEBUG] [LOCAL ] AM: Loaded 51756 existing records from primary storage. [2025-11-04T17:11:43.658577] [INFO ] [MAIN ] AM: Loaded 51756 existing records. [2025-11-04T17:11:43.853193] [WARN ] [MAIN ] AM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:11:43.853479] [INFO ] [MAIN ] AM: Wiping existing primary data before full refresh. [2025-11-04T17:11:43.853535] [INFO ] [LOCAL ] AM: Starting deletion of primary data. [2025-11-04T17:11:43.853969] [DEBUG] [LOCAL ] AM: Deleted primary data file. [2025-11-04T17:11:43.854028] [INFO ] [MAIN ] AM: Starting full primary refresh. [2025-11-04T17:11:43.988579] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-11-04T17:11:44.172128] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-10. [2025-11-04T17:11:44.228130] [INFO ] [REMOTE ] AM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:11:44.731436] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-11-04T17:11:44.967102] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-03. [2025-11-04T17:11:45.312238] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-11-04T17:11:45.767078] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-04. [2025-11-04T17:11:46.132012] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T17:11:46.567086] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-01. [2025-11-04T17:11:47.367091] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-08. [2025-11-04T17:11:47.966119] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T17:11:48.167081] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-02. [2025-11-04T17:11:48.661430] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T17:11:48.967072] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-09. [2025-11-04T17:11:49.540864] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-11-04T17:11:49.767042] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-05. [2025-11-04T17:11:50.167227] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T17:11:50.356109] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T17:11:50.567099] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-10. [2025-11-04T17:11:51.367094] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-06. [2025-11-04T17:11:51.941375] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T17:11:51.942186] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-11-04T17:11:52.167042] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-07. [2025-11-04T17:11:52.669789] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-11-04T17:11:52.967031] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-03. [2025-11-04T17:11:53.767047] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-11. [2025-11-04T17:11:54.554092] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-11-04T17:11:54.567425] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-08. [2025-11-04T17:11:54.892680] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T17:11:54.894486] [INFO ] [REMOTE ] ALV: Full fetch complete (62424 records). [2025-11-04T17:11:54.894576] [DEBUG] [VALIDATOR ] ALV: Validating primary data. [2025-11-04T17:11:55.284979] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-11-04T17:11:55.284747] [DEBUG] [VALIDATOR ] ALV: All expected trading days are present. [2025-11-04T17:11:55.287536] [DEBUG] [VALIDATOR ] ALV: Primary data validation passed. [2025-11-04T17:11:55.288287] [INFO ] [MAIN ] ALV: Fetched primary data is valid. [2025-11-04T17:11:55.288349] [DEBUG] [LOCAL ] ALV: Saving primary data. [2025-11-04T17:11:55.289831] [DEBUG] [LOCAL ] ALV: Trimmed 62424 records to 41776 (keeping 200 days of data) [2025-11-04T17:11:55.376882] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-09. [2025-11-04T17:11:55.543779] [DEBUG] [LOCAL ] ALV: Primary data saved (41776 records). [2025-11-04T17:11:55.543902] [INFO ] [MAIN ] ALV: Full history saved. [2025-11-04T17:11:55.544899] [INFO ] [MAIN ] Update Progress: 91/2367 symbols completed. ALV took 38.6s. Estimated remaining: 4h 56m 54s. Rate: 7.7 symbols/min [2025-11-04T17:11:55.710200] [INFO ] [MAIN ] AMBC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:11:55.748430] [DEBUG] [LOCAL ] AMBC: Loaded 35170 existing records from primary storage. [2025-11-04T17:11:55.748728] [INFO ] [MAIN ] AMBC: Loaded 35170 existing records. [2025-11-04T17:11:55.931185] [WARN ] [MAIN ] AMBC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:11:55.931947] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T17:11:55.932102] [INFO ] [MAIN ] AMBC: Wiping existing primary data before full refresh. [2025-11-04T17:11:55.933971] [INFO ] [LOCAL ] AMBC: Starting deletion of primary data. [2025-11-04T17:11:55.934899] [DEBUG] [LOCAL ] AMBC: Deleted primary data file. [2025-11-04T17:11:55.934956] [INFO ] [MAIN ] AMBC: Starting full primary refresh. [2025-11-04T17:11:56.172241] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-04. [2025-11-04T17:11:56.284705] [INFO ] [REMOTE ] AMBC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:11:56.967058] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-11. [2025-11-04T17:11:57.545468] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-11 (0.58 seconds). [2025-11-04T17:11:57.546291] [INFO ] [REMOTE ] ALUR: Full fetch complete (17688 records). [2025-11-04T17:11:57.546363] [DEBUG] [VALIDATOR ] ALUR: Validating primary data. [2025-11-04T17:11:57.767696] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-01. [2025-11-04T17:11:57.925627] [DEBUG] [VALIDATOR ] ALUR: All expected trading days are present. [2025-11-04T17:11:57.925760] [DEBUG] [VALIDATOR ] ALUR: Primary data validation passed. [2025-11-04T17:11:57.926092] [INFO ] [MAIN ] ALUR: Fetched primary data is valid. [2025-11-04T17:11:57.926168] [DEBUG] [LOCAL ] ALUR: Saving primary data. [2025-11-04T17:11:57.927164] [DEBUG] [LOCAL ] ALUR: Trimmed 17688 records to 8924 (keeping 200 days of data) [2025-11-04T17:11:58.005462] [DEBUG] [LOCAL ] ALUR: Primary data saved (8924 records). [2025-11-04T17:11:58.006170] [INFO ] [MAIN ] ALUR: Full history saved. [2025-11-04T17:11:58.008253] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-11-04T17:11:58.008418] [INFO ] [MAIN ] Update Progress: 92/2367 symbols completed. ALUR took 43.2s. Estimated remaining: 4h 54m 34s. Rate: 7.7 symbols/min [2025-11-04T17:11:58.184788] [INFO ] [MAIN ] AMBP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:11:58.184954] [DEBUG] [LOCAL ] AMBP: No existing data file found in primary storage. [2025-11-04T17:11:58.184991] [WARN ] [MAIN ] AMBP: No existing data found, performing full refresh instead. [2025-11-04T17:11:58.185106] [INFO ] [MAIN ] AMBP: Wiping existing primary data before full refresh. [2025-11-04T17:11:58.185147] [INFO ] [LOCAL ] AMBP: Starting deletion of primary data. [2025-11-04T17:11:58.185186] [DEBUG] [LOCAL ] AMBP: No primary data file found to delete. [2025-11-04T17:11:58.185217] [INFO ] [MAIN ] AMBP: Starting full primary refresh. [2025-11-04T17:11:58.528146] [INFO ] [REMOTE ] AMBP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:11:58.566997] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-01. [2025-11-04T17:11:58.731175] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T17:11:59.367030] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-05. [2025-11-04T17:11:59.920786] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T17:12:00.167082] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-02. [2025-11-04T17:12:00.966995] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-11-04T17:12:00.967058] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-02. [2025-11-04T17:12:01.138713] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T17:12:01.767027] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-10. [2025-11-04T17:12:02.273452] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T17:12:02.542271] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-11-04T17:12:02.567030] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-11. [2025-11-04T17:12:03.084982] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-11 (0.52 seconds). [2025-11-04T17:12:03.085793] [INFO ] [REMOTE ] ALX: Full fetch complete (8616 records). [2025-11-04T17:12:03.085863] [DEBUG] [VALIDATOR ] ALX: Validating primary data. [2025-11-04T17:12:03.086220] [FATAL] [VALIDATOR ] ALX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:12:03.086307] [FATAL] [MAIN ] ALX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:12:03.086351] [WARN ] [MAIN ] ALX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:12:03.086415] [INFO ] [LOCAL ] ALX: Starting deletion of primary data. [2025-11-04T17:12:03.086476] [DEBUG] [LOCAL ] ALX: No primary data file found to delete. [2025-11-04T17:12:03.086512] [INFO ] [LOCAL ] ALX: Starting deletion of derivative data. [2025-11-04T17:12:03.086552] [DEBUG] [LOCAL ] ALX: No derivative data file found to delete. [2025-11-04T17:12:03.086605] [INFO ] [MAIN ] Update Progress: 93/2367 symbols completed. ALX took 27.2s. Estimated remaining: 4h 53m 20s. Rate: 7.8 symbols/min [2025-11-04T17:12:03.273776] [INFO ] [MAIN ] AMBQ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:12:03.273930] [DEBUG] [LOCAL ] AMBQ: No existing data file found in primary storage. [2025-11-04T17:12:03.273965] [WARN ] [MAIN ] AMBQ: No existing data found, performing full refresh instead. [2025-11-04T17:12:03.274071] [INFO ] [MAIN ] AMBQ: Wiping existing primary data before full refresh. [2025-11-04T17:12:03.274111] [INFO ] [LOCAL ] AMBQ: Starting deletion of primary data. [2025-11-04T17:12:03.274150] [DEBUG] [LOCAL ] AMBQ: No primary data file found to delete. [2025-11-04T17:12:03.274183] [INFO ] [MAIN ] AMBQ: Starting full primary refresh. [2025-11-04T17:12:03.372085] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-03. [2025-11-04T17:12:03.623142] [INFO ] [REMOTE ] AMBQ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:12:04.167051] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-06. [2025-11-04T17:12:04.715476] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T17:12:04.967029] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-01. [2025-11-04T17:12:05.235497] [DEBUG] [REMOTE ] AMBQ: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:12:05.235857] [WARN ] [REMOTE ] AMBQ: No data returned for month 2025-01 [2025-11-04T17:12:05.235967] [FATAL] [REMOTE ] AMBQ: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:12:05.236039] [FATAL] [MAIN ] AMBQ: Failed to fetch primary data. [2025-11-04T17:12:05.236107] [WARN ] [MAIN ] AMBQ: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:12:05.236180] [INFO ] [LOCAL ] AMBQ: Starting deletion of primary data. [2025-11-04T17:12:05.236245] [DEBUG] [LOCAL ] AMBQ: No primary data file found to delete. [2025-11-04T17:12:05.236282] [INFO ] [LOCAL ] AMBQ: Starting deletion of derivative data. [2025-11-04T17:12:05.236324] [DEBUG] [LOCAL ] AMBQ: No derivative data file found to delete. [2025-11-04T17:12:05.236373] [INFO ] [MAIN ] Update Progress: 94/2367 symbols completed. AMBQ took 2.1s. Estimated remaining: 4h 50m 57s. Rate: 7.8 symbols/min [2025-11-04T17:12:05.452788] [INFO ] [MAIN ] AMC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:12:05.453034] [DEBUG] [LOCAL ] AMC: No existing data file found in primary storage. [2025-11-04T17:12:05.453139] [WARN ] [MAIN ] AMC: No existing data found, performing full refresh instead. [2025-11-04T17:12:05.453271] [INFO ] [MAIN ] AMC: Wiping existing primary data before full refresh. [2025-11-04T17:12:05.453314] [INFO ] [LOCAL ] AMC: Starting deletion of primary data. [2025-11-04T17:12:05.453357] [DEBUG] [LOCAL ] AMC: No primary data file found to delete. [2025-11-04T17:12:05.453388] [INFO ] [MAIN ] AMC: Starting full primary refresh. [2025-11-04T17:12:05.773294] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-03. [2025-11-04T17:12:05.927441] [INFO ] [REMOTE ] AMC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:12:05.931933] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-11-04T17:12:06.567064] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-04. [2025-11-04T17:12:06.872828] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T17:12:07.367090] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-07. [2025-11-04T17:12:07.875970] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T17:12:08.167057] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-01. [2025-11-04T17:12:08.721077] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T17:12:08.967115] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-04. [2025-11-04T17:12:09.767088] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-08. [2025-11-04T17:12:10.242291] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T17:12:10.371478] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-01 (2.20 seconds). [2025-11-04T17:12:10.567094] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-05. [2025-11-04T17:12:11.192915] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-11-04T17:12:11.367104] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-09. [2025-11-04T17:12:12.167023] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-05. [2025-11-04T17:12:12.216373] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-11-04T17:12:12.640857] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-11-04T17:12:12.967079] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-06. [2025-11-04T17:12:13.355692] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T17:12:13.767041] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-10. [2025-11-04T17:12:14.068027] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T17:12:14.567129] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-07. [2025-11-04T17:12:15.367085] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-06. [2025-11-04T17:12:15.411981] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-11-04T17:12:15.919621] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T17:12:16.167061] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-02. [2025-11-04T17:12:16.493816] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T17:12:16.967034] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-08. [2025-11-04T17:12:17.767089] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-07. [2025-11-04T17:12:18.006080] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-11-04T17:12:18.337055] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-11-04T17:12:18.567144] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-03. [2025-11-04T17:12:18.944686] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T17:12:19.367105] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-11. [2025-11-04T17:12:20.167187] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-08. [2025-11-04T17:12:20.293795] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-11-04T17:12:20.695012] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-11 (1.33 seconds). [2025-11-04T17:12:20.697160] [INFO ] [REMOTE ] AM: Full fetch complete (75644 records). [2025-11-04T17:12:20.697275] [DEBUG] [VALIDATOR ] AM: Validating primary data. [2025-11-04T17:12:20.972293] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-04. [2025-11-04T17:12:21.103810] [DEBUG] [VALIDATOR ] AM: All expected trading days are present. [2025-11-04T17:12:21.103892] [DEBUG] [VALIDATOR ] AM: Primary data validation passed. [2025-11-04T17:12:21.104686] [INFO ] [MAIN ] AM: Fetched primary data is valid. [2025-11-04T17:12:21.104751] [DEBUG] [LOCAL ] AM: Saving primary data. [2025-11-04T17:12:21.106168] [DEBUG] [LOCAL ] AM: Trimmed 75644 records to 51752 (keeping 200 days of data) [2025-11-04T17:12:21.416120] [DEBUG] [LOCAL ] AM: Primary data saved (51752 records). [2025-11-04T17:12:21.416365] [INFO ] [MAIN ] AM: Full history saved. [2025-11-04T17:12:21.417586] [INFO ] [MAIN ] Update Progress: 95/2367 symbols completed. AM took 38.0s. Estimated remaining: 4h 54m 13s. Rate: 7.7 symbols/min [2025-11-04T17:12:21.515176] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-11-04T17:12:21.615114] [INFO ] [MAIN ] AMCR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:12:21.615281] [DEBUG] [LOCAL ] AMCR: No existing data file found in primary storage. [2025-11-04T17:12:21.615340] [WARN ] [MAIN ] AMCR: No existing data found, performing full refresh instead. [2025-11-04T17:12:21.615444] [INFO ] [MAIN ] AMCR: Wiping existing primary data before full refresh. [2025-11-04T17:12:21.615485] [INFO ] [LOCAL ] AMCR: Starting deletion of primary data. [2025-11-04T17:12:21.615524] [DEBUG] [LOCAL ] AMCR: No primary data file found to delete. [2025-11-04T17:12:21.615554] [INFO ] [MAIN ] AMCR: Starting full primary refresh. [2025-11-04T17:12:21.771556] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-09. [2025-11-04T17:12:21.975631] [INFO ] [REMOTE ] AMCR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:12:22.567077] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-01. [2025-11-04T17:12:23.143058] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-11-04T17:12:23.277092] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-11-04T17:12:23.367085] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-10. [2025-11-04T17:12:24.166312] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-11-04T17:12:24.176075] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-05. [2025-11-04T17:12:24.522518] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T17:12:24.967056] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-02. [2025-11-04T17:12:25.767048] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-09. [2025-11-04T17:12:26.211696] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-11-04T17:12:26.253628] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T17:12:26.567085] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-06. [2025-11-04T17:12:26.727911] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T17:12:27.367101] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-10. [2025-11-04T17:12:28.167054] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-11. [2025-11-04T17:12:28.361924] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-11-04T17:12:28.462559] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T17:12:28.967074] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-03. [2025-11-04T17:12:29.298425] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T17:12:29.300766] [INFO ] [REMOTE ] AMBP: Full fetch complete (64868 records). [2025-11-04T17:12:29.301042] [DEBUG] [VALIDATOR ] AMBP: Validating primary data. [2025-11-04T17:12:29.734419] [DEBUG] [VALIDATOR ] AMBP: All expected trading days are present. [2025-11-04T17:12:29.734553] [DEBUG] [VALIDATOR ] AMBP: Primary data validation passed. [2025-11-04T17:12:29.735729] [INFO ] [MAIN ] AMBP: Fetched primary data is valid. [2025-11-04T17:12:29.735823] [DEBUG] [LOCAL ] AMBP: Saving primary data. [2025-11-04T17:12:29.738077] [DEBUG] [LOCAL ] AMBP: Trimmed 64868 records to 44427 (keeping 200 days of data) [2025-11-04T17:12:29.790450] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-11. [2025-11-04T17:12:30.025141] [DEBUG] [LOCAL ] AMBP: Primary data saved (44427 records). [2025-11-04T17:12:30.025360] [INFO ] [MAIN ] AMBP: Full history saved. [2025-11-04T17:12:30.025466] [INFO ] [MAIN ] Update Progress: 96/2367 symbols completed. AMBP took 32.0s. Estimated remaining: 4h 54m 25s. Rate: 7.7 symbols/min [2025-11-04T17:12:30.270030] [INFO ] [MAIN ] AME: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:12:30.270436] [DEBUG] [LOCAL ] AME: No existing data file found in primary storage. [2025-11-04T17:12:30.270512] [WARN ] [MAIN ] AME: No existing data found, performing full refresh instead. [2025-11-04T17:12:30.270710] [INFO ] [MAIN ] AME: Wiping existing primary data before full refresh. [2025-11-04T17:12:30.270791] [INFO ] [LOCAL ] AME: Starting deletion of primary data. [2025-11-04T17:12:30.270872] [DEBUG] [LOCAL ] AME: No primary data file found to delete. [2025-11-04T17:12:30.271053] [INFO ] [MAIN ] AME: Starting full primary refresh. [2025-11-04T17:12:30.572604] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-07. [2025-11-04T17:12:30.737004] [INFO ] [REMOTE ] AME: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:12:30.740565] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-11 (0.95 seconds). [2025-11-04T17:12:30.742770] [INFO ] [REMOTE ] AMBC: Full fetch complete (51787 records). [2025-11-04T17:12:30.745039] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-11-04T17:12:30.745147] [DEBUG] [VALIDATOR ] AMBC: Validating primary data. [2025-11-04T17:12:31.154211] [DEBUG] [VALIDATOR ] AMBC: All expected trading days are present. [2025-11-04T17:12:31.154294] [DEBUG] [VALIDATOR ] AMBC: Primary data validation passed. [2025-11-04T17:12:31.154847] [INFO ] [MAIN ] AMBC: Fetched primary data is valid. [2025-11-04T17:12:31.154910] [DEBUG] [LOCAL ] AMBC: Saving primary data. [2025-11-04T17:12:31.156035] [DEBUG] [LOCAL ] AMBC: Trimmed 51787 records to 35091 (keeping 200 days of data) [2025-11-04T17:12:31.357665] [DEBUG] [LOCAL ] AMBC: Primary data saved (35091 records). [2025-11-04T17:12:31.357809] [INFO ] [MAIN ] AMBC: Full history saved. [2025-11-04T17:12:31.358781] [INFO ] [MAIN ] Update Progress: 97/2367 symbols completed. AMBC took 35.8s. Estimated remaining: 4h 51m 46s. Rate: 7.8 symbols/min [2025-11-04T17:12:31.373205] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-04. [2025-11-04T17:12:31.539204] [INFO ] [MAIN ] AMG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:12:31.559695] [DEBUG] [LOCAL ] AMG: Loaded 19579 existing records from primary storage. [2025-11-04T17:12:31.559949] [INFO ] [MAIN ] AMG: Loaded 19579 existing records. [2025-11-04T17:12:31.742589] [WARN ] [MAIN ] AMG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:12:31.743053] [INFO ] [MAIN ] AMG: Wiping existing primary data before full refresh. [2025-11-04T17:12:31.743128] [INFO ] [LOCAL ] AMG: Starting deletion of primary data. [2025-11-04T17:12:31.743359] [DEBUG] [LOCAL ] AMG: Deleted primary data file. [2025-11-04T17:12:31.743416] [INFO ] [MAIN ] AMG: Starting full primary refresh. [2025-11-04T17:12:32.099081] [INFO ] [REMOTE ] AMG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:12:32.167037] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-01. [2025-11-04T17:12:32.802344] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-11-04T17:12:32.967017] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-08. [2025-11-04T17:12:33.016549] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-11-04T17:12:33.044162] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T17:12:33.767089] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-05. [2025-11-04T17:12:34.567108] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-01. [2025-11-04T17:12:34.932428] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-11-04T17:12:35.060269] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T17:12:35.367073] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-09. [2025-11-04T17:12:36.011453] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T17:12:36.167044] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-02. [2025-11-04T17:12:36.967132] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-02. [2025-11-04T17:12:37.452630] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-11-04T17:12:37.629320] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-11-04T17:12:37.767047] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-03. [2025-11-04T17:12:37.854063] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T17:12:38.567035] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-03. [2025-11-04T17:12:39.219518] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-11-04T17:12:39.367050] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-04. [2025-11-04T17:12:40.118902] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T17:12:40.167021] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-06. [2025-11-04T17:12:40.729987] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T17:12:40.967056] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-04. [2025-11-04T17:12:41.530286] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T17:12:41.761000] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-11-04T17:12:41.767116] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-10. [2025-11-04T17:12:42.567265] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-05. [2025-11-04T17:12:43.367011] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-05. [2025-11-04T17:12:43.471168] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T17:12:43.797810] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-10 (2.03 seconds). [2025-11-04T17:12:44.167070] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-06. [2025-11-04T17:12:44.761193] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-11-04T17:12:44.872947] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T17:12:44.967014] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-07. [2025-11-04T17:12:45.714334] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-11-04T17:12:45.767079] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-08. [2025-11-04T17:12:46.561074] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T17:12:46.566990] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-09. [2025-11-04T17:12:47.248229] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T17:12:47.367047] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-07. [2025-11-04T17:12:48.167077] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-10. [2025-11-04T17:12:48.981690] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-11. [2025-11-04T17:12:48.985020] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-11-04T17:12:49.081009] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-11-04T17:12:49.767042] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-06. [2025-11-04T17:12:50.567050] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-08. [2025-11-04T17:12:51.109870] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-11 (2.13 seconds). [2025-11-04T17:12:51.112514] [INFO ] [REMOTE ] AMC: Full fetch complete (77930 records). [2025-11-04T17:12:51.112731] [DEBUG] [VALIDATOR ] AMC: Validating primary data. [2025-11-04T17:12:51.390686] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-11. [2025-11-04T17:12:51.575463] [DEBUG] [VALIDATOR ] AMC: All expected trading days are present. [2025-11-04T17:12:51.575798] [DEBUG] [VALIDATOR ] AMC: Primary data validation passed. [2025-11-04T17:12:51.576993] [INFO ] [MAIN ] AMC: Fetched primary data is valid. [2025-11-04T17:12:51.577107] [DEBUG] [LOCAL ] AMC: Saving primary data. [2025-11-04T17:12:51.606136] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-11-04T17:12:51.616347] [DEBUG] [LOCAL ] AMC: Trimmed 77930 records to 53772 (keeping 200 days of data) [2025-11-04T17:12:51.953050] [DEBUG] [LOCAL ] AMC: Primary data saved (53772 records). [2025-11-04T17:12:51.953301] [INFO ] [MAIN ] AMC: Full history saved. [2025-11-04T17:12:51.953405] [INFO ] [MAIN ] Update Progress: 98/2367 symbols completed. AMC took 46.7s. Estimated remaining: 4h 56m 37s. Rate: 7.6 symbols/min [2025-11-04T17:12:52.162292] [INFO ] [MAIN ] AMH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:12:52.167062] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-07. [2025-11-04T17:12:52.166925] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T17:12:52.170965] [INFO ] [REMOTE ] AMG: Full fetch complete (29359 records). [2025-11-04T17:12:52.171415] [DEBUG] [VALIDATOR ] AMG: Validating primary data. [2025-11-04T17:12:52.172604] [FATAL] [VALIDATOR ] AMG: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:12:52.172936] [FATAL] [MAIN ] AMG: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:12:52.173111] [WARN ] [MAIN ] AMG: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:12:52.173265] [INFO ] [LOCAL ] AMG: Starting deletion of primary data. [2025-11-04T17:12:52.173450] [DEBUG] [LOCAL ] AMG: No primary data file found to delete. [2025-11-04T17:12:52.173568] [INFO ] [LOCAL ] AMG: Starting deletion of derivative data. [2025-11-04T17:12:52.174057] [DEBUG] [LOCAL ] AMG: Deleted derivative data file. [2025-11-04T17:12:52.174628] [INFO ] [MAIN ] Update Progress: 99/2367 symbols completed. AMG took 20.8s. Estimated remaining: 4h 53m 34s. Rate: 7.7 symbols/min [2025-11-04T17:12:52.326257] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-11-04T17:12:52.422985] [INFO ] [MAIN ] AMN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:12:52.441825] [DEBUG] [LOCAL ] AMH: Loaded 51849 existing records from primary storage. [2025-11-04T17:12:52.442361] [INFO ] [MAIN ] AMH: Loaded 51849 existing records. [2025-11-04T17:12:52.702142] [DEBUG] [LOCAL ] AMN: Loaded 41342 existing records from primary storage. [2025-11-04T17:12:52.700436] [WARN ] [MAIN ] AMH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:12:52.702703] [INFO ] [MAIN ] AMN: Loaded 41342 existing records. [2025-11-04T17:12:52.708098] [INFO ] [MAIN ] AMH: Wiping existing primary data before full refresh. [2025-11-04T17:12:52.771619] [INFO ] [LOCAL ] AMH: Starting deletion of primary data. [2025-11-04T17:12:52.808060] [DEBUG] [LOCAL ] AMH: Deleted primary data file. [2025-11-04T17:12:52.811634] [INFO ] [MAIN ] AMH: Starting full primary refresh. [2025-11-04T17:12:52.971311] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-09. [2025-11-04T17:12:53.012728] [WARN ] [MAIN ] AMN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:12:53.032337] [INFO ] [MAIN ] AMN: Wiping existing primary data before full refresh. [2025-11-04T17:12:53.048113] [INFO ] [LOCAL ] AMN: Starting deletion of primary data. [2025-11-04T17:12:53.071089] [DEBUG] [LOCAL ] AMN: Deleted primary data file. [2025-11-04T17:12:53.077166] [INFO ] [MAIN ] AMN: Starting full primary refresh. [2025-11-04T17:12:53.532129] [INFO ] [REMOTE ] AMH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:12:53.757471] [INFO ] [REMOTE ] AMN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:12:53.772155] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-01. [2025-11-04T17:12:53.793363] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-11-04T17:12:54.567085] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-08. [2025-11-04T17:12:55.307546] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-11-04T17:12:55.367053] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-02. [2025-11-04T17:12:55.571464] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-09 (2.60 seconds). [2025-11-04T17:12:55.988323] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T17:12:56.167045] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-09. [2025-11-04T17:12:56.693288] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T17:12:56.967048] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-10. [2025-11-04T17:12:57.528596] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T17:12:57.767056] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-10. [2025-11-04T17:12:58.567063] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-03. [2025-11-04T17:12:58.830315] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-11-04T17:12:59.224444] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T17:12:59.367073] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-11. [2025-11-04T17:12:59.927018] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T17:13:00.167143] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-11. [2025-11-04T17:13:00.484398] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-11 (0.32 seconds). [2025-11-04T17:13:00.487162] [INFO ] [REMOTE ] AME: Full fetch complete (70461 records). [2025-11-04T17:13:00.487606] [DEBUG] [VALIDATOR ] AME: Validating primary data. [2025-11-04T17:13:00.911266] [DEBUG] [VALIDATOR ] AME: All expected trading days are present. [2025-11-04T17:13:00.911387] [DEBUG] [VALIDATOR ] AME: Primary data validation passed. [2025-11-04T17:13:00.912383] [INFO ] [MAIN ] AME: Fetched primary data is valid. [2025-11-04T17:13:00.912459] [DEBUG] [LOCAL ] AME: Saving primary data. [2025-11-04T17:13:00.914479] [DEBUG] [LOCAL ] AME: Trimmed 70461 records to 48322 (keeping 200 days of data) [2025-11-04T17:13:00.973588] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-01. [2025-11-04T17:13:01.245822] [DEBUG] [LOCAL ] AME: Primary data saved (48322 records). [2025-11-04T17:13:01.246019] [INFO ] [MAIN ] AME: Full history saved. [2025-11-04T17:13:01.246108] [INFO ] [MAIN ] Update Progress: 100/2367 symbols completed. AME took 31.2s. Estimated remaining: 4h 53m 56s. Rate: 7.7 symbols/min [2025-11-04T17:13:01.361012] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-11 (1.99 seconds). [2025-11-04T17:13:01.389815] [INFO ] [REMOTE ] AMCR: Full fetch complete (77843 records). [2025-11-04T17:13:01.411166] [DEBUG] [VALIDATOR ] AMCR: Validating primary data. [2025-11-04T17:13:01.438431] [INFO ] [MAIN ] AMP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:13:01.438724] [DEBUG] [LOCAL ] AMP: No existing data file found in primary storage. [2025-11-04T17:13:01.438783] [WARN ] [MAIN ] AMP: No existing data found, performing full refresh instead. [2025-11-04T17:13:01.438981] [INFO ] [MAIN ] AMP: Wiping existing primary data before full refresh. [2025-11-04T17:13:01.439038] [INFO ] [LOCAL ] AMP: Starting deletion of primary data. [2025-11-04T17:13:01.439083] [DEBUG] [LOCAL ] AMP: No primary data file found to delete. [2025-11-04T17:13:01.439124] [INFO ] [MAIN ] AMP: Starting full primary refresh. [2025-11-04T17:13:01.777360] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-04. [2025-11-04T17:13:02.173629] [INFO ] [REMOTE ] AMP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:13:02.306586] [DEBUG] [VALIDATOR ] AMCR: All expected trading days are present. [2025-11-04T17:13:02.306937] [DEBUG] [VALIDATOR ] AMCR: Primary data validation passed. [2025-11-04T17:13:02.308112] [INFO ] [MAIN ] AMCR: Fetched primary data is valid. [2025-11-04T17:13:02.308207] [DEBUG] [LOCAL ] AMCR: Saving primary data. [2025-11-04T17:13:02.331871] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T17:13:02.332085] [DEBUG] [LOCAL ] AMCR: Trimmed 77843 records to 53671 (keeping 200 days of data) [2025-11-04T17:13:02.570887] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-02. [2025-11-04T17:13:02.686097] [DEBUG] [LOCAL ] AMCR: Primary data saved (53671 records). [2025-11-04T17:13:02.686282] [INFO ] [MAIN ] AMCR: Full history saved. [2025-11-04T17:13:02.686378] [INFO ] [MAIN ] Update Progress: 101/2367 symbols completed. AMCR took 41.3s. Estimated remaining: 4h 51m 26s. Rate: 7.8 symbols/min [2025-11-04T17:13:02.867315] [INFO ] [MAIN ] AMPX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:13:02.867609] [DEBUG] [LOCAL ] AMPX: No existing data file found in primary storage. [2025-11-04T17:13:02.867693] [WARN ] [MAIN ] AMPX: No existing data found, performing full refresh instead. [2025-11-04T17:13:02.867831] [INFO ] [MAIN ] AMPX: Wiping existing primary data before full refresh. [2025-11-04T17:13:02.867876] [INFO ] [LOCAL ] AMPX: Starting deletion of primary data. [2025-11-04T17:13:02.867929] [DEBUG] [LOCAL ] AMPX: No primary data file found to delete. [2025-11-04T17:13:02.867964] [INFO ] [MAIN ] AMPX: Starting full primary refresh. [2025-11-04T17:13:03.246210] [INFO ] [REMOTE ] AMPX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:13:03.285725] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T17:13:03.367057] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-01. [2025-11-04T17:13:04.175154] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-05. [2025-11-04T17:13:04.193199] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-11-04T17:13:04.581797] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T17:13:04.967143] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-01. [2025-11-04T17:13:05.680668] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-11-04T17:13:05.767085] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-02. [2025-11-04T17:13:06.567043] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-06. [2025-11-04T17:13:06.604679] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-11-04T17:13:06.850507] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T17:13:07.367087] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-02. [2025-11-04T17:13:07.907909] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T17:13:08.167128] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-03. [2025-11-04T17:13:08.836309] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-11-04T17:13:08.967081] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-07. [2025-11-04T17:13:09.250612] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T17:13:09.767162] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-03. [2025-11-04T17:13:10.330946] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T17:13:10.567085] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-04. [2025-11-04T17:13:11.004041] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T17:13:11.367049] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-04. [2025-11-04T17:13:11.664310] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T17:13:12.167038] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-03. [2025-11-04T17:13:12.795589] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T17:13:12.967067] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-05. [2025-11-04T17:13:13.523803] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T17:13:13.767090] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-08. [2025-11-04T17:13:14.349010] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-11-04T17:13:14.567073] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-06. [2025-11-04T17:13:15.162290] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T17:13:15.367039] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-09. [2025-11-04T17:13:15.596568] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T17:13:16.167083] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-04. [2025-11-04T17:13:16.856351] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-11-04T17:13:16.967010] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-10. [2025-11-04T17:13:17.736482] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-11-04T17:13:17.767051] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-07. [2025-11-04T17:13:18.475815] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T17:13:18.567119] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-05. [2025-11-04T17:13:18.934879] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T17:13:19.367097] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-08. [2025-11-04T17:13:19.572284] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-11-04T17:13:20.167031] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-11. [2025-11-04T17:13:20.530094] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T17:13:20.967045] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-06. [2025-11-04T17:13:21.238282] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T17:13:21.239937] [INFO ] [REMOTE ] AMH: Full fetch complete (74794 records). [2025-11-04T17:13:21.240027] [DEBUG] [VALIDATOR ] AMH: Validating primary data. [2025-11-04T17:13:21.240755] [FATAL] [VALIDATOR ] AMH: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:13:21.240839] [FATAL] [MAIN ] AMH: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:13:21.240880] [WARN ] [MAIN ] AMH: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:13:21.240940] [INFO ] [LOCAL ] AMH: Starting deletion of primary data. [2025-11-04T17:13:21.241001] [DEBUG] [LOCAL ] AMH: No primary data file found to delete. [2025-11-04T17:13:21.241036] [INFO ] [LOCAL ] AMH: Starting deletion of derivative data. [2025-11-04T17:13:21.241195] [DEBUG] [LOCAL ] AMH: Deleted derivative data file. [2025-11-04T17:13:21.241285] [INFO ] [MAIN ] Update Progress: 102/2367 symbols completed. AMH took 29.3s. Estimated remaining: 4h 55m 19s. Rate: 7.7 symbols/min [2025-11-04T17:13:21.408091] [INFO ] [MAIN ] AMPY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:13:21.444207] [DEBUG] [LOCAL ] AMPY: Loaded 39267 existing records from primary storage. [2025-11-04T17:13:21.444487] [INFO ] [MAIN ] AMPY: Loaded 39267 existing records. [2025-11-04T17:13:21.616193] [WARN ] [MAIN ] AMPY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:13:21.616389] [INFO ] [MAIN ] AMPY: Wiping existing primary data before full refresh. [2025-11-04T17:13:21.616435] [INFO ] [LOCAL ] AMPY: Starting deletion of primary data. [2025-11-04T17:13:21.616630] [DEBUG] [LOCAL ] AMPY: Deleted primary data file. [2025-11-04T17:13:21.616721] [INFO ] [MAIN ] AMPY: Starting full primary refresh. [2025-11-04T17:13:21.775708] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-09. [2025-11-04T17:13:21.986603] [INFO ] [REMOTE ] AMPY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:13:22.080252] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T17:13:22.567102] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-05. [2025-11-04T17:13:22.988926] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T17:13:23.367017] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-10. [2025-11-04T17:13:24.167042] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-07. [2025-11-04T17:13:24.205427] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-11-04T17:13:24.502941] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T17:13:24.967037] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-01. [2025-11-04T17:13:25.418826] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T17:13:25.767006] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-06. [2025-11-04T17:13:25.821019] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-11-04T17:13:26.567081] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-02. [2025-11-04T17:13:27.336186] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-11-04T17:13:27.367017] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-07. [2025-11-04T17:13:27.494733] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-11-04T17:13:28.167085] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-11. [2025-11-04T17:13:28.967034] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-03. [2025-11-04T17:13:29.140754] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T17:13:29.142479] [INFO ] [REMOTE ] AMP: Full fetch complete (47758 records). [2025-11-04T17:13:29.142562] [DEBUG] [VALIDATOR ] AMP: Validating primary data. [2025-11-04T17:13:29.568015] [DEBUG] [VALIDATOR ] AMP: All expected trading days are present. [2025-11-04T17:13:29.568121] [DEBUG] [VALIDATOR ] AMP: Primary data validation passed. [2025-11-04T17:13:29.568671] [INFO ] [MAIN ] AMP: Fetched primary data is valid. [2025-11-04T17:13:29.568842] [DEBUG] [LOCAL ] AMP: Saving primary data. [2025-11-04T17:13:29.570178] [DEBUG] [LOCAL ] AMP: Trimmed 47758 records to 31570 (keeping 200 days of data) [2025-11-04T17:13:29.712033] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-11-04T17:13:29.767420] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-08. [2025-11-04T17:13:29.808976] [DEBUG] [LOCAL ] AMP: Primary data saved (31570 records). [2025-11-04T17:13:29.809085] [INFO ] [MAIN ] AMP: Full history saved. [2025-11-04T17:13:29.809151] [INFO ] [MAIN ] Update Progress: 103/2367 symbols completed. AMP took 28.6s. Estimated remaining: 4h 55m 28s. Rate: 7.7 symbols/min [2025-11-04T17:13:29.976935] [INFO ] [MAIN ] AMR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:13:29.996785] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T17:13:30.027081] [DEBUG] [LOCAL ] AMR: Loaded 28188 existing records from primary storage. [2025-11-04T17:13:30.027327] [INFO ] [MAIN ] AMR: Loaded 28188 existing records. [2025-11-04T17:13:30.204090] [WARN ] [MAIN ] AMR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:13:30.204285] [INFO ] [MAIN ] AMR: Wiping existing primary data before full refresh. [2025-11-04T17:13:30.204331] [INFO ] [LOCAL ] AMR: Starting deletion of primary data. [2025-11-04T17:13:30.204519] [DEBUG] [LOCAL ] AMR: Deleted primary data file. [2025-11-04T17:13:30.204574] [INFO ] [MAIN ] AMR: Starting full primary refresh. [2025-11-04T17:13:30.548465] [INFO ] [REMOTE ] AMR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:13:30.567005] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-01. [2025-11-04T17:13:31.372695] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-08. [2025-11-04T17:13:31.378223] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-11-04T17:13:31.617164] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-11-04T17:13:32.167077] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-02. [2025-11-04T17:13:32.748413] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T17:13:32.967036] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-09. [2025-11-04T17:13:33.039166] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T17:13:33.767040] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-04. [2025-11-04T17:13:34.161117] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T17:13:34.567077] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-10. [2025-11-04T17:13:34.838366] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T17:13:35.367216] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-05. [2025-11-04T17:13:35.972392] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T17:13:36.167026] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-11. [2025-11-04T17:13:36.339971] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-11-04T17:13:36.967042] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-03. [2025-11-04T17:13:37.089354] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T17:13:37.091060] [INFO ] [REMOTE ] AMN: Full fetch complete (60237 records). [2025-11-04T17:13:37.091209] [DEBUG] [VALIDATOR ] AMN: Validating primary data. [2025-11-04T17:13:37.466515] [DEBUG] [VALIDATOR ] AMN: All expected trading days are present. [2025-11-04T17:13:37.466601] [DEBUG] [VALIDATOR ] AMN: Primary data validation passed. [2025-11-04T17:13:37.467270] [INFO ] [MAIN ] AMN: Fetched primary data is valid. [2025-11-04T17:13:37.467333] [DEBUG] [LOCAL ] AMN: Saving primary data. [2025-11-04T17:13:37.468570] [DEBUG] [LOCAL ] AMN: Trimmed 60237 records to 41396 (keeping 200 days of data) [2025-11-04T17:13:37.701591] [DEBUG] [LOCAL ] AMN: Primary data saved (41396 records). [2025-11-04T17:13:37.701738] [INFO ] [MAIN ] AMN: Full history saved. [2025-11-04T17:13:37.703426] [INFO ] [MAIN ] Update Progress: 104/2367 symbols completed. AMN took 45.5s. Estimated remaining: 4h 55m 21s. Rate: 7.7 symbols/min [2025-11-04T17:13:37.776457] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-09. [2025-11-04T17:13:37.878576] [INFO ] [MAIN ] AMRC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:13:37.920927] [DEBUG] [LOCAL ] AMRC: Loaded 37937 existing records from primary storage. [2025-11-04T17:13:37.921364] [INFO ] [MAIN ] AMRC: Loaded 37937 existing records. [2025-11-04T17:13:38.111100] [WARN ] [MAIN ] AMRC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:13:38.124813] [INFO ] [MAIN ] AMRC: Wiping existing primary data before full refresh. [2025-11-04T17:13:38.125193] [INFO ] [LOCAL ] AMRC: Starting deletion of primary data. [2025-11-04T17:13:38.127037] [DEBUG] [LOCAL ] AMRC: Deleted primary data file. [2025-11-04T17:13:38.126947] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-11-04T17:13:38.127289] [INFO ] [MAIN ] AMRC: Starting full primary refresh. [2025-11-04T17:13:38.487037] [INFO ] [REMOTE ] AMRC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:13:38.567033] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-01. [2025-11-04T17:13:39.367039] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-06. [2025-11-04T17:13:39.644006] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-11-04T17:13:39.841616] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T17:13:40.167052] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-10. [2025-11-04T17:13:40.391392] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T17:13:40.967026] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-02. [2025-11-04T17:13:41.767102] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-07. [2025-11-04T17:13:42.209269] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T17:13:42.290365] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-11-04T17:13:42.567085] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-04. [2025-11-04T17:13:42.761320] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-11-04T17:13:43.367036] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-11. [2025-11-04T17:13:43.596478] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-11-04T17:13:44.167057] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-05. [2025-11-04T17:13:44.967049] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-08. [2025-11-04T17:13:45.014354] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-11 (1.65 seconds). [2025-11-04T17:13:45.016455] [INFO ] [REMOTE ] AMPX: Full fetch complete (75376 records). [2025-11-04T17:13:45.016564] [DEBUG] [VALIDATOR ] AMPX: Validating primary data. [2025-11-04T17:13:45.369514] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T17:13:45.465931] [DEBUG] [VALIDATOR ] AMPX: All expected trading days are present. [2025-11-04T17:13:45.466018] [DEBUG] [VALIDATOR ] AMPX: Primary data validation passed. [2025-11-04T17:13:45.466770] [INFO ] [MAIN ] AMPX: Fetched primary data is valid. [2025-11-04T17:13:45.466833] [DEBUG] [LOCAL ] AMPX: Saving primary data. [2025-11-04T17:13:45.468415] [DEBUG] [LOCAL ] AMPX: Trimmed 75376 records to 52687 (keeping 200 days of data) [2025-11-04T17:13:45.768701] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-03. [2025-11-04T17:13:45.788854] [DEBUG] [LOCAL ] AMPX: Primary data saved (52687 records). [2025-11-04T17:13:45.789015] [INFO ] [MAIN ] AMPX: Full history saved. [2025-11-04T17:13:45.789103] [INFO ] [MAIN ] Update Progress: 105/2367 symbols completed. AMPX took 43.1s. Estimated remaining: 4h 55m 19s. Rate: 7.7 symbols/min [2025-11-04T17:13:45.963943] [INFO ] [MAIN ] AMRZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:13:45.964269] [DEBUG] [LOCAL ] AMRZ: No existing data file found in primary storage. [2025-11-04T17:13:45.964319] [WARN ] [MAIN ] AMRZ: No existing data found, performing full refresh instead. [2025-11-04T17:13:45.964451] [INFO ] [MAIN ] AMRZ: Wiping existing primary data before full refresh. [2025-11-04T17:13:45.964492] [INFO ] [LOCAL ] AMRZ: Starting deletion of primary data. [2025-11-04T17:13:45.964532] [DEBUG] [LOCAL ] AMRZ: No primary data file found to delete. [2025-11-04T17:13:45.964563] [INFO ] [MAIN ] AMRZ: Starting full primary refresh. [2025-11-04T17:13:46.329690] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-11-04T17:13:46.349562] [INFO ] [REMOTE ] AMRZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:13:46.567038] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-06. [2025-11-04T17:13:46.912453] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T17:13:47.367026] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-01. [2025-11-04T17:13:47.431237] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T17:13:47.519286] [DEBUG] [REMOTE ] AMRZ: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:13:47.519517] [WARN ] [REMOTE ] AMRZ: No data returned for month 2025-01 [2025-11-04T17:13:47.519605] [FATAL] [REMOTE ] AMRZ: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:13:47.519707] [FATAL] [MAIN ] AMRZ: Failed to fetch primary data. [2025-11-04T17:13:47.519775] [WARN ] [MAIN ] AMRZ: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:13:47.519848] [INFO ] [LOCAL ] AMRZ: Starting deletion of primary data. [2025-11-04T17:13:47.519910] [DEBUG] [LOCAL ] AMRZ: No primary data file found to delete. [2025-11-04T17:13:47.519947] [INFO ] [LOCAL ] AMRZ: Starting deletion of derivative data. [2025-11-04T17:13:47.519990] [DEBUG] [LOCAL ] AMRZ: No derivative data file found to delete. [2025-11-04T17:13:47.520038] [INFO ] [MAIN ] Update Progress: 106/2367 symbols completed. AMRZ took 1.7s. Estimated remaining: 4h 53m 1s. Rate: 7.7 symbols/min [2025-11-04T17:13:47.690422] [INFO ] [MAIN ] AMT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:13:47.740167] [DEBUG] [LOCAL ] AMT: Loaded 52404 existing records from primary storage. [2025-11-04T17:13:47.740528] [INFO ] [MAIN ] AMT: Loaded 52404 existing records. [2025-11-04T17:13:47.914070] [WARN ] [MAIN ] AMT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:13:47.914266] [INFO ] [MAIN ] AMT: Wiping existing primary data before full refresh. [2025-11-04T17:13:47.914311] [INFO ] [LOCAL ] AMT: Starting deletion of primary data. [2025-11-04T17:13:47.914617] [DEBUG] [LOCAL ] AMT: Deleted primary data file. [2025-11-04T17:13:47.914706] [INFO ] [MAIN ] AMT: Starting full primary refresh. [2025-11-04T17:13:48.172254] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-07. [2025-11-04T17:13:48.260933] [INFO ] [REMOTE ] AMT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:13:48.967028] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-04. [2025-11-04T17:13:49.216056] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T17:13:49.767083] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-01. [2025-11-04T17:13:50.287365] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T17:13:50.567055] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-09. [2025-11-04T17:13:51.117505] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T17:13:51.367012] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-05. [2025-11-04T17:13:51.545023] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T17:13:52.167102] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-10. [2025-11-04T17:13:52.509277] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T17:13:52.967086] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-08. [2025-11-04T17:13:53.458513] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T17:13:53.767053] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-06. [2025-11-04T17:13:54.125791] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T17:13:54.561978] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T17:13:54.568448] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-02. [2025-11-04T17:13:55.367033] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-11. [2025-11-04T17:13:56.110506] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-11-04T17:13:56.167009] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-03. [2025-11-04T17:13:56.541860] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-11 (1.17 seconds). [2025-11-04T17:13:56.543472] [INFO ] [REMOTE ] AMPY: Full fetch complete (54050 records). [2025-11-04T17:13:56.543572] [DEBUG] [VALIDATOR ] AMPY: Validating primary data. [2025-11-04T17:13:56.922038] [DEBUG] [VALIDATOR ] AMPY: All expected trading days are present. [2025-11-04T17:13:56.922117] [DEBUG] [VALIDATOR ] AMPY: Primary data validation passed. [2025-11-04T17:13:56.922771] [INFO ] [MAIN ] AMPY: Fetched primary data is valid. [2025-11-04T17:13:56.922833] [DEBUG] [LOCAL ] AMPY: Saving primary data. [2025-11-04T17:13:56.923913] [DEBUG] [LOCAL ] AMPY: Trimmed 54050 records to 39240 (keeping 200 days of data) [2025-11-04T17:13:56.967621] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-07. [2025-11-04T17:13:57.141794] [DEBUG] [LOCAL ] AMPY: Primary data saved (39240 records). [2025-11-04T17:13:57.141961] [INFO ] [MAIN ] AMPY: Full history saved. [2025-11-04T17:13:57.142061] [INFO ] [MAIN ] Update Progress: 107/2367 symbols completed. AMPY took 35.9s. Estimated remaining: 4h 53m 32s. Rate: 7.7 symbols/min [2025-11-04T17:13:57.308751] [INFO ] [MAIN ] AMTB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:13:57.308952] [DEBUG] [LOCAL ] AMTB: No existing data file found in primary storage. [2025-11-04T17:13:57.308989] [WARN ] [MAIN ] AMTB: No existing data found, performing full refresh instead. [2025-11-04T17:13:57.309093] [INFO ] [MAIN ] AMTB: Wiping existing primary data before full refresh. [2025-11-04T17:13:57.309132] [INFO ] [LOCAL ] AMTB: Starting deletion of primary data. [2025-11-04T17:13:57.309171] [DEBUG] [LOCAL ] AMTB: No primary data file found to delete. [2025-11-04T17:13:57.309201] [INFO ] [MAIN ] AMTB: Starting full primary refresh. [2025-11-04T17:13:57.661417] [INFO ] [REMOTE ] AMTB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:13:57.767018] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-01. [2025-11-04T17:13:58.020501] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-11-04T17:13:58.191439] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-11-04T17:13:58.567078] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-04. [2025-11-04T17:13:58.758022] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T17:13:59.367039] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-08. [2025-11-04T17:14:00.087738] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-11-04T17:14:00.167067] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-09. [2025-11-04T17:14:00.673174] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-11-04T17:14:00.967038] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-09. [2025-11-04T17:14:01.192301] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-11-04T17:14:01.767042] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-05. [2025-11-04T17:14:02.043323] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T17:14:02.567039] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-02. [2025-11-04T17:14:03.264177] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-11-04T17:14:03.288621] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T17:14:03.367037] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-03. [2025-11-04T17:14:04.111109] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T17:14:04.167022] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-04. [2025-11-04T17:14:04.969229] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-10. [2025-11-04T17:14:04.970368] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-11-04T17:14:05.767169] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-10. [2025-11-04T17:14:06.078749] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-11-04T17:14:06.567097] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-06. [2025-11-04T17:14:06.991118] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T17:14:07.367018] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-11. [2025-11-04T17:14:07.916133] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-11-04T17:14:08.167145] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-11. [2025-11-04T17:14:08.297771] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T17:14:08.300192] [INFO ] [REMOTE ] AMRC: Full fetch complete (54676 records). [2025-11-04T17:14:08.300322] [DEBUG] [VALIDATOR ] AMRC: Validating primary data. [2025-11-04T17:14:08.692180] [DEBUG] [VALIDATOR ] AMRC: All expected trading days are present. [2025-11-04T17:14:08.692266] [DEBUG] [VALIDATOR ] AMRC: Primary data validation passed. [2025-11-04T17:14:08.692887] [INFO ] [MAIN ] AMRC: Fetched primary data is valid. [2025-11-04T17:14:08.692952] [DEBUG] [LOCAL ] AMRC: Saving primary data. [2025-11-04T17:14:08.694127] [DEBUG] [LOCAL ] AMRC: Trimmed 54676 records to 37911 (keeping 200 days of data) [2025-11-04T17:14:08.909193] [DEBUG] [LOCAL ] AMRC: Primary data saved (37911 records). [2025-11-04T17:14:08.909339] [INFO ] [MAIN ] AMRC: Full history saved. [2025-11-04T17:14:08.911085] [INFO ] [MAIN ] Update Progress: 108/2367 symbols completed. AMRC took 31.2s. Estimated remaining: 4h 54m 47s. Rate: 7.7 symbols/min [2025-11-04T17:14:08.975869] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-07. [2025-11-04T17:14:09.092449] [INFO ] [MAIN ] AMTD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:14:09.100318] [DEBUG] [LOCAL ] AMTD: Loaded 4393 existing records from primary storage. [2025-11-04T17:14:09.100536] [INFO ] [MAIN ] AMTD: Loaded 4393 existing records. [2025-11-04T17:14:09.222898] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T17:14:09.274606] [INFO ] [REMOTE ] AMR: Full fetch complete (40735 records). [2025-11-04T17:14:09.293533] [DEBUG] [VALIDATOR ] AMR: Validating primary data. [2025-11-04T17:14:09.297333] [WARN ] [MAIN ] AMTD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:14:09.297890] [INFO ] [MAIN ] AMTD: Wiping existing primary data before full refresh. [2025-11-04T17:14:09.297948] [INFO ] [LOCAL ] AMTD: Starting deletion of primary data. [2025-11-04T17:14:09.328458] [DEBUG] [LOCAL ] AMTD: Deleted primary data file. [2025-11-04T17:14:09.351990] [INFO ] [MAIN ] AMTD: Starting full primary refresh. [2025-11-04T17:14:09.778430] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-05. [2025-11-04T17:14:10.122401] [DEBUG] [VALIDATOR ] AMR: All expected trading days are present. [2025-11-04T17:14:10.126277] [DEBUG] [VALIDATOR ] AMR: Primary data validation passed. [2025-11-04T17:14:10.126999] [INFO ] [MAIN ] AMR: Fetched primary data is valid. [2025-11-04T17:14:10.126198] [INFO ] [REMOTE ] AMTD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:14:10.127094] [DEBUG] [LOCAL ] AMR: Saving primary data. [2025-11-04T17:14:10.128831] [DEBUG] [LOCAL ] AMR: Trimmed 40735 records to 28241 (keeping 200 days of data) [2025-11-04T17:14:10.295835] [DEBUG] [LOCAL ] AMR: Primary data saved (28241 records). [2025-11-04T17:14:10.295964] [INFO ] [MAIN ] AMR: Full history saved. [2025-11-04T17:14:10.296039] [INFO ] [MAIN ] Update Progress: 109/2367 symbols completed. AMR took 40.5s. Estimated remaining: 4h 52m 26s. Rate: 7.7 symbols/min [2025-11-04T17:14:10.466429] [INFO ] [MAIN ] AMTM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:14:10.509434] [DEBUG] [LOCAL ] AMTM: Loaded 49911 existing records from primary storage. [2025-11-04T17:14:10.509833] [INFO ] [MAIN ] AMTM: Loaded 49911 existing records. [2025-11-04T17:14:10.572010] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-01. [2025-11-04T17:14:10.688813] [WARN ] [MAIN ] AMTM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:14:10.689042] [INFO ] [MAIN ] AMTM: Wiping existing primary data before full refresh. [2025-11-04T17:14:10.689121] [INFO ] [LOCAL ] AMTM: Starting deletion of primary data. [2025-11-04T17:14:10.689550] [DEBUG] [LOCAL ] AMTM: Deleted primary data file. [2025-11-04T17:14:10.689623] [INFO ] [MAIN ] AMTM: Starting full primary refresh. [2025-11-04T17:14:10.936248] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T17:14:11.119884] [INFO ] [REMOTE ] AMTM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:14:11.129062] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-07 (2.15 seconds). [2025-11-04T17:14:11.129367] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T17:14:11.367142] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-02. [2025-11-04T17:14:11.820898] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-11-04T17:14:12.167063] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-03. [2025-11-04T17:14:12.574293] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-11-04T17:14:12.967080] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-08. [2025-11-04T17:14:13.767086] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-06. [2025-11-04T17:14:14.455005] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-11-04T17:14:14.567713] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-04. [2025-11-04T17:14:14.570368] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-11-04T17:14:14.861031] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-04 (0.29 seconds). [2025-11-04T17:14:15.367092] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-09. [2025-11-04T17:14:16.167125] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-01. [2025-11-04T17:14:16.854088] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-11-04T17:14:16.967004] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-05. [2025-11-04T17:14:17.296637] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-11-04T17:14:17.663926] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T17:14:17.767026] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-10. [2025-11-04T17:14:18.567068] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-06. [2025-11-04T17:14:19.009561] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-11-04T17:14:19.342776] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-11-04T17:14:19.367002] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-11. [2025-11-04T17:14:19.774685] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T17:14:19.776980] [INFO ] [REMOTE ] AMT: Full fetch complete (76162 records). [2025-11-04T17:14:19.777151] [DEBUG] [VALIDATOR ] AMT: Validating primary data. [2025-11-04T17:14:20.161716] [DEBUG] [VALIDATOR ] AMT: All expected trading days are present. [2025-11-04T17:14:20.161799] [DEBUG] [VALIDATOR ] AMT: Primary data validation passed. [2025-11-04T17:14:20.162522] [INFO ] [MAIN ] AMT: Fetched primary data is valid. [2025-11-04T17:14:20.162564] [DEBUG] [LOCAL ] AMT: Saving primary data. [2025-11-04T17:14:20.163978] [DEBUG] [LOCAL ] AMT: Trimmed 76162 records to 52415 (keeping 200 days of data) [2025-11-04T17:14:20.218777] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-07. [2025-11-04T17:14:20.483082] [DEBUG] [LOCAL ] AMT: Primary data saved (52415 records). [2025-11-04T17:14:20.483210] [INFO ] [MAIN ] AMT: Full history saved. [2025-11-04T17:14:20.485332] [INFO ] [MAIN ] Update Progress: 110/2367 symbols completed. AMT took 33.0s. Estimated remaining: 4h 53m 8s. Rate: 7.7 symbols/min [2025-11-04T17:14:20.594216] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-11-04T17:14:20.656303] [INFO ] [MAIN ] AMWL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:14:20.669443] [DEBUG] [LOCAL ] AMWL: Loaded 10165 existing records from primary storage. [2025-11-04T17:14:20.669682] [INFO ] [MAIN ] AMWL: Loaded 10165 existing records. [2025-11-04T17:14:20.852827] [WARN ] [MAIN ] AMWL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:14:20.853102] [INFO ] [MAIN ] AMWL: Wiping existing primary data before full refresh. [2025-11-04T17:14:20.853155] [INFO ] [LOCAL ] AMWL: Starting deletion of primary data. [2025-11-04T17:14:20.853293] [DEBUG] [LOCAL ] AMWL: Deleted primary data file. [2025-11-04T17:14:20.853329] [INFO ] [MAIN ] AMWL: Starting full primary refresh. [2025-11-04T17:14:20.972117] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-08. [2025-11-04T17:14:21.215260] [INFO ] [REMOTE ] AMWL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:14:21.509375] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-11-04T17:14:21.767031] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-02. [2025-11-04T17:14:22.567027] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-09. [2025-11-04T17:14:23.048016] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T17:14:23.122291] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T17:14:23.367049] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-01. [2025-11-04T17:14:24.145914] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-11-04T17:14:24.167003] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-02. [2025-11-04T17:14:24.684566] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-11-04T17:14:24.967072] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-10. [2025-11-04T17:14:25.450351] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-11-04T17:14:25.767057] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-11. [2025-11-04T17:14:26.416479] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-11 (0.65 seconds). [2025-11-04T17:14:26.417805] [INFO ] [REMOTE ] AMTD: Full fetch complete (5812 records). [2025-11-04T17:14:26.417896] [DEBUG] [VALIDATOR ] AMTD: Validating primary data. [2025-11-04T17:14:26.572137] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-03. [2025-11-04T17:14:26.825039] [DEBUG] [VALIDATOR ] AMTD: All expected trading days are present. [2025-11-04T17:14:26.825111] [DEBUG] [VALIDATOR ] AMTD: Primary data validation passed. [2025-11-04T17:14:26.825267] [INFO ] [MAIN ] AMTD: Fetched primary data is valid. [2025-11-04T17:14:26.825305] [DEBUG] [LOCAL ] AMTD: Saving primary data. [2025-11-04T17:14:26.825684] [DEBUG] [LOCAL ] AMTD: Trimmed 5812 records to 4501 (keeping 200 days of data) [2025-11-04T17:14:26.851046] [DEBUG] [LOCAL ] AMTD: Primary data saved (4501 records). [2025-11-04T17:14:26.851124] [INFO ] [MAIN ] AMTD: Full history saved. [2025-11-04T17:14:26.851191] [INFO ] [MAIN ] Update Progress: 111/2367 symbols completed. AMTD took 17.9s. Estimated remaining: 4h 52m 31s. Rate: 7.7 symbols/min [2025-11-04T17:14:27.027292] [INFO ] [MAIN ] AMX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:14:27.074690] [DEBUG] [LOCAL ] AMX: Loaded 48982 existing records from primary storage. [2025-11-04T17:14:27.074991] [INFO ] [MAIN ] AMX: Loaded 48982 existing records. [2025-11-04T17:14:27.206430] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-11-04T17:14:27.263445] [WARN ] [MAIN ] AMX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:14:27.263667] [INFO ] [MAIN ] AMX: Wiping existing primary data before full refresh. [2025-11-04T17:14:27.263727] [INFO ] [LOCAL ] AMX: Starting deletion of primary data. [2025-11-04T17:14:27.263934] [DEBUG] [LOCAL ] AMX: Deleted primary data file. [2025-11-04T17:14:27.263971] [INFO ] [MAIN ] AMX: Starting full primary refresh. [2025-11-04T17:14:27.368374] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-03. [2025-11-04T17:14:27.623116] [INFO ] [REMOTE ] AMX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:14:28.167045] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-01. [2025-11-04T17:14:28.839427] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-11-04T17:14:28.967063] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-04. [2025-11-04T17:14:29.548473] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T17:14:29.767042] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-07. [2025-11-04T17:14:30.132690] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-11-04T17:14:30.567012] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-04. [2025-11-04T17:14:30.743179] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-11-04T17:14:31.058186] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-11-04T17:14:31.367038] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-08. [2025-11-04T17:14:32.167056] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-02. [2025-11-04T17:14:32.181765] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-11-04T17:14:32.967047] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-09. [2025-11-04T17:14:33.388054] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T17:14:33.585751] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-11-04T17:14:33.767034] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-10. [2025-11-04T17:14:34.534014] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-11-04T17:14:34.567033] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-11. [2025-11-04T17:14:35.201728] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T17:14:35.203108] [INFO ] [REMOTE ] AMTB: Full fetch complete (30007 records). [2025-11-04T17:14:35.203187] [DEBUG] [VALIDATOR ] AMTB: Validating primary data. [2025-11-04T17:14:35.203853] [FATAL] [VALIDATOR ] AMTB: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:14:35.203955] [FATAL] [MAIN ] AMTB: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:14:35.204001] [WARN ] [MAIN ] AMTB: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:14:35.204045] [INFO ] [LOCAL ] AMTB: Starting deletion of primary data. [2025-11-04T17:14:35.204108] [DEBUG] [LOCAL ] AMTB: No primary data file found to delete. [2025-11-04T17:14:35.204256] [INFO ] [LOCAL ] AMTB: Starting deletion of derivative data. [2025-11-04T17:14:35.204332] [DEBUG] [LOCAL ] AMTB: No derivative data file found to delete. [2025-11-04T17:14:35.204408] [INFO ] [MAIN ] Update Progress: 112/2367 symbols completed. AMTB took 38.1s. Estimated remaining: 4h 52m 35s. Rate: 7.7 symbols/min [2025-11-04T17:14:35.372694] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-05. [2025-11-04T17:14:35.393369] [INFO ] [MAIN ] AN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:14:35.432900] [DEBUG] [LOCAL ] AN: Loaded 30690 existing records from primary storage. [2025-11-04T17:14:35.433304] [INFO ] [MAIN ] AN: Loaded 30690 existing records. [2025-11-04T17:14:35.630286] [WARN ] [MAIN ] AN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:14:35.630562] [INFO ] [MAIN ] AN: Wiping existing primary data before full refresh. [2025-11-04T17:14:35.630617] [INFO ] [LOCAL ] AN: Starting deletion of primary data. [2025-11-04T17:14:35.630928] [DEBUG] [LOCAL ] AN: Deleted primary data file. [2025-11-04T17:14:35.631012] [INFO ] [MAIN ] AN: Starting full primary refresh. [2025-11-04T17:14:35.985028] [INFO ] [REMOTE ] AN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:14:36.167058] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-03. [2025-11-04T17:14:36.839681] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-11-04T17:14:36.967035] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-06. [2025-11-04T17:14:37.363399] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T17:14:37.767021] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-04. [2025-11-04T17:14:38.330450] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T17:14:38.567088] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-05. [2025-11-04T17:14:39.235833] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T17:14:39.266685] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-11-04T17:14:39.367036] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-05. [2025-11-04T17:14:40.167036] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-01. [2025-11-04T17:14:40.931305] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-11-04T17:14:40.967008] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-06. [2025-11-04T17:14:41.188516] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T17:14:41.767077] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-07. [2025-11-04T17:14:42.176791] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T17:14:42.567080] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-07. [2025-11-04T17:14:43.231511] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T17:14:43.367040] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-08. [2025-11-04T17:14:43.907906] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T17:14:44.167052] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-02. [2025-11-04T17:14:44.510721] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T17:14:44.967047] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-06. [2025-11-04T17:14:45.433746] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T17:14:45.478884] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T17:14:45.767068] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-03. [2025-11-04T17:14:46.567080] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-09. [2025-11-04T17:14:46.996754] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T17:14:47.367107] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-07. [2025-11-04T17:14:47.943257] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-11-04T17:14:48.067350] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T17:14:48.167068] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-08. [2025-11-04T17:14:48.765739] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-11-04T17:14:48.967079] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-09. [2025-11-04T17:14:49.738768] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-11-04T17:14:49.767019] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-10. [2025-11-04T17:14:50.385807] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-11-04T17:14:50.567088] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-04. [2025-11-04T17:14:51.367059] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-10. [2025-11-04T17:14:51.763664] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T17:14:52.167038] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-11. [2025-11-04T17:14:52.858046] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-11 (0.69 seconds). [2025-11-04T17:14:52.858953] [INFO ] [REMOTE ] AMWL: Full fetch complete (14885 records). [2025-11-04T17:14:52.859019] [DEBUG] [VALIDATOR ] AMWL: Validating primary data. [2025-11-04T17:14:52.970403] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-08. [2025-11-04T17:14:53.172556] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-11-04T17:14:53.260621] [DEBUG] [VALIDATOR ] AMWL: All expected trading days are present. [2025-11-04T17:14:53.260728] [DEBUG] [VALIDATOR ] AMWL: Primary data validation passed. [2025-11-04T17:14:53.260998] [INFO ] [MAIN ] AMWL: Fetched primary data is valid. [2025-11-04T17:14:53.261075] [DEBUG] [LOCAL ] AMWL: Saving primary data. [2025-11-04T17:14:53.261545] [DEBUG] [LOCAL ] AMWL: Trimmed 14885 records to 10197 (keeping 200 days of data) [2025-11-04T17:14:53.315483] [DEBUG] [LOCAL ] AMWL: Primary data saved (10197 records). [2025-11-04T17:14:53.315595] [INFO ] [MAIN ] AMWL: Full history saved. [2025-11-04T17:14:53.315689] [INFO ] [MAIN ] Update Progress: 113/2367 symbols completed. AMWL took 32.8s. Estimated remaining: 4h 55m 53s. Rate: 7.6 symbols/min [2025-11-04T17:14:53.487027] [INFO ] [MAIN ] ANET: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:14:53.487231] [DEBUG] [LOCAL ] ANET: No existing data file found in primary storage. [2025-11-04T17:14:53.487281] [WARN ] [MAIN ] ANET: No existing data found, performing full refresh instead. [2025-11-04T17:14:53.487388] [INFO ] [MAIN ] ANET: Wiping existing primary data before full refresh. [2025-11-04T17:14:53.487430] [INFO ] [LOCAL ] ANET: Starting deletion of primary data. [2025-11-04T17:14:53.487469] [DEBUG] [LOCAL ] ANET: No primary data file found to delete. [2025-11-04T17:14:53.487499] [INFO ] [MAIN ] ANET: Starting full primary refresh. [2025-11-04T17:14:53.772126] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-05. [2025-11-04T17:14:53.843665] [INFO ] [REMOTE ] ANET: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:14:54.429185] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-11-04T17:14:54.567029] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-09. [2025-11-04T17:14:54.833365] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T17:14:55.367030] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-06. [2025-11-04T17:14:55.886937] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-11-04T17:14:56.167042] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-01. [2025-11-04T17:14:56.225479] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T17:14:56.967063] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-07. [2025-11-04T17:14:57.767026] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-11. [2025-11-04T17:14:58.016810] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-11-04T17:14:58.078472] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T17:14:58.567118] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-10. [2025-11-04T17:14:59.153371] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-11 (1.39 seconds). [2025-11-04T17:14:59.156558] [INFO ] [REMOTE ] AMTM: Full fetch complete (73235 records). [2025-11-04T17:14:59.156898] [DEBUG] [VALIDATOR ] AMTM: Validating primary data. [2025-11-04T17:14:59.384678] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-02. [2025-11-04T17:14:59.647522] [DEBUG] [VALIDATOR ] AMTM: All expected trading days are present. [2025-11-04T17:14:59.647782] [DEBUG] [VALIDATOR ] AMTM: Primary data validation passed. [2025-11-04T17:14:59.648692] [INFO ] [MAIN ] AMTM: Fetched primary data is valid. [2025-11-04T17:14:59.648782] [DEBUG] [LOCAL ] AMTM: Saving primary data. [2025-11-04T17:14:59.650870] [DEBUG] [LOCAL ] AMTM: Trimmed 73235 records to 49931 (keeping 200 days of data) [2025-11-04T17:14:59.767401] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T17:14:59.978997] [DEBUG] [LOCAL ] AMTM: Primary data saved (49931 records). [2025-11-04T17:14:59.979160] [INFO ] [MAIN ] AMTM: Full history saved. [2025-11-04T17:14:59.980379] [INFO ] [MAIN ] Update Progress: 114/2367 symbols completed. AMTM took 49.7s. Estimated remaining: 4h 55m 21s. Rate: 7.6 symbols/min [2025-11-04T17:15:00.152391] [INFO ] [MAIN ] ANF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:15:00.167051] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-11. [2025-11-04T17:15:00.202901] [DEBUG] [LOCAL ] ANF: Loaded 51968 existing records from primary storage. [2025-11-04T17:15:00.203210] [INFO ] [MAIN ] ANF: Loaded 51968 existing records. [2025-11-04T17:15:00.383975] [WARN ] [MAIN ] ANF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:15:00.384176] [INFO ] [MAIN ] ANF: Wiping existing primary data before full refresh. [2025-11-04T17:15:00.384222] [INFO ] [LOCAL ] ANF: Starting deletion of primary data. [2025-11-04T17:15:00.384416] [DEBUG] [LOCAL ] ANF: Deleted primary data file. [2025-11-04T17:15:00.384460] [INFO ] [MAIN ] ANF: Starting full primary refresh. [2025-11-04T17:15:00.746976] [INFO ] [REMOTE ] ANF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:15:00.967048] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-01. [2025-11-04T17:15:01.191866] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T17:15:01.193260] [INFO ] [REMOTE ] AMX: Full fetch complete (70817 records). [2025-11-04T17:15:01.193344] [DEBUG] [VALIDATOR ] AMX: Validating primary data. [2025-11-04T17:15:01.194115] [FATAL] [VALIDATOR ] AMX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:15:01.194494] [FATAL] [MAIN ] AMX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:15:01.194561] [WARN ] [MAIN ] AMX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:15:01.194662] [INFO ] [LOCAL ] AMX: Starting deletion of primary data. [2025-11-04T17:15:01.194741] [DEBUG] [LOCAL ] AMX: No primary data file found to delete. [2025-11-04T17:15:01.194778] [INFO ] [LOCAL ] AMX: Starting deletion of derivative data. [2025-11-04T17:15:01.194919] [DEBUG] [LOCAL ] AMX: Deleted derivative data file. [2025-11-04T17:15:01.195590] [INFO ] [MAIN ] Update Progress: 115/2367 symbols completed. AMX took 34.3s. Estimated remaining: 4h 53m 3s. Rate: 7.7 symbols/min [2025-11-04T17:15:01.365461] [INFO ] [MAIN ] ANGX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:15:01.365632] [DEBUG] [LOCAL ] ANGX: No existing data file found in primary storage. [2025-11-04T17:15:01.365742] [WARN ] [MAIN ] ANGX: No existing data found, performing full refresh instead. [2025-11-04T17:15:01.365884] [INFO ] [MAIN ] ANGX: Wiping existing primary data before full refresh. [2025-11-04T17:15:01.365928] [INFO ] [LOCAL ] ANGX: Starting deletion of primary data. [2025-11-04T17:15:01.365972] [DEBUG] [LOCAL ] ANGX: No primary data file found to delete. [2025-11-04T17:15:01.366005] [INFO ] [MAIN ] ANGX: Starting full primary refresh. [2025-11-04T17:15:01.756591] [INFO ] [REMOTE ] ANGX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:15:01.773609] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-01. [2025-11-04T17:15:01.802198] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-02 (2.42 seconds). [2025-11-04T17:15:02.026272] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:15:02.026571] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-01 [2025-11-04T17:15:02.026696] [FATAL] [REMOTE ] ANGX: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:15:02.026774] [FATAL] [MAIN ] ANGX: Failed to fetch primary data. [2025-11-04T17:15:02.026885] [WARN ] [MAIN ] ANGX: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:15:02.026978] [INFO ] [LOCAL ] ANGX: Starting deletion of primary data. [2025-11-04T17:15:02.027060] [DEBUG] [LOCAL ] ANGX: No primary data file found to delete. [2025-11-04T17:15:02.027098] [INFO ] [LOCAL ] ANGX: Starting deletion of derivative data. [2025-11-04T17:15:02.027136] [DEBUG] [LOCAL ] ANGX: No derivative data file found to delete. [2025-11-04T17:15:02.027180] [INFO ] [MAIN ] Update Progress: 116/2367 symbols completed. ANGX took 0.8s. Estimated remaining: 4h 50m 40s. Rate: 7.7 symbols/min [2025-11-04T17:15:02.203665] [INFO ] [MAIN ] ANRO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:15:02.225712] [DEBUG] [LOCAL ] ANRO: Loaded 20292 existing records from primary storage. [2025-11-04T17:15:02.225988] [INFO ] [MAIN ] ANRO: Loaded 20292 existing records. [2025-11-04T17:15:02.447984] [WARN ] [MAIN ] ANRO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:15:02.448376] [INFO ] [MAIN ] ANRO: Wiping existing primary data before full refresh. [2025-11-04T17:15:02.448445] [INFO ] [LOCAL ] ANRO: Starting deletion of primary data. [2025-11-04T17:15:02.448758] [DEBUG] [LOCAL ] ANRO: Deleted primary data file. [2025-11-04T17:15:02.448824] [INFO ] [MAIN ] ANRO: Starting full primary refresh. [2025-11-04T17:15:02.571570] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-08. [2025-11-04T17:15:02.911995] [INFO ] [REMOTE ] ANRO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:15:02.919854] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-11-04T17:15:03.367039] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-02. [2025-11-04T17:15:03.762436] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T17:15:04.167053] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-03. [2025-11-04T17:15:04.715553] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T17:15:04.967102] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-09. [2025-11-04T17:15:05.767100] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-01. [2025-11-04T17:15:05.964986] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T17:15:06.023397] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-11-04T17:15:06.567066] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-10. [2025-11-04T17:15:07.307780] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-11-04T17:15:07.367067] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-02. [2025-11-04T17:15:07.973682] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T17:15:08.167103] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-11. [2025-11-04T17:15:08.411692] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-11-04T17:15:08.967073] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-03. [2025-11-04T17:15:09.111566] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T17:15:09.114402] [INFO ] [REMOTE ] AN: Full fetch complete (47105 records). [2025-11-04T17:15:09.114550] [DEBUG] [VALIDATOR ] AN: Validating primary data. [2025-11-04T17:15:09.525189] [DEBUG] [VALIDATOR ] AN: All expected trading days are present. [2025-11-04T17:15:09.525298] [DEBUG] [VALIDATOR ] AN: Primary data validation passed. [2025-11-04T17:15:09.525865] [INFO ] [MAIN ] AN: Fetched primary data is valid. [2025-11-04T17:15:09.525931] [DEBUG] [LOCAL ] AN: Saving primary data. [2025-11-04T17:15:09.527029] [DEBUG] [LOCAL ] AN: Trimmed 47105 records to 30656 (keeping 200 days of data) [2025-11-04T17:15:09.706096] [DEBUG] [LOCAL ] AN: Primary data saved (30656 records). [2025-11-04T17:15:09.706231] [INFO ] [MAIN ] AN: Full history saved. [2025-11-04T17:15:09.706309] [INFO ] [MAIN ] Update Progress: 117/2367 symbols completed. AN took 34.5s. Estimated remaining: 4h 50m 31s. Rate: 7.7 symbols/min [2025-11-04T17:15:09.767046] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-04. [2025-11-04T17:15:10.567048] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-03. [2025-11-04T17:15:10.619424] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-11-04T17:15:11.367014] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-04. [2025-11-04T17:15:11.631839] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-11-04T17:15:11.748391] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-11-04T17:15:12.167052] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-04. [2025-11-04T17:15:12.967679] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-05. [2025-11-04T17:15:12.982024] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-11-04T17:15:13.321100] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T17:15:13.767048] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-05. [2025-11-04T17:15:14.481272] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-11-04T17:15:14.567011] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-05. [2025-11-04T17:15:14.859866] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-11-04T17:15:15.367052] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-06. [2025-11-04T17:15:16.167024] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-06. [2025-11-04T17:15:16.251868] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-11-04T17:15:16.967001] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-06. [2025-11-04T17:15:17.069058] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-11-04T17:15:17.395388] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-11-04T17:15:17.767067] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-07. [2025-11-04T17:15:18.415481] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T17:15:18.566289] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-11-04T17:15:18.570570] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-07. [2025-11-04T17:15:19.367060] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-08. [2025-11-04T17:15:19.998489] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-11-04T17:15:20.021783] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-11-04T17:15:20.167036] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-09. [2025-11-04T17:15:20.730744] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T17:15:20.967049] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-10. [2025-11-04T17:15:21.767040] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-07. [2025-11-04T17:15:22.235395] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T17:15:22.567049] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-08. [2025-11-04T17:15:23.367079] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-11. [2025-11-04T17:15:23.860531] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-11-04T17:15:24.144860] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-11-04T17:15:24.167046] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-08. [2025-11-04T17:15:24.294362] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T17:15:24.296259] [INFO ] [REMOTE ] ANRO: Full fetch complete (31605 records). [2025-11-04T17:15:24.296373] [DEBUG] [VALIDATOR ] ANRO: Validating primary data. [2025-11-04T17:15:24.693522] [DEBUG] [VALIDATOR ] ANRO: All expected trading days are present. [2025-11-04T17:15:24.693599] [DEBUG] [VALIDATOR ] ANRO: Primary data validation passed. [2025-11-04T17:15:24.694043] [INFO ] [MAIN ] ANRO: Fetched primary data is valid. [2025-11-04T17:15:24.694104] [DEBUG] [LOCAL ] ANRO: Saving primary data. [2025-11-04T17:15:24.694836] [DEBUG] [LOCAL ] ANRO: Trimmed 31605 records to 20430 (keeping 200 days of data) [2025-11-04T17:15:24.803874] [DEBUG] [LOCAL ] ANRO: Primary data saved (20430 records). [2025-11-04T17:15:24.803990] [INFO ] [MAIN ] ANRO: Full history saved. [2025-11-04T17:15:24.804061] [INFO ] [MAIN ] Update Progress: 118/2367 symbols completed. ANRO took 22.8s. Estimated remaining: 4h 52m 43s. Rate: 7.7 symbols/min [2025-11-04T17:15:24.967030] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-09. [2025-11-04T17:15:25.884520] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-11-04T17:15:25.896256] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-09. [2025-11-04T17:15:26.232995] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-11-04T17:15:26.567091] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-10. [2025-11-04T17:15:27.780023] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-11-04T17:15:27.792314] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-10. [2025-11-04T17:15:28.165810] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-11-04T17:15:28.178098] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-11. [2025-11-04T17:15:29.578088] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-11 (1.40 seconds). [2025-11-04T17:15:29.580070] [INFO ] [REMOTE ] ANF: Full fetch complete (75750 records). [2025-11-04T17:15:29.580183] [DEBUG] [VALIDATOR ] ANF: Validating primary data. [2025-11-04T17:15:29.974882] [DEBUG] [VALIDATOR ] ANF: All expected trading days are present. [2025-11-04T17:15:29.975033] [DEBUG] [VALIDATOR ] ANF: Primary data validation passed. [2025-11-04T17:15:29.975823] [INFO ] [MAIN ] ANF: Fetched primary data is valid. [2025-11-04T17:15:29.975886] [DEBUG] [LOCAL ] ANF: Saving primary data. [2025-11-04T17:15:29.977361] [DEBUG] [LOCAL ] ANF: Trimmed 75750 records to 51975 (keeping 200 days of data) [2025-11-04T17:15:30.281068] [DEBUG] [LOCAL ] ANF: Primary data saved (51975 records). [2025-11-04T17:15:30.281194] [INFO ] [MAIN ] ANF: Full history saved. [2025-11-04T17:15:30.281746] [INFO ] [MAIN ] Update Progress: 119/2367 symbols completed. ANF took 30.3s. Estimated remaining: 4h 51m 51s. Rate: 7.7 symbols/min [2025-11-04T17:15:30.383546] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-10 (2.59 seconds). [2025-11-04T17:15:30.396019] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-11. [2025-11-04T17:15:32.081882] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-11 (1.69 seconds). [2025-11-04T17:15:32.084036] [INFO ] [REMOTE ] ANET: Full fetch complete (77941 records). [2025-11-04T17:15:32.084175] [DEBUG] [VALIDATOR ] ANET: Validating primary data. [2025-11-04T17:15:32.483253] [DEBUG] [VALIDATOR ] ANET: All expected trading days are present. [2025-11-04T17:15:32.483337] [DEBUG] [VALIDATOR ] ANET: Primary data validation passed. [2025-11-04T17:15:32.484109] [INFO ] [MAIN ] ANET: Fetched primary data is valid. [2025-11-04T17:15:32.484174] [DEBUG] [LOCAL ] ANET: Saving primary data. [2025-11-04T17:15:32.485722] [DEBUG] [LOCAL ] ANET: Trimmed 77941 records to 53762 (keeping 200 days of data) [2025-11-04T17:15:32.804633] [DEBUG] [LOCAL ] ANET: Primary data saved (53762 records). [2025-11-04T17:15:32.804803] [INFO ] [MAIN ] ANET: Full history saved. [2025-11-04T17:15:32.804876] [INFO ] [MAIN ] Update Progress: 120/2367 symbols completed. ANET took 39.5s. Estimated remaining: 4h 50m 5s. Rate: 7.7 symbols/min [2025-11-04T17:15:33.458434] [INFO ] [MAIN ] AOD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:15:33.611870] [INFO ] [MAIN ] AOMR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:15:33.605234] [INFO ] [MAIN ] AOMD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:15:33.646124] [DEBUG] [LOCAL ] AOMD: No existing data file found in primary storage. [2025-11-04T17:15:33.646316] [WARN ] [MAIN ] AOMD: No existing data found, performing full refresh instead. [2025-11-04T17:15:33.645517] [INFO ] [MAIN ] ANVS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:15:33.646546] [INFO ] [MAIN ] AOMD: Wiping existing primary data before full refresh. [2025-11-04T17:15:33.646713] [INFO ] [LOCAL ] AOMD: Starting deletion of primary data. [2025-11-04T17:15:33.647256] [DEBUG] [LOCAL ] AOMD: No primary data file found to delete. [2025-11-04T17:15:33.647363] [INFO ] [MAIN ] AOMD: Starting full primary refresh. [2025-11-04T17:15:33.769105] [DEBUG] [LOCAL ] AOMR: Loaded 15954 existing records from primary storage. [2025-11-04T17:15:33.783463] [INFO ] [MAIN ] AOMR: Loaded 15954 existing records. [2025-11-04T17:15:33.907163] [DEBUG] [LOCAL ] ANVS: Loaded 24154 existing records from primary storage. [2025-11-04T17:15:33.920775] [DEBUG] [LOCAL ] AOD: Loaded 30300 existing records from primary storage. [2025-11-04T17:15:33.984918] [INFO ] [MAIN ] AOD: Loaded 30300 existing records. [2025-11-04T17:15:34.018012] [INFO ] [MAIN ] ANVS: Loaded 24154 existing records. [2025-11-04T17:15:34.365259] [WARN ] [MAIN ] AOMR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:15:34.415553] [INFO ] [MAIN ] AOMR: Wiping existing primary data before full refresh. [2025-11-04T17:15:34.480137] [INFO ] [LOCAL ] AOMR: Starting deletion of primary data. [2025-11-04T17:15:34.474808] [INFO ] [REMOTE ] AOMD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:15:34.493721] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-01. [2025-11-04T17:15:34.494613] [DEBUG] [LOCAL ] AOMR: Deleted primary data file. [2025-11-04T17:15:34.506051] [INFO ] [MAIN ] AOMR: Starting full primary refresh. [2025-11-04T17:15:34.763118] [WARN ] [MAIN ] AOD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:15:34.759125] [WARN ] [MAIN ] ANVS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:15:34.769018] [DEBUG] [REMOTE ] AOMD: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:15:34.779671] [INFO ] [MAIN ] AOD: Wiping existing primary data before full refresh. [2025-11-04T17:15:34.779467] [INFO ] [MAIN ] ANVS: Wiping existing primary data before full refresh. [2025-11-04T17:15:34.793438] [INFO ] [LOCAL ] ANVS: Starting deletion of primary data. [2025-11-04T17:15:34.793273] [WARN ] [REMOTE ] AOMD: No data returned for month 2025-01 [2025-11-04T17:15:34.793381] [INFO ] [LOCAL ] AOD: Starting deletion of primary data. [2025-11-04T17:15:34.793801] [FATAL] [REMOTE ] AOMD: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:15:34.793980] [FATAL] [MAIN ] AOMD: Failed to fetch primary data. [2025-11-04T17:15:34.799676] [DEBUG] [LOCAL ] ANVS: Deleted primary data file. [2025-11-04T17:15:34.794430] [DEBUG] [LOCAL ] AOD: Deleted primary data file. [2025-11-04T17:15:34.804854] [WARN ] [MAIN ] AOMD: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:15:34.820569] [INFO ] [LOCAL ] AOMD: Starting deletion of primary data. [2025-11-04T17:15:34.820435] [INFO ] [MAIN ] ANVS: Starting full primary refresh. [2025-11-04T17:15:34.820669] [INFO ] [MAIN ] AOD: Starting full primary refresh. [2025-11-04T17:15:34.953005] [DEBUG] [LOCAL ] AOMD: No primary data file found to delete. [2025-11-04T17:15:35.027961] [INFO ] [LOCAL ] AOMD: Starting deletion of derivative data. [2025-11-04T17:15:35.119283] [DEBUG] [LOCAL ] AOMD: No derivative data file found to delete. [2025-11-04T17:15:35.146036] [INFO ] [MAIN ] Update Progress: 121/2367 symbols completed. AOMD took 2.3s. Estimated remaining: 4h 48m 17s. Rate: 7.8 symbols/min [2025-11-04T17:15:35.781978] [INFO ] [REMOTE ] AOMR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:15:35.830887] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-01. [2025-11-04T17:15:35.986064] [INFO ] [REMOTE ] AOD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:15:36.012273] [INFO ] [MAIN ] AON: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:15:36.012633] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-01. [2025-11-04T17:15:36.046518] [INFO ] [REMOTE ] ANVS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:15:36.047885] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-01. [2025-11-04T17:15:36.093040] [DEBUG] [LOCAL ] AON: Loaded 44957 existing records from primary storage. [2025-11-04T17:15:36.093375] [INFO ] [MAIN ] AON: Loaded 44957 existing records. [2025-11-04T17:15:36.272893] [WARN ] [MAIN ] AON: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:15:36.273098] [INFO ] [MAIN ] AON: Wiping existing primary data before full refresh. [2025-11-04T17:15:36.273147] [INFO ] [LOCAL ] AON: Starting deletion of primary data. [2025-11-04T17:15:36.273352] [DEBUG] [LOCAL ] AON: Deleted primary data file. [2025-11-04T17:15:36.273409] [INFO ] [MAIN ] AON: Starting full primary refresh. [2025-11-04T17:15:36.644804] [INFO ] [REMOTE ] AON: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:15:36.645118] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-01. [2025-11-04T17:15:36.664822] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-11-04T17:15:36.667751] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-02. [2025-11-04T17:15:36.880597] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-11-04T17:15:36.886962] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-02. [2025-11-04T17:15:36.952044] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-11-04T17:15:36.960633] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-02. [2025-11-04T17:15:37.364171] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T17:15:37.368739] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-03. [2025-11-04T17:15:37.747610] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T17:15:37.754436] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-03. [2025-11-04T17:15:38.053671] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T17:15:38.064569] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-02. [2025-11-04T17:15:38.079215] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T17:15:38.113760] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T17:15:38.567003] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-03. [2025-11-04T17:15:38.670630] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-11-04T17:15:39.281728] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T17:15:39.367032] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-03. [2025-11-04T17:15:39.491021] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-11-04T17:15:40.167073] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-04. [2025-11-04T17:15:40.777212] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T17:15:40.967015] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-04. [2025-11-04T17:15:41.155715] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-11-04T17:15:41.767067] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-04. [2025-11-04T17:15:42.415482] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T17:15:42.567015] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-05. [2025-11-04T17:15:42.625724] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T17:15:43.367092] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-05. [2025-11-04T17:15:43.868112] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T17:15:44.167063] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-06. [2025-11-04T17:15:44.245897] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T17:15:44.967040] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-05. [2025-11-04T17:15:45.258336] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T17:15:45.646381] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-11-04T17:15:45.767158] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-06. [2025-11-04T17:15:46.567094] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-06. [2025-11-04T17:15:46.594987] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T17:15:47.367075] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-07. [2025-11-04T17:15:47.617197] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T17:15:48.167045] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-07. [2025-11-04T17:15:48.306030] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-11-04T17:15:48.967036] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-04. [2025-11-04T17:15:49.472011] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:15:49.767121] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-07. [2025-11-04T17:15:49.831697] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T17:15:50.567011] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-05. [2025-11-04T17:15:50.770245] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T17:15:51.262210] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-11-04T17:15:51.367005] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-08. [2025-11-04T17:15:52.167039] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-08. [2025-11-04T17:15:52.576319] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T17:15:52.881072] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T17:15:52.967078] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-08. [2025-11-04T17:15:53.767022] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-09. [2025-11-04T17:15:53.829830] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-11-04T17:15:54.547540] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-11-04T17:15:54.567061] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-10. [2025-11-04T17:15:55.367062] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-09. [2025-11-04T17:15:55.533959] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T17:15:56.167094] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-09. [2025-11-04T17:15:56.416694] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T17:15:56.967026] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-06. [2025-11-04T17:15:57.443056] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T17:15:57.534350] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-11-04T17:15:57.767066] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-11. [2025-11-04T17:15:58.567099] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-10. [2025-11-04T17:15:59.204150] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-11 (1.44 seconds). [2025-11-04T17:15:59.205323] [INFO ] [REMOTE ] ANVS: Full fetch complete (36604 records). [2025-11-04T17:15:59.205391] [DEBUG] [VALIDATOR ] ANVS: Validating primary data. [2025-11-04T17:15:59.368821] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-10. [2025-11-04T17:15:59.628198] [DEBUG] [VALIDATOR ] ANVS: All expected trading days are present. [2025-11-04T17:15:59.628395] [DEBUG] [VALIDATOR ] ANVS: Primary data validation passed. [2025-11-04T17:15:59.629026] [INFO ] [MAIN ] ANVS: Fetched primary data is valid. [2025-11-04T17:15:59.629092] [DEBUG] [LOCAL ] ANVS: Saving primary data. [2025-11-04T17:15:59.630317] [DEBUG] [LOCAL ] ANVS: Trimmed 36604 records to 24258 (keeping 200 days of data) [2025-11-04T17:15:59.783150] [DEBUG] [LOCAL ] ANVS: Primary data saved (24258 records). [2025-11-04T17:15:59.783338] [INFO ] [MAIN ] ANVS: Full history saved. [2025-11-04T17:15:59.783569] [INFO ] [MAIN ] Update Progress: 122/2367 symbols completed. ANVS took 27.0s. Estimated remaining: 4h 53m 21s. Rate: 7.7 symbols/min [2025-11-04T17:15:59.937555] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T17:16:00.003250] [INFO ] [MAIN ] AORT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:16:00.003442] [DEBUG] [LOCAL ] AORT: No existing data file found in primary storage. [2025-11-04T17:16:00.003478] [WARN ] [MAIN ] AORT: No existing data found, performing full refresh instead. [2025-11-04T17:16:00.003610] [INFO ] [MAIN ] AORT: Wiping existing primary data before full refresh. [2025-11-04T17:16:00.003676] [INFO ] [LOCAL ] AORT: Starting deletion of primary data. [2025-11-04T17:16:00.003739] [DEBUG] [LOCAL ] AORT: No primary data file found to delete. [2025-11-04T17:16:00.003773] [INFO ] [MAIN ] AORT: Starting full primary refresh. [2025-11-04T17:16:00.178540] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-07. [2025-11-04T17:16:00.363764] [INFO ] [REMOTE ] AORT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:16:00.903503] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-11-04T17:16:00.967022] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-11. [2025-11-04T17:16:01.140741] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T17:16:01.767099] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-11. [2025-11-04T17:16:02.465073] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-11 (1.50 seconds). [2025-11-04T17:16:02.467803] [INFO ] [REMOTE ] AON: Full fetch complete (65599 records). [2025-11-04T17:16:02.467957] [DEBUG] [VALIDATOR ] AON: Validating primary data. [2025-11-04T17:16:02.573558] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-08. [2025-11-04T17:16:02.799335] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T17:16:02.843846] [INFO ] [REMOTE ] AOD: Full fetch complete (44235 records). [2025-11-04T17:16:02.870095] [DEBUG] [VALIDATOR ] AOD: Validating primary data. [2025-11-04T17:16:02.900257] [DEBUG] [VALIDATOR ] AON: All expected trading days are present. [2025-11-04T17:16:02.900637] [DEBUG] [VALIDATOR ] AON: Primary data validation passed. [2025-11-04T17:16:02.901528] [INFO ] [MAIN ] AON: Fetched primary data is valid. [2025-11-04T17:16:02.963368] [DEBUG] [LOCAL ] AON: Saving primary data. [2025-11-04T17:16:03.031536] [DEBUG] [LOCAL ] AON: Trimmed 65599 records to 44993 (keeping 200 days of data) [2025-11-04T17:16:03.372998] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-01. [2025-11-04T17:16:03.560693] [DEBUG] [VALIDATOR ] AOD: All expected trading days are present. [2025-11-04T17:16:03.561324] [DEBUG] [VALIDATOR ] AOD: Primary data validation passed. [2025-11-04T17:16:03.562451] [INFO ] [MAIN ] AOD: Fetched primary data is valid. [2025-11-04T17:16:03.563779] [DEBUG] [LOCAL ] AOD: Saving primary data. [2025-11-04T17:16:03.571133] [DEBUG] [LOCAL ] AOD: Trimmed 44235 records to 30349 (keeping 200 days of data) [2025-11-04T17:16:03.590209] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-11-04T17:16:03.757171] [DEBUG] [LOCAL ] AON: Primary data saved (44993 records). [2025-11-04T17:16:03.760686] [INFO ] [MAIN ] AON: Full history saved. [2025-11-04T17:16:03.761472] [INFO ] [MAIN ] Update Progress: 123/2367 symbols completed. AON took 28.5s. Estimated remaining: 4h 52m 3s. Rate: 7.7 symbols/min [2025-11-04T17:16:04.018280] [DEBUG] [LOCAL ] AOD: Primary data saved (30349 records). [2025-11-04T17:16:04.020614] [INFO ] [MAIN ] AOS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:16:04.020788] [INFO ] [MAIN ] AOD: Full history saved. [2025-11-04T17:16:04.020873] [DEBUG] [LOCAL ] AOS: No existing data file found in primary storage. [2025-11-04T17:16:04.020943] [WARN ] [MAIN ] AOS: No existing data found, performing full refresh instead. [2025-11-04T17:16:04.021076] [INFO ] [MAIN ] AOS: Wiping existing primary data before full refresh. [2025-11-04T17:16:04.021187] [INFO ] [MAIN ] Update Progress: 124/2367 symbols completed. AOD took 31.2s. Estimated remaining: 4h 49m 38s. Rate: 7.7 symbols/min [2025-11-04T17:16:04.021268] [INFO ] [LOCAL ] AOS: Starting deletion of primary data. [2025-11-04T17:16:04.032948] [DEBUG] [LOCAL ] AOS: No primary data file found to delete. [2025-11-04T17:16:04.065691] [INFO ] [MAIN ] AOS: Starting full primary refresh. [2025-11-04T17:16:04.178702] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-09. [2025-11-04T17:16:04.361720] [INFO ] [MAIN ] AP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:16:04.378416] [DEBUG] [LOCAL ] AP: No existing data file found in primary storage. [2025-11-04T17:16:04.388523] [WARN ] [MAIN ] AP: No existing data found, performing full refresh instead. [2025-11-04T17:16:04.417998] [INFO ] [MAIN ] AP: Wiping existing primary data before full refresh. [2025-11-04T17:16:04.452569] [INFO ] [LOCAL ] AP: Starting deletion of primary data. [2025-11-04T17:16:04.469760] [DEBUG] [LOCAL ] AP: No primary data file found to delete. [2025-11-04T17:16:04.492361] [INFO ] [MAIN ] AP: Starting full primary refresh. [2025-11-04T17:16:04.669219] [INFO ] [REMOTE ] AOS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:16:04.970315] [INFO ] [REMOTE ] AP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:16:04.947526] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-11-04T17:16:04.970492] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-01. [2025-11-04T17:16:04.971720] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T17:16:05.767090] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-02. [2025-11-04T17:16:06.325425] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T17:16:06.567124] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-10. [2025-11-04T17:16:06.841627] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T17:16:07.172331] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-11-04T17:16:07.367018] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-11. [2025-11-04T17:16:07.939606] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T17:16:07.940614] [INFO ] [REMOTE ] AOMR: Full fetch complete (23288 records). [2025-11-04T17:16:07.940707] [DEBUG] [VALIDATOR ] AOMR: Validating primary data. [2025-11-04T17:16:08.176093] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-02. [2025-11-04T17:16:08.312822] [DEBUG] [VALIDATOR ] AOMR: All expected trading days are present. [2025-11-04T17:16:08.312903] [DEBUG] [VALIDATOR ] AOMR: Primary data validation passed. [2025-11-04T17:16:08.313235] [INFO ] [MAIN ] AOMR: Fetched primary data is valid. [2025-11-04T17:16:08.313272] [DEBUG] [LOCAL ] AOMR: Saving primary data. [2025-11-04T17:16:08.313897] [DEBUG] [LOCAL ] AOMR: Trimmed 23288 records to 15990 (keeping 200 days of data) [2025-11-04T17:16:08.403455] [DEBUG] [LOCAL ] AOMR: Primary data saved (15990 records). [2025-11-04T17:16:08.403599] [INFO ] [MAIN ] AOMR: Full history saved. [2025-11-04T17:16:08.403709] [INFO ] [MAIN ] Update Progress: 125/2367 symbols completed. AOMR took 35.6s. Estimated remaining: 4h 48m 30s. Rate: 7.8 symbols/min [2025-11-04T17:16:08.567906] [INFO ] [MAIN ] APAM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:16:08.606198] [DEBUG] [LOCAL ] APAM: Loaded 35143 existing records from primary storage. [2025-11-04T17:16:08.606482] [INFO ] [MAIN ] APAM: Loaded 35143 existing records. [2025-11-04T17:16:08.787858] [WARN ] [MAIN ] APAM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:16:08.788101] [INFO ] [MAIN ] APAM: Wiping existing primary data before full refresh. [2025-11-04T17:16:08.788150] [INFO ] [LOCAL ] APAM: Starting deletion of primary data. [2025-11-04T17:16:08.788328] [DEBUG] [LOCAL ] APAM: Deleted primary data file. [2025-11-04T17:16:08.788366] [INFO ] [MAIN ] APAM: Starting full primary refresh. [2025-11-04T17:16:08.972217] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-03. [2025-11-04T17:16:09.149742] [INFO ] [REMOTE ] APAM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:16:09.590864] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-11-04T17:16:09.767058] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-01. [2025-11-04T17:16:09.957892] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T17:16:10.317920] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-11-04T17:16:10.567047] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-01. [2025-11-04T17:16:11.367015] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-02. [2025-11-04T17:16:11.551918] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T17:16:11.808566] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-11-04T17:16:12.167051] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-03. [2025-11-04T17:16:12.967045] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-02. [2025-11-04T17:16:13.689989] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T17:16:13.767012] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-04. [2025-11-04T17:16:14.019399] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T17:16:14.567033] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-03. [2025-11-04T17:16:15.011029] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-11-04T17:16:15.161258] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T17:16:15.367043] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-04. [2025-11-04T17:16:16.167032] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-05. [2025-11-04T17:16:16.426180] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-11-04T17:16:16.967067] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-04. [2025-11-04T17:16:17.353355] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-04 (0.39 seconds). [2025-11-04T17:16:17.463511] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T17:16:17.767094] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-05. [2025-11-04T17:16:18.567025] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-06. [2025-11-04T17:16:18.678023] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-11-04T17:16:19.367104] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-03. [2025-11-04T17:16:19.947145] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-11-04T17:16:20.167098] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-05. [2025-11-04T17:16:20.465501] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T17:16:20.712909] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T17:16:20.967044] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-06. [2025-11-04T17:16:21.572307] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T17:16:21.767050] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-07. [2025-11-04T17:16:22.567094] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-04. [2025-11-04T17:16:23.083371] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T17:16:23.367022] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-08. [2025-11-04T17:16:23.802989] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T17:16:24.167035] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-07. [2025-11-04T17:16:24.760802] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T17:16:24.770897] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T17:16:24.967071] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-08. [2025-11-04T17:16:25.432668] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-11-04T17:16:25.767066] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-06. [2025-11-04T17:16:26.567029] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-09. [2025-11-04T17:16:26.655495] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T17:16:27.367081] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-05. [2025-11-04T17:16:27.722408] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T17:16:28.167155] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-09. [2025-11-04T17:16:28.528150] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T17:16:28.696436] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-11-04T17:16:28.967098] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-10. [2025-11-04T17:16:29.767067] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-07. [2025-11-04T17:16:30.290292] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T17:16:30.567077] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-10. [2025-11-04T17:16:30.686938] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T17:16:31.209723] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-11-04T17:16:31.367012] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-11. [2025-11-04T17:16:31.841054] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T17:16:31.841799] [INFO ] [REMOTE ] AP: Full fetch complete (9252 records). [2025-11-04T17:16:31.841865] [DEBUG] [VALIDATOR ] AP: Validating primary data. [2025-11-04T17:16:32.174919] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-08. [2025-11-04T17:16:32.197261] [DEBUG] [VALIDATOR ] AP: All expected trading days are present. [2025-11-04T17:16:32.197899] [DEBUG] [VALIDATOR ] AP: Primary data validation passed. [2025-11-04T17:16:32.198133] [INFO ] [MAIN ] AP: Fetched primary data is valid. [2025-11-04T17:16:32.198173] [DEBUG] [LOCAL ] AP: Saving primary data. [2025-11-04T17:16:32.198585] [DEBUG] [LOCAL ] AP: Trimmed 9252 records to 7540 (keeping 200 days of data) [2025-11-04T17:16:32.239900] [DEBUG] [LOCAL ] AP: Primary data saved (7540 records). [2025-11-04T17:16:32.239994] [INFO ] [MAIN ] AP: Full history saved. [2025-11-04T17:16:32.240056] [INFO ] [MAIN ] Update Progress: 126/2367 symbols completed. AP took 28.2s. Estimated remaining: 4h 53m 9s. Rate: 7.6 symbols/min [2025-11-04T17:16:32.403523] [INFO ] [MAIN ] APD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:16:32.447998] [DEBUG] [LOCAL ] APD: Loaded 45308 existing records from primary storage. [2025-11-04T17:16:32.448270] [INFO ] [MAIN ] APD: Loaded 45308 existing records. [2025-11-04T17:16:32.621885] [WARN ] [MAIN ] APD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:16:32.622059] [INFO ] [MAIN ] APD: Wiping existing primary data before full refresh. [2025-11-04T17:16:32.622103] [INFO ] [LOCAL ] APD: Starting deletion of primary data. [2025-11-04T17:16:32.622339] [DEBUG] [LOCAL ] APD: Deleted primary data file. [2025-11-04T17:16:32.622396] [INFO ] [MAIN ] APD: Starting full primary refresh. [2025-11-04T17:16:32.967011] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-06. [2025-11-04T17:16:32.966965] [INFO ] [REMOTE ] APD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:16:33.289584] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T17:16:33.767065] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-11. [2025-11-04T17:16:34.110465] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-11-04T17:16:34.567042] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-09. [2025-11-04T17:16:34.980910] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-11 (1.21 seconds). [2025-11-04T17:16:34.982799] [INFO ] [REMOTE ] AOS: Full fetch complete (71095 records). [2025-11-04T17:16:34.982895] [DEBUG] [VALIDATOR ] AOS: Validating primary data. [2025-11-04T17:16:35.368898] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-01. [2025-11-04T17:16:35.369162] [DEBUG] [VALIDATOR ] AOS: All expected trading days are present. [2025-11-04T17:16:35.370197] [DEBUG] [VALIDATOR ] AOS: Primary data validation passed. [2025-11-04T17:16:35.370940] [INFO ] [MAIN ] AOS: Fetched primary data is valid. [2025-11-04T17:16:35.371075] [DEBUG] [LOCAL ] AOS: Saving primary data. [2025-11-04T17:16:35.372518] [DEBUG] [LOCAL ] AOS: Trimmed 71095 records to 48523 (keeping 200 days of data) [2025-11-04T17:16:35.617421] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T17:16:35.698750] [DEBUG] [LOCAL ] AOS: Primary data saved (48523 records). [2025-11-04T17:16:35.698884] [INFO ] [MAIN ] AOS: Full history saved. [2025-11-04T17:16:35.698954] [INFO ] [MAIN ] Update Progress: 127/2367 symbols completed. AOS took 31.9s. Estimated remaining: 4h 51m 44s. Rate: 7.7 symbols/min [2025-11-04T17:16:35.865706] [INFO ] [MAIN ] APG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:16:35.865878] [DEBUG] [LOCAL ] APG: No existing data file found in primary storage. [2025-11-04T17:16:35.865915] [WARN ] [MAIN ] APG: No existing data found, performing full refresh instead. [2025-11-04T17:16:35.866027] [INFO ] [MAIN ] APG: Wiping existing primary data before full refresh. [2025-11-04T17:16:35.866068] [INFO ] [LOCAL ] APG: Starting deletion of primary data. [2025-11-04T17:16:35.866110] [DEBUG] [LOCAL ] APG: No primary data file found to delete. [2025-11-04T17:16:35.866143] [INFO ] [MAIN ] APG: Starting full primary refresh. [2025-11-04T17:16:36.181023] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-07. [2025-11-04T17:16:36.245733] [INFO ] [REMOTE ] APG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:16:36.967093] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-01. [2025-11-04T17:16:37.310819] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T17:16:37.767093] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-08. [2025-11-04T17:16:38.404843] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T17:16:38.567059] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-02. [2025-11-04T17:16:38.746263] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-11-04T17:16:39.367088] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-09. [2025-11-04T17:16:39.724018] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T17:16:40.167024] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-03. [2025-11-04T17:16:40.233796] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-11-04T17:16:40.967028] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-10. [2025-11-04T17:16:41.244973] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T17:16:41.767111] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-04. [2025-11-04T17:16:42.567045] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-10. [2025-11-04T17:16:43.047253] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T17:16:43.367079] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-05. [2025-11-04T17:16:43.689265] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-11-04T17:16:44.167081] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-11. [2025-11-04T17:16:58.273207] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-01 (22.90 seconds). [2025-11-04T17:16:58.283150] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-02. [2025-11-04T17:16:59.671281] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-11-04T17:16:59.680811] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-03. [2025-11-04T17:17:00.247614] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-10 (19.28 seconds). [2025-11-04T17:17:00.255763] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-11. [2025-11-04T17:17:00.629465] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-11 (16.46 seconds). [2025-11-04T17:17:00.632791] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-05 (17.27 seconds). [2025-11-04T17:17:00.643387] [INFO ] [REMOTE ] AORT: Full fetch complete (42367 records). [2025-11-04T17:17:00.643807] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-06. [2025-11-04T17:17:00.644614] [DEBUG] [VALIDATOR ] AORT: Validating primary data. [2025-11-04T17:17:01.032365] [DEBUG] [VALIDATOR ] AORT: All expected trading days are present. [2025-11-04T17:17:01.032500] [DEBUG] [VALIDATOR ] AORT: Primary data validation passed. [2025-11-04T17:17:01.033123] [INFO ] [MAIN ] AORT: Fetched primary data is valid. [2025-11-04T17:17:01.033220] [DEBUG] [LOCAL ] AORT: Saving primary data. [2025-11-04T17:17:01.034636] [DEBUG] [LOCAL ] AORT: Trimmed 42367 records to 28588 (keeping 200 days of data) [2025-11-04T17:17:01.222011] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-11-04T17:17:01.245807] [DEBUG] [LOCAL ] AORT: Primary data saved (28588 records). [2025-11-04T17:17:01.245923] [INFO ] [MAIN ] AORT: Full history saved. [2025-11-04T17:17:01.246026] [INFO ] [MAIN ] Update Progress: 128/2367 symbols completed. AORT took 1m 1s. Estimated remaining: 4h 56m 46s. Rate: 7.5 symbols/min [2025-11-04T17:17:01.243166] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-04. [2025-11-04T17:17:01.421537] [INFO ] [MAIN ] APH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:17:01.426333] [DEBUG] [LOCAL ] APH: No existing data file found in primary storage. [2025-11-04T17:17:01.426219] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-11 (1.17 seconds). [2025-11-04T17:17:01.426427] [WARN ] [MAIN ] APH: No existing data found, performing full refresh instead. [2025-11-04T17:17:01.427962] [INFO ] [MAIN ] APH: Wiping existing primary data before full refresh. [2025-11-04T17:17:01.428021] [INFO ] [LOCAL ] APH: Starting deletion of primary data. [2025-11-04T17:17:01.428216] [DEBUG] [LOCAL ] APH: No primary data file found to delete. [2025-11-04T17:17:01.428264] [INFO ] [MAIN ] APH: Starting full primary refresh. [2025-11-04T17:17:01.470478] [INFO ] [REMOTE ] APAM: Full fetch complete (52029 records). [2025-11-04T17:17:01.518859] [DEBUG] [VALIDATOR ] APAM: Validating primary data. [2025-11-04T17:17:02.122944] [INFO ] [REMOTE ] APH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:17:02.150726] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-01. [2025-11-04T17:17:02.270576] [DEBUG] [VALIDATOR ] APAM: All expected trading days are present. [2025-11-04T17:17:02.270731] [DEBUG] [VALIDATOR ] APAM: Primary data validation passed. [2025-11-04T17:17:02.271382] [INFO ] [MAIN ] APAM: Fetched primary data is valid. [2025-11-04T17:17:02.271485] [DEBUG] [LOCAL ] APAM: Saving primary data. [2025-11-04T17:17:02.273144] [DEBUG] [LOCAL ] APAM: Trimmed 52029 records to 35134 (keeping 200 days of data) [2025-11-04T17:17:02.399358] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-11-04T17:17:02.417555] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-07. [2025-11-04T17:17:02.547004] [DEBUG] [LOCAL ] APAM: Primary data saved (35134 records). [2025-11-04T17:17:02.547309] [INFO ] [MAIN ] APAM: Full history saved. [2025-11-04T17:17:02.547487] [INFO ] [MAIN ] Update Progress: 129/2367 symbols completed. APAM took 54.1s. Estimated remaining: 4h 54m 43s. Rate: 7.6 symbols/min [2025-11-04T17:17:02.728311] [INFO ] [MAIN ] APLE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:17:02.763620] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-11-04T17:17:02.775556] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-05. [2025-11-04T17:17:02.816418] [DEBUG] [LOCAL ] APLE: Loaded 51889 existing records from primary storage. [2025-11-04T17:17:02.816768] [INFO ] [MAIN ] APLE: Loaded 51889 existing records. [2025-11-04T17:17:02.992104] [WARN ] [MAIN ] APLE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:17:02.992326] [INFO ] [MAIN ] APLE: Wiping existing primary data before full refresh. [2025-11-04T17:17:02.992374] [INFO ] [LOCAL ] APLE: Starting deletion of primary data. [2025-11-04T17:17:02.992588] [DEBUG] [LOCAL ] APLE: Deleted primary data file. [2025-11-04T17:17:02.992677] [INFO ] [MAIN ] APLE: Starting full primary refresh. [2025-11-04T17:17:03.345057] [INFO ] [REMOTE ] APLE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:17:03.345407] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-01. [2025-11-04T17:17:03.761028] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-11-04T17:17:03.774093] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-02. [2025-11-04T17:17:03.814134] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T17:17:03.825379] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-08. [2025-11-04T17:17:04.222380] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T17:17:04.233214] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-06. [2025-11-04T17:17:04.526931] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T17:17:04.536737] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-02. [2025-11-04T17:17:05.071556] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T17:17:05.082802] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-09. [2025-11-04T17:17:05.113009] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-11-04T17:17:05.123112] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-03. [2025-11-04T17:17:05.367910] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T17:17:05.378382] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-07. [2025-11-04T17:17:05.921709] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-11-04T17:17:05.931071] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-03. [2025-11-04T17:17:06.552112] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-11-04T17:17:06.564327] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-10. [2025-11-04T17:17:06.597347] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-11-04T17:17:06.609288] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-04. [2025-11-04T17:17:06.802139] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-11-04T17:17:06.812252] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-08. [2025-11-04T17:17:07.587103] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-11-04T17:17:07.600049] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-04. [2025-11-04T17:17:07.886368] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T17:17:07.896119] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-09. [2025-11-04T17:17:07.960445] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T17:17:07.974178] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-11. [2025-11-04T17:17:08.130428] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-11-04T17:17:08.143415] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-05. [2025-11-04T17:17:09.097544] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T17:17:09.100412] [INFO ] [REMOTE ] APG: Full fetch complete (72470 records). [2025-11-04T17:17:09.100549] [DEBUG] [VALIDATOR ] APG: Validating primary data. [2025-11-04T17:17:09.550076] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-11-04T17:17:09.656994] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-05. [2025-11-04T17:17:09.657198] [DEBUG] [VALIDATOR ] APG: All expected trading days are present. [2025-11-04T17:17:09.658781] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-11-04T17:17:09.657516] [DEBUG] [VALIDATOR ] APG: Primary data validation passed. [2025-11-04T17:17:09.668988] [INFO ] [MAIN ] APG: Fetched primary data is valid. [2025-11-04T17:17:09.669957] [DEBUG] [LOCAL ] APG: Saving primary data. [2025-11-04T17:17:09.670577] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-10. [2025-11-04T17:17:09.673077] [DEBUG] [LOCAL ] APG: Trimmed 72470 records to 50765 (keeping 200 days of data) [2025-11-04T17:17:09.752822] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-11-04T17:17:09.798210] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-06. [2025-11-04T17:17:10.008134] [DEBUG] [LOCAL ] APG: Primary data saved (50765 records). [2025-11-04T17:17:10.008265] [INFO ] [MAIN ] APG: Full history saved. [2025-11-04T17:17:10.008331] [INFO ] [MAIN ] Update Progress: 130/2367 symbols completed. APG took 34.3s. Estimated remaining: 4h 54m 27s. Rate: 7.6 symbols/min [2025-11-04T17:17:10.174129] [INFO ] [MAIN ] APO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:17:10.174329] [DEBUG] [LOCAL ] APO: No existing data file found in primary storage. [2025-11-04T17:17:10.174366] [WARN ] [MAIN ] APO: No existing data found, performing full refresh instead. [2025-11-04T17:17:10.174479] [INFO ] [MAIN ] APO: Wiping existing primary data before full refresh. [2025-11-04T17:17:10.174519] [INFO ] [LOCAL ] APO: Starting deletion of primary data. [2025-11-04T17:17:10.174559] [DEBUG] [LOCAL ] APO: No primary data file found to delete. [2025-11-04T17:17:10.174591] [INFO ] [MAIN ] APO: Starting full primary refresh. [2025-11-04T17:17:10.535812] [INFO ] [REMOTE ] APO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:17:10.536130] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-01. [2025-11-04T17:17:11.059303] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T17:17:11.071385] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-06. [2025-11-04T17:17:11.101204] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T17:17:11.113863] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-07. [2025-11-04T17:17:11.143767] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T17:17:11.154701] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-11. [2025-11-04T17:17:11.827752] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T17:17:11.838219] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-02. [2025-11-04T17:17:12.264746] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T17:17:12.266725] [INFO ] [REMOTE ] APD: Full fetch complete (67246 records). [2025-11-04T17:17:12.266817] [DEBUG] [VALIDATOR ] APD: Validating primary data. [2025-11-04T17:17:12.705283] [DEBUG] [VALIDATOR ] APD: All expected trading days are present. [2025-11-04T17:17:12.705506] [DEBUG] [VALIDATOR ] APD: Primary data validation passed. [2025-11-04T17:17:12.706234] [INFO ] [MAIN ] APD: Fetched primary data is valid. [2025-11-04T17:17:12.735218] [DEBUG] [LOCAL ] APD: Saving primary data. [2025-11-04T17:17:12.737590] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-11-04T17:17:12.749219] [DEBUG] [LOCAL ] APD: Trimmed 67246 records to 45331 (keeping 200 days of data) [2025-11-04T17:17:12.749090] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-07. [2025-11-04T17:17:13.038753] [DEBUG] [LOCAL ] APD: Primary data saved (45331 records). [2025-11-04T17:17:13.038887] [INFO ] [MAIN ] APD: Full history saved. [2025-11-04T17:17:13.040076] [INFO ] [MAIN ] Update Progress: 131/2367 symbols completed. APD took 40.8s. Estimated remaining: 4h 52m 56s. Rate: 7.6 symbols/min [2025-11-04T17:17:13.274022] [INFO ] [MAIN ] APOS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:17:13.277160] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-11-04T17:17:13.291434] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-08. [2025-11-04T17:17:13.298685] [DEBUG] [LOCAL ] APOS: Loaded 6771 existing records from primary storage. [2025-11-04T17:17:13.298914] [INFO ] [MAIN ] APOS: Loaded 6771 existing records. [2025-11-04T17:17:13.489473] [WARN ] [MAIN ] APOS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:17:13.489749] [INFO ] [MAIN ] APOS: Wiping existing primary data before full refresh. [2025-11-04T17:17:13.489822] [INFO ] [LOCAL ] APOS: Starting deletion of primary data. [2025-11-04T17:17:13.490002] [DEBUG] [LOCAL ] APOS: Deleted primary data file. [2025-11-04T17:17:13.490060] [INFO ] [MAIN ] APOS: Starting full primary refresh. [2025-11-04T17:17:13.733046] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-11-04T17:17:13.804207] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-03. [2025-11-04T17:17:13.892164] [INFO ] [REMOTE ] APOS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:17:13.892492] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-01. [2025-11-04T17:17:14.371376] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-01 (0.48 seconds). [2025-11-04T17:17:14.373156] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-02. [2025-11-04T17:17:14.464503] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-11-04T17:17:14.567026] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-08. [2025-11-04T17:17:14.884028] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T17:17:15.022690] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-11-04T17:17:15.365998] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T17:17:15.376234] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-03. [2025-11-04T17:17:16.003315] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-11-04T17:17:16.005783] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-11-04T17:17:16.167043] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-04. [2025-11-04T17:17:16.703330] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-11-04T17:17:16.967047] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-04. [2025-11-04T17:17:17.767021] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-09. [2025-11-04T17:17:18.549373] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-11-04T17:17:18.567026] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-05. [2025-11-04T17:17:19.103628] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T17:17:19.367058] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-05. [2025-11-04T17:17:19.947274] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-11-04T17:17:20.164390] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-11-04T17:17:20.174935] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-06. [2025-11-04T17:17:20.625364] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-11-04T17:17:20.967030] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-09. [2025-11-04T17:17:21.767059] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-06. [2025-11-04T17:17:22.527824] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-11-04T17:17:22.567049] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-10. [2025-11-04T17:17:23.088138] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T17:17:23.367078] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-07. [2025-11-04T17:17:24.167059] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-07. [2025-11-04T17:17:24.637499] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-10 (2.07 seconds). [2025-11-04T17:17:24.694767] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-11-04T17:17:24.967009] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-08. [2025-11-04T17:17:25.096073] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-11-04T17:17:25.440424] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-11-04T17:17:25.767055] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-09. [2025-11-04T17:17:26.386702] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-11-04T17:17:26.567059] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-10. [2025-11-04T17:17:26.990047] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-11-04T17:17:27.367050] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-10. [2025-11-04T17:17:28.167095] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-11. [2025-11-04T17:17:28.754597] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-11 (0.59 seconds). [2025-11-04T17:17:28.755464] [INFO ] [REMOTE ] APOS: Full fetch complete (10309 records). [2025-11-04T17:17:28.755531] [DEBUG] [VALIDATOR ] APOS: Validating primary data. [2025-11-04T17:17:28.972747] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-11. [2025-11-04T17:17:29.146832] [DEBUG] [VALIDATOR ] APOS: All expected trading days are present. [2025-11-04T17:17:29.146949] [DEBUG] [VALIDATOR ] APOS: Primary data validation passed. [2025-11-04T17:17:29.147299] [INFO ] [MAIN ] APOS: Fetched primary data is valid. [2025-11-04T17:17:29.147398] [DEBUG] [LOCAL ] APOS: Saving primary data. [2025-11-04T17:17:29.148088] [DEBUG] [LOCAL ] APOS: Trimmed 10309 records to 6817 (keeping 200 days of data) [2025-11-04T17:17:29.189920] [DEBUG] [LOCAL ] APOS: Primary data saved (6817 records). [2025-11-04T17:17:29.190138] [INFO ] [MAIN ] APOS: Full history saved. [2025-11-04T17:17:29.190322] [INFO ] [MAIN ] Update Progress: 132/2367 symbols completed. APOS took 16.2s. Estimated remaining: 4h 55m 9s. Rate: 7.6 symbols/min [2025-11-04T17:17:29.346147] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-11 (0.37 seconds). [2025-11-04T17:17:29.401159] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-10 (2.03 seconds). [2025-11-04T17:17:29.397521] [INFO ] [MAIN ] APTV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:17:29.412402] [INFO ] [REMOTE ] APH: Full fetch complete (77481 records). [2025-11-04T17:17:29.413621] [DEBUG] [VALIDATOR ] APH: Validating primary data. [2025-11-04T17:17:29.413049] [DEBUG] [LOCAL ] APTV: No existing data file found in primary storage. [2025-11-04T17:17:29.413873] [WARN ] [MAIN ] APTV: No existing data found, performing full refresh instead. [2025-11-04T17:17:29.414734] [FATAL] [VALIDATOR ] APH: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:17:29.414981] [FATAL] [MAIN ] APH: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:17:29.415461] [WARN ] [MAIN ] APH: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:17:29.414896] [INFO ] [MAIN ] APTV: Wiping existing primary data before full refresh. [2025-11-04T17:17:29.415557] [INFO ] [LOCAL ] APH: Starting deletion of primary data. [2025-11-04T17:17:29.415780] [INFO ] [LOCAL ] APTV: Starting deletion of primary data. [2025-11-04T17:17:29.415864] [DEBUG] [LOCAL ] APH: No primary data file found to delete. [2025-11-04T17:17:29.415952] [DEBUG] [LOCAL ] APTV: No primary data file found to delete. [2025-11-04T17:17:29.416082] [INFO ] [LOCAL ] APH: Starting deletion of derivative data. [2025-11-04T17:17:29.416146] [INFO ] [MAIN ] APTV: Starting full primary refresh. [2025-11-04T17:17:29.416295] [DEBUG] [LOCAL ] APH: No derivative data file found to delete. [2025-11-04T17:17:29.447955] [INFO ] [MAIN ] Update Progress: 133/2367 symbols completed. APH took 28.2s. Estimated remaining: 4h 52m 52s. Rate: 7.6 symbols/min [2025-11-04T17:17:29.772162] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-11. [2025-11-04T17:17:29.831053] [INFO ] [MAIN ] AQN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:17:29.960895] [INFO ] [REMOTE ] APTV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:17:29.996052] [DEBUG] [LOCAL ] AQN: Loaded 51512 existing records from primary storage. [2025-11-04T17:17:29.996335] [INFO ] [MAIN ] AQN: Loaded 51512 existing records. [2025-11-04T17:17:30.170208] [WARN ] [MAIN ] AQN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:17:30.170405] [INFO ] [MAIN ] AQN: Wiping existing primary data before full refresh. [2025-11-04T17:17:30.170451] [INFO ] [LOCAL ] AQN: Starting deletion of primary data. [2025-11-04T17:17:30.170626] [DEBUG] [LOCAL ] AQN: Deleted primary data file. [2025-11-04T17:17:30.170701] [INFO ] [MAIN ] AQN: Starting full primary refresh. [2025-11-04T17:17:30.519452] [INFO ] [REMOTE ] AQN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:17:30.566999] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-01. [2025-11-04T17:17:31.185451] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-11 (1.41 seconds). [2025-11-04T17:17:31.187823] [INFO ] [REMOTE ] APLE: Full fetch complete (74849 records). [2025-11-04T17:17:31.187982] [DEBUG] [VALIDATOR ] APLE: Validating primary data. [2025-11-04T17:17:31.379853] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-01. [2025-11-04T17:17:31.577002] [DEBUG] [VALIDATOR ] APLE: All expected trading days are present. [2025-11-04T17:17:31.577087] [DEBUG] [VALIDATOR ] APLE: Primary data validation passed. [2025-11-04T17:17:31.577880] [INFO ] [MAIN ] APLE: Fetched primary data is valid. [2025-11-04T17:17:31.577944] [DEBUG] [LOCAL ] APLE: Saving primary data. [2025-11-04T17:17:31.579388] [DEBUG] [LOCAL ] APLE: Trimmed 74849 records to 51889 (keeping 200 days of data) [2025-11-04T17:17:31.900897] [DEBUG] [LOCAL ] APLE: Primary data saved (51889 records). [2025-11-04T17:17:31.901218] [INFO ] [MAIN ] APLE: Full history saved. [2025-11-04T17:17:31.903989] [INFO ] [MAIN ] Update Progress: 134/2367 symbols completed. APLE took 29.4s. Estimated remaining: 4h 51m 14s. Rate: 7.7 symbols/min [2025-11-04T17:17:32.019162] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T17:17:32.118236] [INFO ] [MAIN ] AQNB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:17:32.118394] [DEBUG] [LOCAL ] AQNB: No existing data file found in primary storage. [2025-11-04T17:17:32.118430] [WARN ] [MAIN ] AQNB: No existing data found, performing full refresh instead. [2025-11-04T17:17:32.118537] [INFO ] [MAIN ] AQNB: Wiping existing primary data before full refresh. [2025-11-04T17:17:32.118576] [INFO ] [LOCAL ] AQNB: Starting deletion of primary data. [2025-11-04T17:17:32.118615] [DEBUG] [LOCAL ] AQNB: No primary data file found to delete. [2025-11-04T17:17:32.118674] [INFO ] [MAIN ] AQNB: Starting full primary refresh. [2025-11-04T17:17:32.179403] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-02. [2025-11-04T17:17:32.463530] [INFO ] [REMOTE ] AQNB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:17:32.972210] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-01. [2025-11-04T17:17:32.989977] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-11-04T17:17:33.408212] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-11-04T17:17:33.530671] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T17:17:33.767036] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-02. [2025-11-04T17:17:34.151138] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T17:17:34.567083] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-03. [2025-11-04T17:17:35.032165] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-11-04T17:17:35.367042] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-02. [2025-11-04T17:17:36.167038] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-03. [2025-11-04T17:17:36.719078] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T17:17:36.967048] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-04. [2025-11-04T17:17:37.369906] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-11-04T17:17:37.458580] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-11-04T17:17:37.767031] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-03. [2025-11-04T17:17:38.567042] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-05. [2025-11-04T17:17:38.936549] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-11-04T17:17:39.196393] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-11-04T17:17:39.367044] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-06. [2025-11-04T17:17:39.790809] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-11-04T17:17:40.167014] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-07. [2025-11-04T17:17:40.521022] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-11-04T17:17:40.967166] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-04. [2025-11-04T17:17:41.767057] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-04. [2025-11-04T17:17:42.357634] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T17:17:42.567020] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-08. [2025-11-04T17:17:43.063806] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T17:17:43.367081] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-05. [2025-11-04T17:17:43.974563] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-11-04T17:17:44.167067] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-08. [2025-11-04T17:17:44.624313] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T17:17:44.690879] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-11-04T17:17:44.967053] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-09. [2025-11-04T17:17:45.767082] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-05. [2025-11-04T17:17:46.553987] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T17:17:46.567000] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-10. [2025-11-04T17:17:47.355091] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-11-04T17:17:47.366996] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-06. [2025-11-04T17:17:48.167042] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-06. [2025-11-04T17:17:48.243963] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-11-04T17:17:48.850869] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-11-04T17:17:48.967068] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-09. [2025-11-04T17:17:49.342058] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-11-04T17:17:49.550926] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-11-04T17:17:49.767015] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-10. [2025-11-04T17:17:50.023515] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-10 (0.26 seconds). [2025-11-04T17:17:50.567133] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-11. [2025-11-04T17:17:51.032151] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T17:17:51.032842] [INFO ] [REMOTE ] AQNB: Full fetch complete (5339 records). [2025-11-04T17:17:51.032913] [DEBUG] [VALIDATOR ] AQNB: Validating primary data. [2025-11-04T17:17:51.377185] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-07. [2025-11-04T17:17:51.384878] [DEBUG] [VALIDATOR ] AQNB: All expected trading days are present. [2025-11-04T17:17:51.384935] [DEBUG] [VALIDATOR ] AQNB: Primary data validation passed. [2025-11-04T17:17:51.385088] [INFO ] [MAIN ] AQNB: Fetched primary data is valid. [2025-11-04T17:17:51.385128] [DEBUG] [LOCAL ] AQNB: Saving primary data. [2025-11-04T17:17:51.385488] [DEBUG] [LOCAL ] AQNB: Trimmed 5339 records to 3607 (keeping 200 days of data) [2025-11-04T17:17:51.408296] [DEBUG] [LOCAL ] AQNB: Primary data saved (3607 records). [2025-11-04T17:17:51.408373] [INFO ] [MAIN ] AQNB: Full history saved. [2025-11-04T17:17:51.408433] [INFO ] [MAIN ] Update Progress: 135/2367 symbols completed. AQNB took 19.5s. Estimated remaining: 4h 54m 19s. Rate: 7.6 symbols/min [2025-11-04T17:17:51.576452] [INFO ] [MAIN ] AR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:17:51.576679] [DEBUG] [LOCAL ] AR: No existing data file found in primary storage. [2025-11-04T17:17:51.576731] [WARN ] [MAIN ] AR: No existing data found, performing full refresh instead. [2025-11-04T17:17:51.576859] [INFO ] [MAIN ] AR: Wiping existing primary data before full refresh. [2025-11-04T17:17:51.576915] [INFO ] [LOCAL ] AR: Starting deletion of primary data. [2025-11-04T17:17:51.576962] [DEBUG] [LOCAL ] AR: No primary data file found to delete. [2025-11-04T17:17:51.576994] [INFO ] [MAIN ] AR: Starting full primary refresh. [2025-11-04T17:17:51.933912] [INFO ] [REMOTE ] AR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:17:52.167078] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-01. [2025-11-04T17:17:52.750019] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T17:17:52.967049] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-08. [2025-11-04T17:17:53.559837] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-11-04T17:17:53.767127] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-07. [2025-11-04T17:17:54.035081] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T17:17:54.567045] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-09. [2025-11-04T17:17:55.245443] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T17:17:55.367061] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-08. [2025-11-04T17:17:55.961259] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T17:17:56.167158] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-11. [2025-11-04T17:17:56.726899] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-11-04T17:17:56.967017] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-09. [2025-11-04T17:17:57.453756] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-11 (1.29 seconds). [2025-11-04T17:17:57.456063] [INFO ] [REMOTE ] APO: Full fetch complete (77455 records). [2025-11-04T17:17:57.456211] [DEBUG] [VALIDATOR ] APO: Validating primary data. [2025-11-04T17:17:57.773069] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-10. [2025-11-04T17:17:57.871942] [DEBUG] [VALIDATOR ] APO: All expected trading days are present. [2025-11-04T17:17:57.872034] [DEBUG] [VALIDATOR ] APO: Primary data validation passed. [2025-11-04T17:17:57.872859] [INFO ] [MAIN ] APO: Fetched primary data is valid. [2025-11-04T17:17:57.872926] [DEBUG] [LOCAL ] APO: Saving primary data. [2025-11-04T17:17:57.874435] [DEBUG] [LOCAL ] APO: Trimmed 77455 records to 53370 (keeping 200 days of data) [2025-11-04T17:17:58.208109] [DEBUG] [LOCAL ] APO: Primary data saved (53370 records). [2025-11-04T17:17:58.208391] [INFO ] [MAIN ] APO: Full history saved. [2025-11-04T17:17:58.208570] [INFO ] [MAIN ] Update Progress: 136/2367 symbols completed. APO took 48.2s. Estimated remaining: 4h 53m 53s. Rate: 7.6 symbols/min [2025-11-04T17:17:58.417290] [INFO ] [MAIN ] ARCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:17:58.417713] [DEBUG] [LOCAL ] ARCO: No existing data file found in primary storage. [2025-11-04T17:17:58.417770] [WARN ] [MAIN ] ARCO: No existing data found, performing full refresh instead. [2025-11-04T17:17:58.417908] [INFO ] [MAIN ] ARCO: Wiping existing primary data before full refresh. [2025-11-04T17:17:58.436026] [INFO ] [LOCAL ] ARCO: Starting deletion of primary data. [2025-11-04T17:17:58.449200] [DEBUG] [LOCAL ] ARCO: No primary data file found to delete. [2025-11-04T17:17:58.449836] [INFO ] [MAIN ] ARCO: Starting full primary refresh. [2025-11-04T17:17:58.456835] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-11-04T17:17:58.569036] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-10. [2025-11-04T17:17:58.829722] [INFO ] [REMOTE ] ARCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:17:59.372902] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-02. [2025-11-04T17:17:59.389388] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-11-04T17:17:59.844118] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-11-04T17:18:00.167108] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-01. [2025-11-04T17:18:00.745841] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-11-04T17:18:00.967069] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-11. [2025-11-04T17:18:01.642602] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-11-04T17:18:01.767046] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-03. [2025-11-04T17:18:02.288170] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-11 (1.32 seconds). [2025-11-04T17:18:02.290285] [INFO ] [REMOTE ] AQN: Full fetch complete (75245 records). [2025-11-04T17:18:02.290442] [DEBUG] [VALIDATOR ] AQN: Validating primary data. [2025-11-04T17:18:02.576980] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-11. [2025-11-04T17:18:02.683465] [DEBUG] [VALIDATOR ] AQN: All expected trading days are present. [2025-11-04T17:18:02.683557] [DEBUG] [VALIDATOR ] AQN: Primary data validation passed. [2025-11-04T17:18:02.684311] [INFO ] [MAIN ] AQN: Fetched primary data is valid. [2025-11-04T17:18:02.684376] [DEBUG] [LOCAL ] AQN: Saving primary data. [2025-11-04T17:18:02.685838] [DEBUG] [LOCAL ] AQN: Trimmed 75245 records to 51506 (keeping 200 days of data) [2025-11-04T17:18:02.962753] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-11 (0.39 seconds). [2025-11-04T17:18:02.972797] [INFO ] [REMOTE ] APTV: Full fetch complete (75315 records). [2025-11-04T17:18:02.973762] [DEBUG] [VALIDATOR ] APTV: Validating primary data. [2025-11-04T17:18:02.978437] [FATAL] [VALIDATOR ] APTV: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:18:02.985764] [FATAL] [MAIN ] APTV: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:18:02.985673] [DEBUG] [LOCAL ] AQN: Primary data saved (51506 records). [2025-11-04T17:18:02.985924] [WARN ] [MAIN ] APTV: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:18:02.986090] [INFO ] [MAIN ] AQN: Full history saved. [2025-11-04T17:18:02.986196] [INFO ] [MAIN ] Update Progress: 137/2367 symbols completed. AQN took 33.5s. Estimated remaining: 4h 52m 55s. Rate: 7.6 symbols/min [2025-11-04T17:18:02.986373] [INFO ] [LOCAL ] APTV: Starting deletion of primary data. [2025-11-04T17:18:02.999506] [DEBUG] [LOCAL ] APTV: No primary data file found to delete. [2025-11-04T17:18:03.007438] [INFO ] [LOCAL ] APTV: Starting deletion of derivative data. [2025-11-04T17:18:03.056829] [DEBUG] [LOCAL ] APTV: No derivative data file found to delete. [2025-11-04T17:18:03.062808] [INFO ] [MAIN ] Update Progress: 138/2367 symbols completed. APTV took 33.9s. Estimated remaining: 4h 50m 41s. Rate: 7.7 symbols/min [2025-11-04T17:18:03.268119] [INFO ] [MAIN ] ARDC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:18:03.342720] [INFO ] [MAIN ] ARDT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:18:03.343137] [DEBUG] [LOCAL ] ARDT: No existing data file found in primary storage. [2025-11-04T17:18:03.343229] [WARN ] [MAIN ] ARDT: No existing data found, performing full refresh instead. [2025-11-04T17:18:03.343456] [INFO ] [MAIN ] ARDT: Wiping existing primary data before full refresh. [2025-11-04T17:18:03.343507] [INFO ] [LOCAL ] ARDT: Starting deletion of primary data. [2025-11-04T17:18:03.343550] [DEBUG] [LOCAL ] ARDT: No primary data file found to delete. [2025-11-04T17:18:03.343583] [INFO ] [MAIN ] ARDT: Starting full primary refresh. [2025-11-04T17:18:03.384494] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-02. [2025-11-04T17:18:03.456834] [DEBUG] [LOCAL ] ARDC: Loaded 14885 existing records from primary storage. [2025-11-04T17:18:03.469194] [INFO ] [MAIN ] ARDC: Loaded 14885 existing records. [2025-11-04T17:18:03.703441] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-11-04T17:18:03.874381] [WARN ] [MAIN ] ARDC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:18:03.904740] [INFO ] [MAIN ] ARDC: Wiping existing primary data before full refresh. [2025-11-04T17:18:03.905109] [INFO ] [LOCAL ] ARDC: Starting deletion of primary data. [2025-11-04T17:18:03.915975] [DEBUG] [LOCAL ] ARDC: Deleted primary data file. [2025-11-04T17:18:03.921162] [INFO ] [MAIN ] ARDC: Starting full primary refresh. [2025-11-04T17:18:04.089605] [INFO ] [REMOTE ] ARDT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:18:04.177048] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-04. [2025-11-04T17:18:04.346592] [INFO ] [REMOTE ] ARDC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:18:04.552157] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T17:18:04.967077] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-03. [2025-11-04T17:18:05.770952] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-01. [2025-11-04T17:18:05.776969] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-11-04T17:18:05.964245] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-11-04T17:18:06.567049] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-01. [2025-11-04T17:18:06.999498] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T17:18:07.357634] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-11-04T17:18:07.367004] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-02. [2025-11-04T17:18:07.930018] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-11-04T17:18:08.167049] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-05. [2025-11-04T17:18:08.967032] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-04. [2025-11-04T17:18:09.450586] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T17:18:09.767046] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-02. [2025-11-04T17:18:10.173079] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T17:18:10.567056] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-03. [2025-11-04T17:18:10.670171] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T17:18:11.143049] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-11-04T17:18:11.367041] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-06. [2025-11-04T17:18:12.167012] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-03. [2025-11-04T17:18:12.810069] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-11-04T17:18:12.967028] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-07. [2025-11-04T17:18:13.195845] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T17:18:13.767073] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-04. [2025-11-04T17:18:14.567018] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-04. [2025-11-04T17:18:14.652634] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-11-04T17:18:14.687162] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T17:18:15.259197] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-11-04T17:18:15.367044] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-05. [2025-11-04T17:18:15.910315] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-11-04T17:18:16.167098] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-06. [2025-11-04T17:18:16.780009] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T17:18:16.967055] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-08. [2025-11-04T17:18:17.767037] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-07. [2025-11-04T17:18:18.567057] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-05. [2025-11-04T17:18:18.671881] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T17:18:18.723988] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-11-04T17:18:19.367109] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-09. [2025-11-04T17:18:19.524247] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T17:18:20.167108] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-06. [2025-11-04T17:18:20.769069] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T17:18:20.948997] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T17:18:20.966994] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-07. [2025-11-04T17:18:21.767483] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-05. [2025-11-04T17:18:21.973614] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T17:18:22.567032] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-08. [2025-11-04T17:18:23.367009] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-08. [2025-11-04T17:18:23.429663] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-11-04T17:18:23.565567] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T17:18:23.908059] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-11-04T17:18:24.167142] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-09. [2025-11-04T17:18:24.893008] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-11-04T17:18:24.967040] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-10. [2025-11-04T17:18:25.715852] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-11-04T17:18:25.767082] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-11. [2025-11-04T17:18:26.393750] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T17:18:26.394713] [INFO ] [REMOTE ] ARDC: Full fetch complete (21602 records). [2025-11-04T17:18:26.394784] [DEBUG] [VALIDATOR ] ARDC: Validating primary data. [2025-11-04T17:18:26.395244] [FATAL] [VALIDATOR ] ARDC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:18:26.395338] [FATAL] [MAIN ] ARDC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:18:26.395412] [WARN ] [MAIN ] ARDC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:18:26.395502] [INFO ] [LOCAL ] ARDC: Starting deletion of primary data. [2025-11-04T17:18:26.395577] [DEBUG] [LOCAL ] ARDC: No primary data file found to delete. [2025-11-04T17:18:26.395613] [INFO ] [LOCAL ] ARDC: Starting deletion of derivative data. [2025-11-04T17:18:26.395798] [DEBUG] [LOCAL ] ARDC: Deleted derivative data file. [2025-11-04T17:18:26.395899] [INFO ] [MAIN ] Update Progress: 139/2367 symbols completed. ARDC took 23.4s. Estimated remaining: 4h 54m 41s. Rate: 7.6 symbols/min [2025-11-04T17:18:26.572260] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-10. [2025-11-04T17:18:26.631660] [INFO ] [MAIN ] ARE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:18:26.633212] [DEBUG] [LOCAL ] ARE: No existing data file found in primary storage. [2025-11-04T17:18:26.633281] [WARN ] [MAIN ] ARE: No existing data found, performing full refresh instead. [2025-11-04T17:18:26.633443] [INFO ] [MAIN ] ARE: Wiping existing primary data before full refresh. [2025-11-04T17:18:26.633497] [INFO ] [LOCAL ] ARE: Starting deletion of primary data. [2025-11-04T17:18:26.633548] [DEBUG] [LOCAL ] ARE: No primary data file found to delete. [2025-11-04T17:18:26.633599] [INFO ] [MAIN ] ARE: Starting full primary refresh. [2025-11-04T17:18:27.112814] [INFO ] [REMOTE ] ARE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:18:27.367063] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-06. [2025-11-04T17:18:28.167020] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-01. [2025-11-04T17:18:28.265948] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T17:18:28.384111] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T17:18:28.967091] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-07. [2025-11-04T17:18:29.284147] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T17:18:29.767053] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-02. [2025-11-04T17:18:30.264120] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:18:30.567134] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-11. [2025-11-04T17:18:30.904317] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T17:18:30.908845] [INFO ] [REMOTE ] AR: Full fetch complete (77736 records). [2025-11-04T17:18:30.909021] [DEBUG] [VALIDATOR ] AR: Validating primary data. [2025-11-04T17:18:31.336304] [DEBUG] [VALIDATOR ] AR: All expected trading days are present. [2025-11-04T17:18:31.336508] [DEBUG] [VALIDATOR ] AR: Primary data validation passed. [2025-11-04T17:18:31.337502] [INFO ] [MAIN ] AR: Fetched primary data is valid. [2025-11-04T17:18:31.337668] [DEBUG] [LOCAL ] AR: Saving primary data. [2025-11-04T17:18:31.350460] [DEBUG] [LOCAL ] AR: Trimmed 77736 records to 53589 (keeping 200 days of data) [2025-11-04T17:18:31.368060] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-09. [2025-11-04T17:18:31.372312] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-11-04T17:18:31.709486] [DEBUG] [LOCAL ] AR: Primary data saved (53589 records). [2025-11-04T17:18:31.709621] [INFO ] [MAIN ] AR: Full history saved. [2025-11-04T17:18:31.709727] [INFO ] [MAIN ] Update Progress: 140/2367 symbols completed. AR took 40.3s. Estimated remaining: 4h 53m 52s. Rate: 7.6 symbols/min [2025-11-04T17:18:31.877255] [INFO ] [MAIN ] ARES: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:18:31.923786] [DEBUG] [LOCAL ] ARES: Loaded 49578 existing records from primary storage. [2025-11-04T17:18:31.924102] [INFO ] [MAIN ] ARES: Loaded 49578 existing records. [2025-11-04T17:18:32.104840] [WARN ] [MAIN ] ARES: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:18:32.105063] [INFO ] [MAIN ] ARES: Wiping existing primary data before full refresh. [2025-11-04T17:18:32.105110] [INFO ] [LOCAL ] ARES: Starting deletion of primary data. [2025-11-04T17:18:32.105445] [DEBUG] [LOCAL ] ARES: Deleted primary data file. [2025-11-04T17:18:32.105501] [INFO ] [MAIN ] ARES: Starting full primary refresh. [2025-11-04T17:18:32.172360] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-03. [2025-11-04T17:18:32.480197] [INFO ] [REMOTE ] ARES: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:18:32.496021] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-11-04T17:18:32.967066] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-08. [2025-11-04T17:18:33.530309] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T17:18:33.767192] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-04. [2025-11-04T17:18:34.055844] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-11-04T17:18:34.567023] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-09. [2025-11-04T17:18:35.070836] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T17:18:35.367076] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-01. [2025-11-04T17:18:35.670613] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T17:18:36.167085] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-10. [2025-11-04T17:18:36.697912] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T17:18:36.967042] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-10. [2025-11-04T17:18:37.245437] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-11-04T17:18:37.767105] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-02. [2025-11-04T17:18:37.858098] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-11-04T17:18:38.567110] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-11. [2025-11-04T17:18:38.996247] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T17:18:39.367058] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-05. [2025-11-04T17:18:39.424355] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T17:18:39.426155] [INFO ] [REMOTE ] ARCO: Full fetch complete (64577 records). [2025-11-04T17:18:39.426252] [DEBUG] [VALIDATOR ] ARCO: Validating primary data. [2025-11-04T17:18:39.859432] [DEBUG] [VALIDATOR ] ARCO: All expected trading days are present. [2025-11-04T17:18:39.859535] [DEBUG] [VALIDATOR ] ARCO: Primary data validation passed. [2025-11-04T17:18:39.860245] [INFO ] [MAIN ] ARCO: Fetched primary data is valid. [2025-11-04T17:18:39.860309] [DEBUG] [LOCAL ] ARCO: Saving primary data. [2025-11-04T17:18:39.861550] [DEBUG] [LOCAL ] ARCO: Trimmed 64577 records to 43834 (keeping 200 days of data) [2025-11-04T17:18:40.106403] [DEBUG] [LOCAL ] ARCO: Primary data saved (43834 records). [2025-11-04T17:18:40.106532] [INFO ] [MAIN ] ARCO: Full history saved. [2025-11-04T17:18:40.106602] [INFO ] [MAIN ] Update Progress: 141/2367 symbols completed. ARCO took 41.9s. Estimated remaining: 4h 53m 51s. Rate: 7.6 symbols/min [2025-11-04T17:18:40.172175] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-11. [2025-11-04T17:18:40.276306] [INFO ] [MAIN ] ARI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:18:40.276474] [DEBUG] [LOCAL ] ARI: No existing data file found in primary storage. [2025-11-04T17:18:40.276511] [WARN ] [MAIN ] ARI: No existing data found, performing full refresh instead. [2025-11-04T17:18:40.276671] [INFO ] [MAIN ] ARI: Wiping existing primary data before full refresh. [2025-11-04T17:18:40.276728] [INFO ] [LOCAL ] ARI: Starting deletion of primary data. [2025-11-04T17:18:40.276772] [DEBUG] [LOCAL ] ARI: No primary data file found to delete. [2025-11-04T17:18:40.276805] [INFO ] [MAIN ] ARI: Starting full primary refresh. [2025-11-04T17:18:40.630383] [INFO ] [REMOTE ] ARI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:18:40.683127] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T17:18:40.967062] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-01. [2025-11-04T17:18:40.986864] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T17:18:40.988199] [INFO ] [REMOTE ] ARDT: Full fetch complete (41656 records). [2025-11-04T17:18:40.988274] [DEBUG] [VALIDATOR ] ARDT: Validating primary data. [2025-11-04T17:18:41.360798] [DEBUG] [VALIDATOR ] ARDT: All expected trading days are present. [2025-11-04T17:18:41.360880] [DEBUG] [VALIDATOR ] ARDT: Primary data validation passed. [2025-11-04T17:18:41.361349] [INFO ] [MAIN ] ARDT: Fetched primary data is valid. [2025-11-04T17:18:41.361449] [DEBUG] [LOCAL ] ARDT: Saving primary data. [2025-11-04T17:18:41.362425] [DEBUG] [LOCAL ] ARDT: Trimmed 41656 records to 27159 (keeping 200 days of data) [2025-11-04T17:18:41.520566] [DEBUG] [LOCAL ] ARDT: Primary data saved (27159 records). [2025-11-04T17:18:41.520720] [INFO ] [MAIN ] ARDT: Full history saved. [2025-11-04T17:18:41.520801] [INFO ] [MAIN ] Update Progress: 142/2367 symbols completed. ARDT took 38.5s. Estimated remaining: 4h 52m 1s. Rate: 7.6 symbols/min [2025-11-04T17:18:41.686588] [INFO ] [MAIN ] ARLO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:18:41.686803] [DEBUG] [LOCAL ] ARLO: No existing data file found in primary storage. [2025-11-04T17:18:41.686881] [WARN ] [MAIN ] ARLO: No existing data found, performing full refresh instead. [2025-11-04T17:18:41.687000] [INFO ] [MAIN ] ARLO: Wiping existing primary data before full refresh. [2025-11-04T17:18:41.687064] [INFO ] [LOCAL ] ARLO: Starting deletion of primary data. [2025-11-04T17:18:41.687132] [DEBUG] [LOCAL ] ARLO: No primary data file found to delete. [2025-11-04T17:18:41.687177] [INFO ] [MAIN ] ARLO: Starting full primary refresh. [2025-11-04T17:18:41.775751] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-03. [2025-11-04T17:18:42.075884] [INFO ] [REMOTE ] ARLO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:18:42.098837] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T17:18:42.567037] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-02. [2025-11-04T17:18:43.367043] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-06. [2025-11-04T17:18:43.420600] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-11-04T17:18:43.754198] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T17:18:44.167040] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-01. [2025-11-04T17:18:44.967047] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-03. [2025-11-04T17:18:45.107057] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-11-04T17:18:45.421981] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T17:18:45.767093] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-04. [2025-11-04T17:18:46.033294] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T17:18:46.567080] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-04. [2025-11-04T17:18:47.367012] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-07. [2025-11-04T17:18:47.446797] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-11-04T17:18:47.945437] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T17:18:48.167034] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-05. [2025-11-04T17:18:48.909522] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-11-04T17:18:48.967008] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-05. [2025-11-04T17:18:49.314134] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T17:18:49.767064] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-06. [2025-11-04T17:18:50.305618] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T17:18:50.567047] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-08. [2025-11-04T17:18:50.792624] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T17:18:51.367052] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-07. [2025-11-04T17:18:51.734120] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T17:18:52.167036] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-06. [2025-11-04T17:18:52.698233] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T17:18:52.967078] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-02. [2025-11-04T17:18:53.510403] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T17:18:53.767040] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-07. [2025-11-04T17:18:53.938246] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T17:18:54.567080] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-03. [2025-11-04T17:18:55.083108] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T17:18:55.367078] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-08. [2025-11-04T17:18:55.925334] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T17:18:56.167047] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-08. [2025-11-04T17:18:56.363620] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T17:18:56.967037] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-09. [2025-11-04T17:18:57.271535] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-11-04T17:18:57.767037] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-09. [2025-11-04T17:18:58.099844] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-11-04T17:18:58.567083] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-04. [2025-11-04T17:18:59.061709] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T17:18:59.367065] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-09. [2025-11-04T17:18:59.538538] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-11-04T17:19:00.167092] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-10. [2025-11-04T17:19:00.967039] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-10. [2025-11-04T17:19:01.275619] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-11-04T17:19:01.776022] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-05. [2025-11-04T17:19:01.798862] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-11-04T17:19:02.376986] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T17:19:02.567099] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-10. [2025-11-04T17:19:02.685373] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-11-04T17:19:03.367049] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-11. [2025-11-04T17:19:04.079809] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T17:19:04.167005] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-11. [2025-11-04T17:19:04.546977] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T17:19:04.548830] [INFO ] [REMOTE ] ARI: Full fetch complete (62268 records). [2025-11-04T17:19:04.548920] [DEBUG] [VALIDATOR ] ARI: Validating primary data. [2025-11-04T17:19:04.936553] [DEBUG] [VALIDATOR ] ARI: All expected trading days are present. [2025-11-04T17:19:04.936628] [DEBUG] [VALIDATOR ] ARI: Primary data validation passed. [2025-11-04T17:19:04.937320] [INFO ] [MAIN ] ARI: Fetched primary data is valid. [2025-11-04T17:19:04.937383] [DEBUG] [LOCAL ] ARI: Saving primary data. [2025-11-04T17:19:04.938700] [DEBUG] [LOCAL ] ARI: Trimmed 62268 records to 42896 (keeping 200 days of data) [2025-11-04T17:19:04.989914] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-11. [2025-11-04T17:19:05.192418] [DEBUG] [LOCAL ] ARI: Primary data saved (42896 records). [2025-11-04T17:19:05.192539] [INFO ] [MAIN ] ARI: Full history saved. [2025-11-04T17:19:05.192606] [INFO ] [MAIN ] Update Progress: 143/2367 symbols completed. ARI took 25.1s. Estimated remaining: 4h 55m 59s. Rate: 7.5 symbols/min [2025-11-04T17:19:05.374172] [INFO ] [MAIN ] ARMK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:19:05.379016] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-11 (1.21 seconds). [2025-11-04T17:19:05.381354] [INFO ] [REMOTE ] ARE: Full fetch complete (73259 records). [2025-11-04T17:19:05.381453] [DEBUG] [VALIDATOR ] ARE: Validating primary data. [2025-11-04T17:19:05.712853] [DEBUG] [LOCAL ] ARMK: Loaded 50273 existing records from primary storage. [2025-11-04T17:19:05.713284] [INFO ] [MAIN ] ARMK: Loaded 50273 existing records. [2025-11-04T17:19:05.774714] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-06. [2025-11-04T17:19:05.925830] [DEBUG] [VALIDATOR ] ARE: All expected trading days are present. [2025-11-04T17:19:05.936141] [DEBUG] [VALIDATOR ] ARE: Primary data validation passed. [2025-11-04T17:19:05.942032] [INFO ] [MAIN ] ARE: Fetched primary data is valid. [2025-11-04T17:19:05.954525] [DEBUG] [LOCAL ] ARE: Saving primary data. [2025-11-04T17:19:05.994824] [WARN ] [MAIN ] ARMK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:19:05.995151] [INFO ] [MAIN ] ARMK: Wiping existing primary data before full refresh. [2025-11-04T17:19:05.995213] [INFO ] [LOCAL ] ARMK: Starting deletion of primary data. [2025-11-04T17:19:05.995328] [DEBUG] [LOCAL ] ARE: Trimmed 73259 records to 50919 (keeping 200 days of data) [2025-11-04T17:19:05.995601] [DEBUG] [LOCAL ] ARMK: Deleted primary data file. [2025-11-04T17:19:05.995710] [INFO ] [MAIN ] ARMK: Starting full primary refresh. [2025-11-04T17:19:06.520545] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-11 (1.53 seconds). [2025-11-04T17:19:06.543782] [INFO ] [REMOTE ] ARES: Full fetch complete (72108 records). [2025-11-04T17:19:06.549478] [DEBUG] [VALIDATOR ] ARES: Validating primary data. [2025-11-04T17:19:06.757008] [INFO ] [REMOTE ] ARMK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:19:06.791411] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-01. [2025-11-04T17:19:06.799070] [DEBUG] [LOCAL ] ARE: Primary data saved (50919 records). [2025-11-04T17:19:06.804421] [INFO ] [MAIN ] ARE: Full history saved. [2025-11-04T17:19:06.810803] [INFO ] [MAIN ] Update Progress: 144/2367 symbols completed. ARE took 40.4s. Estimated remaining: 4h 54m 13s. Rate: 7.6 symbols/min [2025-11-04T17:19:07.215627] [INFO ] [MAIN ] AROC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:19:07.317995] [DEBUG] [VALIDATOR ] ARES: All expected trading days are present. [2025-11-04T17:19:07.318190] [DEBUG] [VALIDATOR ] ARES: Primary data validation passed. [2025-11-04T17:19:07.318993] [INFO ] [MAIN ] ARES: Fetched primary data is valid. [2025-11-04T17:19:07.319134] [DEBUG] [LOCAL ] ARES: Saving primary data. [2025-11-04T17:19:07.342977] [DEBUG] [LOCAL ] ARES: Trimmed 72108 records to 49594 (keeping 200 days of data) [2025-11-04T17:19:07.344605] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-11-04T17:19:07.416795] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-07. [2025-11-04T17:19:07.516789] [DEBUG] [LOCAL ] AROC: Loaded 49018 existing records from primary storage. [2025-11-04T17:19:07.517200] [INFO ] [MAIN ] AROC: Loaded 49018 existing records. [2025-11-04T17:19:07.829248] [WARN ] [MAIN ] AROC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:19:07.857428] [INFO ] [MAIN ] AROC: Wiping existing primary data before full refresh. [2025-11-04T17:19:07.870361] [INFO ] [LOCAL ] AROC: Starting deletion of primary data. [2025-11-04T17:19:07.874827] [DEBUG] [LOCAL ] AROC: Deleted primary data file. [2025-11-04T17:19:07.875130] [INFO ] [MAIN ] AROC: Starting full primary refresh. [2025-11-04T17:19:07.965170] [DEBUG] [LOCAL ] ARES: Primary data saved (49594 records). [2025-11-04T17:19:07.990008] [INFO ] [MAIN ] ARES: Full history saved. [2025-11-04T17:19:07.995208] [INFO ] [MAIN ] Update Progress: 145/2367 symbols completed. ARES took 36.3s. Estimated remaining: 4h 52m 22s. Rate: 7.6 symbols/min [2025-11-04T17:19:08.372301] [INFO ] [MAIN ] ARR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:19:08.450616] [INFO ] [REMOTE ] AROC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:19:08.451188] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-01. [2025-11-04T17:19:08.486805] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-11-04T17:19:08.543179] [DEBUG] [LOCAL ] ARR: Loaded 52017 existing records from primary storage. [2025-11-04T17:19:08.543694] [INFO ] [MAIN ] ARR: Loaded 52017 existing records. [2025-11-04T17:19:08.753770] [WARN ] [MAIN ] ARR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:19:08.754291] [INFO ] [MAIN ] ARR: Wiping existing primary data before full refresh. [2025-11-04T17:19:08.754802] [INFO ] [LOCAL ] ARR: Starting deletion of primary data. [2025-11-04T17:19:08.755916] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T17:19:08.765452] [DEBUG] [LOCAL ] ARR: Deleted primary data file. [2025-11-04T17:19:08.765545] [INFO ] [MAIN ] ARR: Starting full primary refresh. [2025-11-04T17:19:08.972466] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-08. [2025-11-04T17:19:09.119149] [INFO ] [REMOTE ] ARR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:19:09.767021] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-02. [2025-11-04T17:19:09.868384] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-11-04T17:19:10.227231] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T17:19:10.567050] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-01. [2025-11-04T17:19:10.898563] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T17:19:11.367043] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-02. [2025-11-04T17:19:12.097855] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-11-04T17:19:12.167132] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-09. [2025-11-04T17:19:12.654196] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T17:19:12.967053] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-03. [2025-11-04T17:19:13.332736] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T17:19:13.767053] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-02. [2025-11-04T17:19:14.150173] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-11-04T17:19:14.567052] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-10. [2025-11-04T17:19:15.367038] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-04. [2025-11-04T17:19:15.955089] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T17:19:16.167035] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-03. [2025-11-04T17:19:16.479898] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-02 (2.71 seconds). [2025-11-04T17:19:16.622195] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T17:19:16.967117] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-03. [2025-11-04T17:19:17.433301] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T17:19:17.767105] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-05. [2025-11-04T17:19:18.429743] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T17:19:18.567037] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-04. [2025-11-04T17:19:18.924487] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T17:19:19.367040] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-06. [2025-11-04T17:19:19.854737] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T17:19:20.167066] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-04. [2025-11-04T17:19:20.755453] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T17:19:20.967116] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-11. [2025-11-04T17:19:21.680068] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T17:19:21.767044] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-05. [2025-11-04T17:19:22.158874] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T17:19:22.160861] [INFO ] [REMOTE ] ARLO: Full fetch complete (63094 records). [2025-11-04T17:19:22.160982] [DEBUG] [VALIDATOR ] ARLO: Validating primary data. [2025-11-04T17:19:22.161756] [FATAL] [VALIDATOR ] ARLO: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:19:22.161848] [FATAL] [MAIN ] ARLO: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:19:22.161895] [WARN ] [MAIN ] ARLO: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:19:22.162059] [INFO ] [LOCAL ] ARLO: Starting deletion of primary data. [2025-11-04T17:19:22.162173] [DEBUG] [LOCAL ] ARLO: No primary data file found to delete. [2025-11-04T17:19:22.162215] [INFO ] [LOCAL ] ARLO: Starting deletion of derivative data. [2025-11-04T17:19:22.162256] [DEBUG] [LOCAL ] ARLO: No derivative data file found to delete. [2025-11-04T17:19:22.162320] [INFO ] [MAIN ] Update Progress: 146/2367 symbols completed. ARLO took 40.6s. Estimated remaining: 4h 53m 49s. Rate: 7.6 symbols/min [2025-11-04T17:19:22.352025] [INFO ] [MAIN ] ARW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:19:22.401041] [DEBUG] [LOCAL ] ARW: Loaded 33027 existing records from primary storage. [2025-11-04T17:19:22.401440] [INFO ] [MAIN ] ARW: Loaded 33027 existing records. [2025-11-04T17:19:22.567707] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-07. [2025-11-04T17:19:22.590476] [WARN ] [MAIN ] ARW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:19:22.590947] [INFO ] [MAIN ] ARW: Wiping existing primary data before full refresh. [2025-11-04T17:19:22.591011] [INFO ] [LOCAL ] ARW: Starting deletion of primary data. [2025-11-04T17:19:22.591392] [DEBUG] [LOCAL ] ARW: Deleted primary data file. [2025-11-04T17:19:22.591438] [INFO ] [MAIN ] ARW: Starting full primary refresh. [2025-11-04T17:19:22.978101] [INFO ] [REMOTE ] ARW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:19:23.013494] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-11-04T17:19:23.367081] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-06. [2025-11-04T17:19:23.958687] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-11-04T17:19:24.167084] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-08. [2025-11-04T17:19:24.699493] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T17:19:24.967108] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-01. [2025-11-04T17:19:25.390990] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T17:19:25.767057] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-07. [2025-11-04T17:19:26.045495] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T17:19:26.567134] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-09. [2025-11-04T17:19:27.015796] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T17:19:27.367158] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-05. [2025-11-04T17:19:27.868474] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T17:19:28.167091] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-02. [2025-11-04T17:19:28.736170] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T17:19:28.967013] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-06. [2025-11-04T17:19:29.158476] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T17:19:29.767139] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-03. [2025-11-04T17:19:30.147634] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T17:19:30.567046] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-07. [2025-11-04T17:19:30.744185] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T17:19:31.367047] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-10. [2025-11-04T17:19:32.009623] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:19:32.167012] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-08. [2025-11-04T17:19:32.638946] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T17:19:32.967089] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-11. [2025-11-04T17:19:33.422186] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T17:19:33.767046] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-08. [2025-11-04T17:19:34.039160] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T17:19:34.040592] [INFO ] [REMOTE ] AROC: Full fetch complete (71115 records). [2025-11-04T17:19:34.040715] [DEBUG] [VALIDATOR ] AROC: Validating primary data. [2025-11-04T17:19:34.041282] [FATAL] [VALIDATOR ] AROC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:19:34.041360] [FATAL] [MAIN ] AROC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:19:34.041404] [WARN ] [MAIN ] AROC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:19:34.041491] [INFO ] [LOCAL ] AROC: Starting deletion of primary data. [2025-11-04T17:19:34.041567] [DEBUG] [LOCAL ] AROC: No primary data file found to delete. [2025-11-04T17:19:34.041622] [INFO ] [LOCAL ] AROC: Starting deletion of derivative data. [2025-11-04T17:19:34.041824] [DEBUG] [LOCAL ] AROC: Deleted derivative data file. [2025-11-04T17:19:34.041928] [INFO ] [MAIN ] Update Progress: 147/2367 symbols completed. AROC took 27.2s. Estimated remaining: 4h 54m 41s. Rate: 7.5 symbols/min [2025-11-04T17:19:34.219943] [INFO ] [MAIN ] ARX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:19:34.220101] [DEBUG] [LOCAL ] ARX: No existing data file found in primary storage. [2025-11-04T17:19:34.220137] [WARN ] [MAIN ] ARX: No existing data found, performing full refresh instead. [2025-11-04T17:19:34.220245] [INFO ] [MAIN ] ARX: Wiping existing primary data before full refresh. [2025-11-04T17:19:34.220286] [INFO ] [LOCAL ] ARX: Starting deletion of primary data. [2025-11-04T17:19:34.220346] [DEBUG] [LOCAL ] ARX: No primary data file found to delete. [2025-11-04T17:19:34.220381] [INFO ] [MAIN ] ARX: Starting full primary refresh. [2025-11-04T17:19:34.572271] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-04. [2025-11-04T17:19:34.609045] [INFO ] [REMOTE ] ARX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:19:35.284852] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-11-04T17:19:35.367047] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-09. [2025-11-04T17:19:35.854117] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T17:19:36.167084] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-09. [2025-11-04T17:19:36.568852] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T17:19:36.967040] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-01. [2025-11-04T17:19:37.143513] [DEBUG] [REMOTE ] ARX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:19:37.143807] [WARN ] [REMOTE ] ARX: No data returned for month 2025-01 [2025-11-04T17:19:37.143893] [FATAL] [REMOTE ] ARX: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:19:37.143942] [FATAL] [MAIN ] ARX: Failed to fetch primary data. [2025-11-04T17:19:37.144000] [WARN ] [MAIN ] ARX: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:19:37.144071] [INFO ] [LOCAL ] ARX: Starting deletion of primary data. [2025-11-04T17:19:37.144133] [DEBUG] [LOCAL ] ARX: No primary data file found to delete. [2025-11-04T17:19:37.144169] [INFO ] [LOCAL ] ARX: Starting deletion of derivative data. [2025-11-04T17:19:37.144208] [DEBUG] [LOCAL ] ARX: No derivative data file found to delete. [2025-11-04T17:19:37.144253] [INFO ] [MAIN ] Update Progress: 148/2367 symbols completed. ARX took 3.1s. Estimated remaining: 4h 53m 20s. Rate: 7.6 symbols/min [2025-11-04T17:19:37.318152] [INFO ] [MAIN ] AS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:19:37.366132] [DEBUG] [LOCAL ] AS: Loaded 53208 existing records from primary storage. [2025-11-04T17:19:37.366448] [INFO ] [MAIN ] AS: Loaded 53208 existing records. [2025-11-04T17:19:37.549351] [WARN ] [MAIN ] AS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:19:37.549673] [INFO ] [MAIN ] AS: Wiping existing primary data before full refresh. [2025-11-04T17:19:37.549750] [INFO ] [LOCAL ] AS: Starting deletion of primary data. [2025-11-04T17:19:37.550003] [DEBUG] [LOCAL ] AS: Deleted primary data file. [2025-11-04T17:19:37.550054] [INFO ] [MAIN ] AS: Starting full primary refresh. [2025-11-04T17:19:37.774982] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-10. [2025-11-04T17:19:37.975959] [INFO ] [REMOTE ] AS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:19:37.978284] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-11-04T17:19:38.567090] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-01. [2025-11-04T17:19:39.286265] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T17:19:39.367011] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-11. [2025-11-04T17:19:39.904607] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T17:19:40.167054] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-05. [2025-11-04T17:19:40.456979] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T17:19:40.459441] [INFO ] [REMOTE ] ARMK: Full fetch complete (72944 records). [2025-11-04T17:19:40.459586] [DEBUG] [VALIDATOR ] ARMK: Validating primary data. [2025-11-04T17:19:40.903943] [DEBUG] [VALIDATOR ] ARMK: All expected trading days are present. [2025-11-04T17:19:40.904043] [DEBUG] [VALIDATOR ] ARMK: Primary data validation passed. [2025-11-04T17:19:40.904827] [INFO ] [MAIN ] ARMK: Fetched primary data is valid. [2025-11-04T17:19:40.904892] [DEBUG] [LOCAL ] ARMK: Saving primary data. [2025-11-04T17:19:40.906370] [DEBUG] [LOCAL ] ARMK: Trimmed 72944 records to 50278 (keeping 200 days of data) [2025-11-04T17:19:40.969978] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-02. [2025-11-04T17:19:41.221782] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T17:19:41.249012] [DEBUG] [LOCAL ] ARMK: Primary data saved (50278 records). [2025-11-04T17:19:41.249176] [INFO ] [MAIN ] ARMK: Full history saved. [2025-11-04T17:19:41.251261] [INFO ] [MAIN ] Update Progress: 149/2367 symbols completed. ARMK took 36.1s. Estimated remaining: 4h 52m 15s. Rate: 7.6 symbols/min [2025-11-04T17:19:41.431449] [INFO ] [MAIN ] ASA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:19:41.450339] [DEBUG] [LOCAL ] ASA: Loaded 15954 existing records from primary storage. [2025-11-04T17:19:41.450591] [INFO ] [MAIN ] ASA: Loaded 15954 existing records. [2025-11-04T17:19:41.644592] [WARN ] [MAIN ] ASA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:19:41.644830] [INFO ] [MAIN ] ASA: Wiping existing primary data before full refresh. [2025-11-04T17:19:41.644878] [INFO ] [LOCAL ] ASA: Starting deletion of primary data. [2025-11-04T17:19:41.645048] [DEBUG] [LOCAL ] ASA: Deleted primary data file. [2025-11-04T17:19:41.645106] [INFO ] [MAIN ] ASA: Starting full primary refresh. [2025-11-04T17:19:41.773414] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-10. [2025-11-04T17:19:42.053421] [INFO ] [REMOTE ] ASA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:19:42.422285] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-11-04T17:19:42.567036] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-03. [2025-11-04T17:19:43.367082] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-06. [2025-11-04T17:19:43.644710] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-11-04T17:19:43.920758] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-11-04T17:19:44.167005] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-11. [2025-11-04T17:19:44.189619] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-11-04T17:19:44.967072] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-01. [2025-11-04T17:19:45.475161] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-11 (1.31 seconds). [2025-11-04T17:19:45.477261] [INFO ] [REMOTE ] ARR: Full fetch complete (75516 records). [2025-11-04T17:19:45.477419] [DEBUG] [VALIDATOR ] ARR: Validating primary data. [2025-11-04T17:19:45.705410] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-11-04T17:19:45.772152] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-07. [2025-11-04T17:19:45.902734] [DEBUG] [VALIDATOR ] ARR: All expected trading days are present. [2025-11-04T17:19:45.902832] [DEBUG] [VALIDATOR ] ARR: Primary data validation passed. [2025-11-04T17:19:45.903579] [INFO ] [MAIN ] ARR: Fetched primary data is valid. [2025-11-04T17:19:45.903687] [DEBUG] [LOCAL ] ARR: Saving primary data. [2025-11-04T17:19:45.906892] [DEBUG] [LOCAL ] ARR: Trimmed 75516 records to 52003 (keeping 200 days of data) [2025-11-04T17:19:46.225357] [DEBUG] [LOCAL ] ARR: Primary data saved (52003 records). [2025-11-04T17:19:46.225483] [INFO ] [MAIN ] ARR: Full history saved. [2025-11-04T17:19:46.226347] [INFO ] [MAIN ] Update Progress: 150/2367 symbols completed. ARR took 38.2s. Estimated remaining: 4h 51m 24s. Rate: 7.6 symbols/min [2025-11-04T17:19:46.390470] [INFO ] [MAIN ] ASAN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:19:46.447093] [DEBUG] [LOCAL ] ASAN: Loaded 52708 existing records from primary storage. [2025-11-04T17:19:46.447415] [INFO ] [MAIN ] ASAN: Loaded 52708 existing records. [2025-11-04T17:19:46.572189] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-04. [2025-11-04T17:19:46.631137] [WARN ] [MAIN ] ASAN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:19:46.632337] [INFO ] [MAIN ] ASAN: Wiping existing primary data before full refresh. [2025-11-04T17:19:46.632390] [INFO ] [LOCAL ] ASAN: Starting deletion of primary data. [2025-11-04T17:19:46.632822] [DEBUG] [LOCAL ] ASAN: Deleted primary data file. [2025-11-04T17:19:46.632977] [INFO ] [MAIN ] ASAN: Starting full primary refresh. [2025-11-04T17:19:46.839213] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T17:19:47.019747] [INFO ] [REMOTE ] ASAN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:19:47.367029] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-01. [2025-11-04T17:19:47.867869] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T17:19:48.167086] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-08. [2025-11-04T17:19:48.967091] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-05. [2025-11-04T17:19:49.068076] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-11-04T17:19:49.156272] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-11-04T17:19:49.767143] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-02. [2025-11-04T17:19:50.257500] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T17:19:50.567056] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-09. [2025-11-04T17:19:51.074344] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T17:19:51.367070] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-03. [2025-11-04T17:19:51.645684] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T17:19:52.167077] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-02. [2025-11-04T17:19:52.578841] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-11-04T17:19:52.809098] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T17:19:52.967051] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-03. [2025-11-04T17:19:53.610895] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T17:19:53.767032] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-04. [2025-11-04T17:19:54.567051] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-10. [2025-11-04T17:19:55.100920] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T17:19:55.367097] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-04. [2025-11-04T17:19:55.650489] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-11-04T17:19:56.169540] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-11. [2025-11-04T17:19:56.171138] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-11-04T17:19:56.967003] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-05. [2025-11-04T17:19:57.044207] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T17:19:57.045797] [INFO ] [REMOTE ] ARW: Full fetch complete (49037 records). [2025-11-04T17:19:57.045895] [DEBUG] [VALIDATOR ] ARW: Validating primary data. [2025-11-04T17:19:57.419243] [DEBUG] [VALIDATOR ] ARW: All expected trading days are present. [2025-11-04T17:19:57.419324] [DEBUG] [VALIDATOR ] ARW: Primary data validation passed. [2025-11-04T17:19:57.419877] [INFO ] [MAIN ] ARW: Fetched primary data is valid. [2025-11-04T17:19:57.419937] [DEBUG] [LOCAL ] ARW: Saving primary data. [2025-11-04T17:19:57.420898] [DEBUG] [LOCAL ] ARW: Trimmed 49037 records to 33064 (keeping 200 days of data) [2025-11-04T17:19:57.623994] [DEBUG] [LOCAL ] ARW: Primary data saved (33064 records). [2025-11-04T17:19:57.628022] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T17:19:57.627770] [INFO ] [MAIN ] ARW: Full history saved. [2025-11-04T17:19:57.631926] [INFO ] [MAIN ] Update Progress: 151/2367 symbols completed. ARW took 35.5s. Estimated remaining: 4h 52m 7s. Rate: 7.6 symbols/min [2025-11-04T17:19:57.777598] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-05. [2025-11-04T17:19:57.834025] [INFO ] [MAIN ] ASB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:19:57.879616] [DEBUG] [LOCAL ] ASB: Loaded 49668 existing records from primary storage. [2025-11-04T17:19:57.879933] [INFO ] [MAIN ] ASB: Loaded 49668 existing records. [2025-11-04T17:19:58.065778] [WARN ] [MAIN ] ASB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:19:58.066019] [INFO ] [MAIN ] ASB: Wiping existing primary data before full refresh. [2025-11-04T17:19:58.066076] [INFO ] [LOCAL ] ASB: Starting deletion of primary data. [2025-11-04T17:19:58.066246] [DEBUG] [LOCAL ] ASB: Deleted primary data file. [2025-11-04T17:19:58.066284] [INFO ] [MAIN ] ASB: Starting full primary refresh. [2025-11-04T17:19:58.430314] [INFO ] [REMOTE ] ASB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:19:58.567051] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-01. [2025-11-04T17:19:59.248186] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-11-04T17:19:59.367046] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-06. [2025-11-04T17:19:59.921802] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T17:20:00.167051] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-06. [2025-11-04T17:20:00.753780] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-11-04T17:20:00.946129] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-11-04T17:20:00.967043] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-07. [2025-11-04T17:20:01.767077] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-07. [2025-11-04T17:20:02.567071] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-06. [2025-11-04T17:20:03.367109] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-02. [2025-11-04T17:20:03.544674] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-11-04T17:20:04.090946] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-11-04T17:20:04.167006] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-07. [2025-11-04T17:20:04.503532] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-07 (3.54 seconds). [2025-11-04T17:20:04.787458] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-11-04T17:20:04.967108] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-08. [2025-11-04T17:20:05.515354] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T17:20:05.661094] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-11-04T17:20:05.767022] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-09. [2025-11-04T17:20:06.567060] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-08. [2025-11-04T17:20:06.725849] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T17:20:07.367048] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-08. [2025-11-04T17:20:08.152913] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-11-04T17:20:08.166990] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-09. [2025-11-04T17:20:08.749754] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T17:20:08.967059] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-03. [2025-11-04T17:20:09.767134] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-09. [2025-11-04T17:20:10.572223] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-10. [2025-11-04T17:20:10.593001] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-11-04T17:20:10.841845] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-09 (2.67 seconds). [2025-11-04T17:20:11.208504] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T17:20:11.367020] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-10. [2025-11-04T17:20:11.393437] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T17:20:12.167077] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-11. [2025-11-04T17:20:12.803822] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T17:20:12.967061] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-11. [2025-11-04T17:20:13.198623] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T17:20:13.199926] [INFO ] [REMOTE ] ASA: Full fetch complete (21396 records). [2025-11-04T17:20:13.200017] [DEBUG] [VALIDATOR ] ASA: Validating primary data. [2025-11-04T17:20:13.570115] [DEBUG] [VALIDATOR ] ASA: All expected trading days are present. [2025-11-04T17:20:13.570222] [DEBUG] [VALIDATOR ] ASA: Primary data validation passed. [2025-11-04T17:20:13.570567] [INFO ] [MAIN ] ASA: Fetched primary data is valid. [2025-11-04T17:20:13.570611] [DEBUG] [LOCAL ] ASA: Saving primary data. [2025-11-04T17:20:13.571323] [DEBUG] [LOCAL ] ASA: Trimmed 21396 records to 16073 (keeping 200 days of data) [2025-11-04T17:20:13.662737] [DEBUG] [LOCAL ] ASA: Primary data saved (16073 records). [2025-11-04T17:20:13.662868] [INFO ] [MAIN ] ASA: Full history saved. [2025-11-04T17:20:13.662944] [INFO ] [MAIN ] Update Progress: 152/2367 symbols completed. ASA took 32.4s. Estimated remaining: 4h 53m 58s. Rate: 7.5 symbols/min [2025-11-04T17:20:13.772208] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-10. [2025-11-04T17:20:13.832827] [INFO ] [MAIN ] ASBA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:20:13.832967] [DEBUG] [LOCAL ] ASBA: No existing data file found in primary storage. [2025-11-04T17:20:13.833044] [WARN ] [MAIN ] ASBA: No existing data found, performing full refresh instead. [2025-11-04T17:20:13.833124] [INFO ] [MAIN ] ASBA: Wiping existing primary data before full refresh. [2025-11-04T17:20:13.833160] [INFO ] [LOCAL ] ASBA: Starting deletion of primary data. [2025-11-04T17:20:13.833200] [DEBUG] [LOCAL ] ASBA: No primary data file found to delete. [2025-11-04T17:20:13.833232] [INFO ] [MAIN ] ASBA: Starting full primary refresh. [2025-11-04T17:20:14.177751] [INFO ] [REMOTE ] ASBA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:20:14.355717] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-11 (1.39 seconds). [2025-11-04T17:20:14.357915] [INFO ] [REMOTE ] ASAN: Full fetch complete (76639 records). [2025-11-04T17:20:14.358011] [DEBUG] [VALIDATOR ] ASAN: Validating primary data. [2025-11-04T17:20:14.573010] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-04. [2025-11-04T17:20:14.788319] [DEBUG] [VALIDATOR ] ASAN: All expected trading days are present. [2025-11-04T17:20:14.788398] [DEBUG] [VALIDATOR ] ASAN: Primary data validation passed. [2025-11-04T17:20:14.789197] [INFO ] [MAIN ] ASAN: Fetched primary data is valid. [2025-11-04T17:20:14.789261] [DEBUG] [LOCAL ] ASAN: Saving primary data. [2025-11-04T17:20:14.790692] [DEBUG] [LOCAL ] ASAN: Trimmed 76639 records to 52713 (keeping 200 days of data) [2025-11-04T17:20:15.106497] [DEBUG] [LOCAL ] ASAN: Primary data saved (52713 records). [2025-11-04T17:20:15.106620] [INFO ] [MAIN ] ASAN: Full history saved. [2025-11-04T17:20:15.107710] [INFO ] [MAIN ] Update Progress: 153/2367 symbols completed. ASAN took 28.9s. Estimated remaining: 4h 52m 16s. Rate: 7.6 symbols/min [2025-11-04T17:20:15.289981] [INFO ] [MAIN ] ASC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:20:15.326387] [DEBUG] [LOCAL ] ASC: Loaded 36778 existing records from primary storage. [2025-11-04T17:20:15.326671] [INFO ] [MAIN ] ASC: Loaded 36778 existing records. [2025-11-04T17:20:15.373346] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-01. [2025-11-04T17:20:15.523224] [WARN ] [MAIN ] ASC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:20:15.523444] [INFO ] [MAIN ] ASC: Wiping existing primary data before full refresh. [2025-11-04T17:20:15.523491] [INFO ] [LOCAL ] ASC: Starting deletion of primary data. [2025-11-04T17:20:15.523707] [DEBUG] [LOCAL ] ASC: Deleted primary data file. [2025-11-04T17:20:15.523762] [INFO ] [MAIN ] ASC: Starting full primary refresh. [2025-11-04T17:20:15.900179] [INFO ] [REMOTE ] ASC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:20:15.990830] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T17:20:16.007601] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-11-04T17:20:16.167046] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-05. [2025-11-04T17:20:16.312833] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-10 (2.54 seconds). [2025-11-04T17:20:16.967098] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-02. [2025-11-04T17:20:17.374482] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-11-04T17:20:17.376113] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T17:20:17.767094] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-06. [2025-11-04T17:20:18.567108] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-03. [2025-11-04T17:20:18.965797] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T17:20:19.012900] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-11-04T17:20:19.367055] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-01. [2025-11-04T17:20:20.167077] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-11. [2025-11-04T17:20:20.677920] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T17:20:20.967089] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-07. [2025-11-04T17:20:21.433736] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-11 (1.27 seconds). [2025-11-04T17:20:21.435711] [INFO ] [REMOTE ] AS: Full fetch complete (76974 records). [2025-11-04T17:20:21.435798] [DEBUG] [VALIDATOR ] AS: Validating primary data. [2025-11-04T17:20:21.771908] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-04. [2025-11-04T17:20:21.825957] [DEBUG] [VALIDATOR ] AS: All expected trading days are present. [2025-11-04T17:20:21.826050] [DEBUG] [VALIDATOR ] AS: Primary data validation passed. [2025-11-04T17:20:21.826866] [INFO ] [MAIN ] AS: Fetched primary data is valid. [2025-11-04T17:20:21.826974] [DEBUG] [LOCAL ] AS: Saving primary data. [2025-11-04T17:20:21.828432] [DEBUG] [LOCAL ] AS: Trimmed 76974 records to 53206 (keeping 200 days of data) [2025-11-04T17:20:22.128897] [DEBUG] [LOCAL ] AS: Primary data saved (53206 records). [2025-11-04T17:20:22.129023] [INFO ] [MAIN ] AS: Full history saved. [2025-11-04T17:20:22.129098] [INFO ] [MAIN ] Update Progress: 154/2367 symbols completed. AS took 45.0s. Estimated remaining: 4h 51m 55s. Rate: 7.6 symbols/min [2025-11-04T17:20:22.303463] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T17:20:22.305055] [INFO ] [MAIN ] ASG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:20:22.307445] [DEBUG] [LOCAL ] ASG: No existing data file found in primary storage. [2025-11-04T17:20:22.307560] [WARN ] [MAIN ] ASG: No existing data found, performing full refresh instead. [2025-11-04T17:20:22.307698] [INFO ] [MAIN ] ASG: Wiping existing primary data before full refresh. [2025-11-04T17:20:22.307750] [INFO ] [LOCAL ] ASG: Starting deletion of primary data. [2025-11-04T17:20:22.307793] [DEBUG] [LOCAL ] ASG: No primary data file found to delete. [2025-11-04T17:20:22.307824] [INFO ] [MAIN ] ASG: Starting full primary refresh. [2025-11-04T17:20:22.577585] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-02. [2025-11-04T17:20:22.669115] [INFO ] [REMOTE ] ASG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:20:22.697341] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-11-04T17:20:23.367058] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-01. [2025-11-04T17:20:23.671074] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T17:20:24.167045] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-08. [2025-11-04T17:20:24.381312] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T17:20:24.967095] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-02. [2025-11-04T17:20:25.433272] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T17:20:25.670700] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T17:20:25.767062] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-03. [2025-11-04T17:20:26.567067] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-03. [2025-11-04T17:20:27.044142] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T17:20:27.367082] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-05. [2025-11-04T17:20:27.869342] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T17:20:27.899410] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-11-04T17:20:28.167108] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-04. [2025-11-04T17:20:28.967136] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-06. [2025-11-04T17:20:29.240316] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T17:20:29.356796] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-11-04T17:20:29.767061] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-09. [2025-11-04T17:20:30.567088] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-07. [2025-11-04T17:20:30.951845] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-11-04T17:20:31.130705] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T17:20:31.367054] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-04. [2025-11-04T17:20:32.167154] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-08. [2025-11-04T17:20:32.235424] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-11-04T17:20:32.735268] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-11-04T17:20:32.967039] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-09. [2025-11-04T17:20:33.317056] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-11-04T17:20:33.767041] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-10. [2025-11-04T17:20:34.567050] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-05. [2025-11-04T17:20:35.372973] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-05. [2025-11-04T17:20:35.399765] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-11-04T17:20:35.846352] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T17:20:36.167046] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-10. [2025-11-04T17:20:36.287522] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-11-04T17:20:36.616908] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-11-04T17:20:36.967060] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-11. [2025-11-04T17:20:37.489862] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-11 (0.52 seconds). [2025-11-04T17:20:37.490557] [INFO ] [REMOTE ] ASBA: Full fetch complete (5175 records). [2025-11-04T17:20:37.490623] [DEBUG] [VALIDATOR ] ASBA: Validating primary data. [2025-11-04T17:20:37.772177] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-06. [2025-11-04T17:20:37.845674] [DEBUG] [VALIDATOR ] ASBA: All expected trading days are present. [2025-11-04T17:20:37.845761] [DEBUG] [VALIDATOR ] ASBA: Primary data validation passed. [2025-11-04T17:20:37.845927] [INFO ] [MAIN ] ASBA: Fetched primary data is valid. [2025-11-04T17:20:37.845970] [DEBUG] [LOCAL ] ASBA: Saving primary data. [2025-11-04T17:20:37.846332] [DEBUG] [LOCAL ] ASBA: Trimmed 5175 records to 3524 (keeping 200 days of data) [2025-11-04T17:20:37.867408] [DEBUG] [LOCAL ] ASBA: Primary data saved (3524 records). [2025-11-04T17:20:37.867475] [INFO ] [MAIN ] ASBA: Full history saved. [2025-11-04T17:20:37.867538] [INFO ] [MAIN ] Update Progress: 155/2367 symbols completed. ASBA took 24.2s. Estimated remaining: 4h 53m 39s. Rate: 7.5 symbols/min [2025-11-04T17:20:38.040097] [INFO ] [MAIN ] ASGI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:20:38.040271] [DEBUG] [LOCAL ] ASGI: No existing data file found in primary storage. [2025-11-04T17:20:38.040326] [WARN ] [MAIN ] ASGI: No existing data found, performing full refresh instead. [2025-11-04T17:20:38.040432] [INFO ] [MAIN ] ASGI: Wiping existing primary data before full refresh. [2025-11-04T17:20:38.040473] [INFO ] [LOCAL ] ASGI: Starting deletion of primary data. [2025-11-04T17:20:38.040522] [DEBUG] [LOCAL ] ASGI: No primary data file found to delete. [2025-11-04T17:20:38.040553] [INFO ] [MAIN ] ASGI: Starting full primary refresh. [2025-11-04T17:20:38.385952] [INFO ] [REMOTE ] ASGI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:20:38.559810] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T17:20:38.566988] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-07. [2025-11-04T17:20:39.367518] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-01. [2025-11-04T17:20:39.370749] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-11-04T17:20:40.167063] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-08. [2025-11-04T17:20:40.362012] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T17:20:40.862482] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T17:20:40.967064] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-11. [2025-11-04T17:20:41.767050] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-09. [2025-11-04T17:20:42.117383] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T17:20:42.118907] [INFO ] [REMOTE ] ASB: Full fetch complete (72061 records). [2025-11-04T17:20:42.119041] [DEBUG] [VALIDATOR ] ASB: Validating primary data. [2025-11-04T17:20:42.120048] [FATAL] [VALIDATOR ] ASB: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:20:42.120147] [FATAL] [MAIN ] ASB: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:20:42.120189] [WARN ] [MAIN ] ASB: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:20:42.120226] [INFO ] [LOCAL ] ASB: Starting deletion of primary data. [2025-11-04T17:20:42.120281] [DEBUG] [LOCAL ] ASB: No primary data file found to delete. [2025-11-04T17:20:42.120314] [INFO ] [LOCAL ] ASB: Starting deletion of derivative data. [2025-11-04T17:20:42.120460] [DEBUG] [LOCAL ] ASB: Deleted derivative data file. [2025-11-04T17:20:42.121013] [INFO ] [MAIN ] Update Progress: 156/2367 symbols completed. ASB took 44.5s. Estimated remaining: 4h 52m 38s. Rate: 7.6 symbols/min [2025-11-04T17:20:42.297346] [INFO ] [MAIN ] ASGN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:20:42.297546] [DEBUG] [LOCAL ] ASGN: No existing data file found in primary storage. [2025-11-04T17:20:42.297584] [WARN ] [MAIN ] ASGN: No existing data found, performing full refresh instead. [2025-11-04T17:20:42.297725] [INFO ] [MAIN ] ASGN: Wiping existing primary data before full refresh. [2025-11-04T17:20:42.297772] [INFO ] [LOCAL ] ASGN: Starting deletion of primary data. [2025-11-04T17:20:42.297816] [DEBUG] [LOCAL ] ASGN: No primary data file found to delete. [2025-11-04T17:20:42.297849] [INFO ] [MAIN ] ASGN: Starting full primary refresh. [2025-11-04T17:20:42.569623] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-02. [2025-11-04T17:20:42.583977] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-11-04T17:20:42.756194] [INFO ] [REMOTE ] ASGN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:20:43.367019] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-01. [2025-11-04T17:20:43.465516] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T17:20:44.167052] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-06. [2025-11-04T17:20:44.925820] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-11-04T17:20:44.966998] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-02. [2025-11-04T17:20:45.767094] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-03. [2025-11-04T17:20:46.081520] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T17:20:46.397915] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-06 (2.23 seconds). [2025-11-04T17:20:46.567040] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-07. [2025-11-04T17:20:46.729052] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T17:20:47.367093] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-03. [2025-11-04T17:20:47.747457] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T17:20:48.167080] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-08. [2025-11-04T17:20:48.524522] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-11-04T17:20:48.967065] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-04. [2025-11-04T17:20:49.185818] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-11-04T17:20:49.767175] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-04. [2025-11-04T17:20:49.770524] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-11-04T17:20:50.567084] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-10. [2025-11-04T17:20:50.921188] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T17:20:51.367047] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-05. [2025-11-04T17:20:51.523086] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-11-04T17:20:52.167036] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-05. [2025-11-04T17:20:52.544197] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-11-04T17:20:52.967042] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-11. [2025-11-04T17:20:53.141451] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-11-04T17:20:53.603869] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T17:20:53.604999] [INFO ] [REMOTE ] ASG: Full fetch complete (29885 records). [2025-11-04T17:20:53.605081] [DEBUG] [VALIDATOR ] ASG: Validating primary data. [2025-11-04T17:20:53.772141] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-06. [2025-11-04T17:20:54.073026] [DEBUG] [VALIDATOR ] ASG: All expected trading days are present. [2025-11-04T17:20:54.073145] [DEBUG] [VALIDATOR ] ASG: Primary data validation passed. [2025-11-04T17:20:54.073597] [INFO ] [MAIN ] ASG: Fetched primary data is valid. [2025-11-04T17:20:54.073686] [DEBUG] [LOCAL ] ASG: Saving primary data. [2025-11-04T17:20:54.074592] [DEBUG] [LOCAL ] ASG: Trimmed 29885 records to 18870 (keeping 200 days of data) [2025-11-04T17:20:54.181884] [DEBUG] [LOCAL ] ASG: Primary data saved (18870 records). [2025-11-04T17:20:54.182020] [INFO ] [MAIN ] ASG: Full history saved. [2025-11-04T17:20:54.182091] [INFO ] [MAIN ] Update Progress: 157/2367 symbols completed. ASG took 32.1s. Estimated remaining: 4h 53m 28s. Rate: 7.5 symbols/min [2025-11-04T17:20:54.566204] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T17:20:54.570633] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-09. [2025-11-04T17:20:55.367075] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-06. [2025-11-04T17:20:55.843757] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-11-04T17:20:56.167044] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-07. [2025-11-04T17:20:56.296997] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T17:20:56.967033] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-07. [2025-11-04T17:20:57.063512] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T17:20:57.767053] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-10. [2025-11-04T17:20:58.277775] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T17:20:58.567107] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-08. [2025-11-04T17:20:58.943115] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T17:20:59.367184] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-11. [2025-11-04T17:20:59.625911] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-11-04T17:21:00.167123] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-09. [2025-11-04T17:21:00.274233] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T17:21:00.275455] [INFO ] [REMOTE ] ASC: Full fetch complete (55287 records). [2025-11-04T17:21:00.275546] [DEBUG] [VALIDATOR ] ASC: Validating primary data. [2025-11-04T17:21:00.276210] [FATAL] [VALIDATOR ] ASC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:21:00.276409] [FATAL] [MAIN ] ASC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:21:00.276470] [WARN ] [MAIN ] ASC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:21:00.276513] [INFO ] [LOCAL ] ASC: Starting deletion of primary data. [2025-11-04T17:21:00.276566] [DEBUG] [LOCAL ] ASC: No primary data file found to delete. [2025-11-04T17:21:00.276600] [INFO ] [LOCAL ] ASC: Starting deletion of derivative data. [2025-11-04T17:21:00.276776] [DEBUG] [LOCAL ] ASC: Deleted derivative data file. [2025-11-04T17:21:00.276898] [INFO ] [MAIN ] Update Progress: 158/2367 symbols completed. ASC took 45.2s. Estimated remaining: 4h 52m 54s. Rate: 7.5 symbols/min [2025-11-04T17:21:00.967064] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-08. [2025-11-04T17:21:01.559254] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T17:21:01.767060] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-10. [2025-11-04T17:21:01.937235] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-11-04T17:21:02.567012] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-09. [2025-11-04T17:21:02.998701] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T17:21:03.362451] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-11-04T17:21:03.367678] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-11. [2025-11-04T17:21:04.167112] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-10. [2025-11-04T17:21:04.359086] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T17:21:04.362284] [INFO ] [REMOTE ] ASGN: Full fetch complete (54410 records). [2025-11-04T17:21:04.362551] [DEBUG] [VALIDATOR ] ASGN: Validating primary data. [2025-11-04T17:21:04.862262] [DEBUG] [VALIDATOR ] ASGN: All expected trading days are present. [2025-11-04T17:21:04.862354] [DEBUG] [VALIDATOR ] ASGN: Primary data validation passed. [2025-11-04T17:21:04.863079] [INFO ] [MAIN ] ASGN: Fetched primary data is valid. [2025-11-04T17:21:04.863149] [DEBUG] [LOCAL ] ASGN: Saving primary data. [2025-11-04T17:21:04.864452] [DEBUG] [LOCAL ] ASGN: Trimmed 54410 records to 37061 (keeping 200 days of data) [2025-11-04T17:21:05.097550] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-11-04T17:21:05.115034] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-11. [2025-11-04T17:21:05.137729] [DEBUG] [LOCAL ] ASGN: Primary data saved (37061 records). [2025-11-04T17:21:05.137912] [INFO ] [MAIN ] ASGN: Full history saved. [2025-11-04T17:21:05.138062] [INFO ] [MAIN ] Update Progress: 159/2367 symbols completed. ASGN took 23.0s. Estimated remaining: 4h 52m 3s. Rate: 7.6 symbols/min [2025-11-04T17:21:05.820566] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T17:21:05.821634] [INFO ] [REMOTE ] ASGI: Full fetch complete (32304 records). [2025-11-04T17:21:05.821742] [DEBUG] [VALIDATOR ] ASGI: Validating primary data. [2025-11-04T17:21:05.822208] [FATAL] [VALIDATOR ] ASGI: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:21:05.822311] [FATAL] [MAIN ] ASGI: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:21:05.822353] [WARN ] [MAIN ] ASGI: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:21:05.822389] [INFO ] [LOCAL ] ASGI: Starting deletion of primary data. [2025-11-04T17:21:05.822442] [DEBUG] [LOCAL ] ASGI: No primary data file found to delete. [2025-11-04T17:21:05.822477] [INFO ] [LOCAL ] ASGI: Starting deletion of derivative data. [2025-11-04T17:21:05.822515] [DEBUG] [LOCAL ] ASGI: No derivative data file found to delete. [2025-11-04T17:21:05.822568] [INFO ] [MAIN ] Update Progress: 160/2367 symbols completed. ASGI took 28.0s. Estimated remaining: 4h 50m 15s. Rate: 7.6 symbols/min [2025-11-04T17:21:06.507088] [INFO ] [MAIN ] ASH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:21:06.538890] [DEBUG] [LOCAL ] ASH: No existing data file found in primary storage. [2025-11-04T17:21:06.507868] [INFO ] [MAIN ] ASR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:21:06.567816] [INFO ] [MAIN ] ASIX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:21:06.571711] [DEBUG] [LOCAL ] ASR: No existing data file found in primary storage. [2025-11-04T17:21:06.588993] [WARN ] [MAIN ] ASR: No existing data found, performing full refresh instead. [2025-11-04T17:21:06.592551] [INFO ] [MAIN ] ASR: Wiping existing primary data before full refresh. [2025-11-04T17:21:06.592380] [INFO ] [MAIN ] ASPN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:21:06.592459] [WARN ] [MAIN ] ASH: No existing data found, performing full refresh instead. [2025-11-04T17:21:06.592672] [DEBUG] [LOCAL ] ASIX: No existing data file found in primary storage. [2025-11-04T17:21:06.593001] [INFO ] [MAIN ] ASH: Wiping existing primary data before full refresh. [2025-11-04T17:21:06.593391] [WARN ] [MAIN ] ASIX: No existing data found, performing full refresh instead. [2025-11-04T17:21:06.593529] [INFO ] [LOCAL ] ASH: Starting deletion of primary data. [2025-11-04T17:21:06.593674] [INFO ] [LOCAL ] ASR: Starting deletion of primary data. [2025-11-04T17:21:06.593785] [INFO ] [MAIN ] ASIX: Wiping existing primary data before full refresh. [2025-11-04T17:21:06.594022] [DEBUG] [LOCAL ] ASH: No primary data file found to delete. [2025-11-04T17:21:06.594212] [INFO ] [MAIN ] ASH: Starting full primary refresh. [2025-11-04T17:21:06.594260] [INFO ] [LOCAL ] ASIX: Starting deletion of primary data. [2025-11-04T17:21:06.594154] [DEBUG] [LOCAL ] ASR: No primary data file found to delete. [2025-11-04T17:21:06.625597] [INFO ] [MAIN ] ASR: Starting full primary refresh. [2025-11-04T17:21:06.631790] [DEBUG] [LOCAL ] ASIX: No primary data file found to delete. [2025-11-04T17:21:06.637868] [INFO ] [MAIN ] ASIX: Starting full primary refresh. [2025-11-04T17:21:07.325353] [DEBUG] [LOCAL ] ASPN: Loaded 49745 existing records from primary storage. [2025-11-04T17:21:07.423803] [INFO ] [MAIN ] ASPN: Loaded 49745 existing records. [2025-11-04T17:21:07.726688] [INFO ] [REMOTE ] ASIX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:21:07.769571] [INFO ] [REMOTE ] ASH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:21:07.769875] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-01. [2025-11-04T17:21:07.761265] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-01. [2025-11-04T17:21:07.924346] [WARN ] [MAIN ] ASPN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:21:07.919073] [INFO ] [REMOTE ] ASR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:21:07.924800] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-01. [2025-11-04T17:21:07.924941] [INFO ] [MAIN ] ASPN: Wiping existing primary data before full refresh. [2025-11-04T17:21:07.925637] [INFO ] [LOCAL ] ASPN: Starting deletion of primary data. [2025-11-04T17:21:07.925907] [DEBUG] [LOCAL ] ASPN: Deleted primary data file. [2025-11-04T17:21:07.925963] [INFO ] [MAIN ] ASPN: Starting full primary refresh. [2025-11-04T17:21:08.276197] [INFO ] [REMOTE ] ASPN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:21:08.276516] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-01. [2025-11-04T17:21:08.572345] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-11-04T17:21:08.713836] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T17:21:08.967019] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-02. [2025-11-04T17:21:09.159584] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-11-04T17:21:09.387876] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-11-04T17:21:09.747698] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T17:21:09.766999] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-02. [2025-11-04T17:21:10.567028] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-03. [2025-11-04T17:21:11.198420] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-11-04T17:21:11.243023] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-11-04T17:21:11.367058] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-02. [2025-11-04T17:21:12.093723] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-11-04T17:21:12.167027] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-04. [2025-11-04T17:21:12.866438] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T17:21:12.967063] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-03. [2025-11-04T17:21:13.594018] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-11-04T17:21:13.767075] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-04. [2025-11-04T17:21:14.567027] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-02. [2025-11-04T17:21:14.645551] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-11-04T17:21:15.367048] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-03. [2025-11-04T17:21:15.693917] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T17:21:16.167248] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-05. [2025-11-04T17:21:16.846937] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-11-04T17:21:16.967103] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-04. [2025-11-04T17:21:17.076034] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-11-04T17:21:17.767059] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-06. [2025-11-04T17:21:18.071667] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T17:21:18.293283] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-11-04T17:21:18.567046] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-07. [2025-11-04T17:21:19.367026] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-03. [2025-11-04T17:21:19.866482] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:21:20.167104] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-05. [2025-11-04T17:21:20.666238] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T17:21:20.754099] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T17:21:20.967106] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-06. [2025-11-04T17:21:21.573806] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T17:21:21.767149] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-07. [2025-11-04T17:21:22.323879] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-11-04T17:21:22.567023] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-05. [2025-11-04T17:21:23.367075] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-08. [2025-11-04T17:21:23.995563] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-11-04T17:21:24.107982] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T17:21:24.167107] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-08. [2025-11-04T17:21:24.938297] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T17:21:24.967026] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-09. [2025-11-04T17:21:25.599331] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T17:21:25.767040] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-10. [2025-11-04T17:21:26.567073] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-09. [2025-11-04T17:21:26.691491] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-11-04T17:21:27.179943] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-11-04T17:21:27.367095] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-10. [2025-11-04T17:21:27.923939] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-11-04T17:21:28.167082] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-11. [2025-11-04T17:21:28.784142] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T17:21:28.785440] [INFO ] [REMOTE ] ASR: Full fetch complete (12835 records). [2025-11-04T17:21:28.785515] [DEBUG] [VALIDATOR ] ASR: Validating primary data. [2025-11-04T17:21:28.971676] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-04. [2025-11-04T17:21:29.182084] [DEBUG] [VALIDATOR ] ASR: All expected trading days are present. [2025-11-04T17:21:29.182181] [DEBUG] [VALIDATOR ] ASR: Primary data validation passed. [2025-11-04T17:21:29.182636] [INFO ] [MAIN ] ASR: Fetched primary data is valid. [2025-11-04T17:21:29.182721] [DEBUG] [LOCAL ] ASR: Saving primary data. [2025-11-04T17:21:29.183304] [DEBUG] [LOCAL ] ASR: Trimmed 12835 records to 9518 (keeping 200 days of data) [2025-11-04T17:21:29.246932] [DEBUG] [LOCAL ] ASR: Primary data saved (9518 records). [2025-11-04T17:21:29.247090] [INFO ] [MAIN ] ASR: Full history saved. [2025-11-04T17:21:29.247171] [INFO ] [MAIN ] Update Progress: 161/2367 symbols completed. ASR took 23.4s. Estimated remaining: 4h 53m 40s. Rate: 7.5 symbols/min [2025-11-04T17:21:29.421177] [INFO ] [MAIN ] ASX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:21:29.472009] [DEBUG] [LOCAL ] ASX: Loaded 52847 existing records from primary storage. [2025-11-04T17:21:29.472356] [INFO ] [MAIN ] ASX: Loaded 52847 existing records. [2025-11-04T17:21:29.650687] [WARN ] [MAIN ] ASX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:21:29.650905] [INFO ] [MAIN ] ASX: Wiping existing primary data before full refresh. [2025-11-04T17:21:29.650954] [INFO ] [LOCAL ] ASX: Starting deletion of primary data. [2025-11-04T17:21:29.651205] [DEBUG] [LOCAL ] ASX: Deleted primary data file. [2025-11-04T17:21:29.651263] [INFO ] [MAIN ] ASX: Starting full primary refresh. [2025-11-04T17:21:29.772161] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-06. [2025-11-04T17:21:30.003028] [INFO ] [REMOTE ] ASX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:21:30.162385] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T17:21:30.567040] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-01. [2025-11-04T17:21:31.152325] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-11-04T17:21:31.367056] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-11. [2025-11-04T17:21:31.994422] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T17:21:32.166999] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-02. [2025-11-04T17:21:32.274425] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T17:21:32.275677] [INFO ] [REMOTE ] ASIX: Full fetch complete (31498 records). [2025-11-04T17:21:32.275763] [DEBUG] [VALIDATOR ] ASIX: Validating primary data. [2025-11-04T17:21:32.640878] [DEBUG] [VALIDATOR ] ASIX: All expected trading days are present. [2025-11-04T17:21:32.640961] [DEBUG] [VALIDATOR ] ASIX: Primary data validation passed. [2025-11-04T17:21:32.641364] [INFO ] [MAIN ] ASIX: Fetched primary data is valid. [2025-11-04T17:21:32.641403] [DEBUG] [LOCAL ] ASIX: Saving primary data. [2025-11-04T17:21:32.642249] [DEBUG] [LOCAL ] ASIX: Trimmed 31498 records to 22909 (keeping 200 days of data) [2025-11-04T17:21:32.773981] [DEBUG] [LOCAL ] ASIX: Primary data saved (22909 records). [2025-11-04T17:21:32.774113] [INFO ] [MAIN ] ASIX: Full history saved. [2025-11-04T17:21:32.774182] [INFO ] [MAIN ] Update Progress: 162/2367 symbols completed. ASIX took 27.0s. Estimated remaining: 4h 52m 31s. Rate: 7.5 symbols/min [2025-11-04T17:21:32.940692] [INFO ] [MAIN ] ATEN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:21:32.940886] [DEBUG] [LOCAL ] ATEN: No existing data file found in primary storage. [2025-11-04T17:21:32.940922] [WARN ] [MAIN ] ATEN: No existing data found, performing full refresh instead. [2025-11-04T17:21:32.941033] [INFO ] [MAIN ] ATEN: Wiping existing primary data before full refresh. [2025-11-04T17:21:32.941074] [INFO ] [LOCAL ] ATEN: Starting deletion of primary data. [2025-11-04T17:21:32.941113] [DEBUG] [LOCAL ] ATEN: No primary data file found to delete. [2025-11-04T17:21:32.941142] [INFO ] [MAIN ] ATEN: Starting full primary refresh. [2025-11-04T17:21:32.972204] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-07. [2025-11-04T17:21:33.298423] [INFO ] [REMOTE ] ATEN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:21:33.463813] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T17:21:33.767048] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-05. [2025-11-04T17:21:34.534477] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-11-04T17:21:34.566998] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-08. [2025-11-04T17:21:34.903472] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T17:21:35.367047] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-03. [2025-11-04T17:21:36.073771] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T17:21:36.167039] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-09. [2025-11-04T17:21:36.802264] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T17:21:36.967083] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-04. [2025-11-04T17:21:37.687902] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-11-04T17:21:37.767019] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-10. [2025-11-04T17:21:38.467035] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-11-04T17:21:38.567082] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-05. [2025-11-04T17:21:39.367047] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-06. [2025-11-04T17:21:39.679835] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-11-04T17:21:40.143629] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-11-04T17:21:40.167013] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-06. [2025-11-04T17:21:40.421986] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T17:21:40.967037] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-01. [2025-11-04T17:21:41.568555] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-11-04T17:21:41.767070] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-07. [2025-11-04T17:21:42.264099] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T17:21:42.567083] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-07. [2025-11-04T17:21:42.937599] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T17:21:43.367069] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-08. [2025-11-04T17:21:44.040292] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-11-04T17:21:44.167048] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-02. [2025-11-04T17:21:44.409845] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T17:21:44.967030] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-09. [2025-11-04T17:21:45.069767] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T17:21:45.767100] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-03. [2025-11-04T17:21:45.928857] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T17:21:46.567019] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-10. [2025-11-04T17:21:46.858034] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T17:21:47.367068] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-08. [2025-11-04T17:21:47.765803] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T17:21:48.167054] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-11. [2025-11-04T17:21:48.967050] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-04. [2025-11-04T17:21:49.088352] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-11-04T17:21:49.500952] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-11 (1.33 seconds). [2025-11-04T17:21:49.502912] [INFO ] [REMOTE ] ASPN: Full fetch complete (72595 records). [2025-11-04T17:21:49.503004] [DEBUG] [VALIDATOR ] ASPN: Validating primary data. [2025-11-04T17:21:49.772152] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-09. [2025-11-04T17:21:49.901487] [DEBUG] [VALIDATOR ] ASPN: All expected trading days are present. [2025-11-04T17:21:49.901570] [DEBUG] [VALIDATOR ] ASPN: Primary data validation passed. [2025-11-04T17:21:49.902313] [INFO ] [MAIN ] ASPN: Fetched primary data is valid. [2025-11-04T17:21:49.902375] [DEBUG] [LOCAL ] ASPN: Saving primary data. [2025-11-04T17:21:49.903970] [DEBUG] [LOCAL ] ASPN: Trimmed 72595 records to 49774 (keeping 200 days of data) [2025-11-04T17:21:50.181262] [DEBUG] [LOCAL ] ASPN: Primary data saved (49774 records). [2025-11-04T17:21:50.181380] [INFO ] [MAIN ] ASPN: Full history saved. [2025-11-04T17:21:50.181457] [INFO ] [MAIN ] Update Progress: 163/2367 symbols completed. ASPN took 44.3s. Estimated remaining: 4h 54m 31s. Rate: 7.5 symbols/min [2025-11-04T17:21:50.360545] [INFO ] [MAIN ] ATGE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:21:50.386302] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T17:21:50.419833] [DEBUG] [LOCAL ] ATGE: Loaded 29912 existing records from primary storage. [2025-11-04T17:21:50.420101] [INFO ] [MAIN ] ATGE: Loaded 29912 existing records. [2025-11-04T17:21:50.578792] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-05. [2025-11-04T17:21:50.598205] [WARN ] [MAIN ] ATGE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:21:50.599127] [INFO ] [MAIN ] ATGE: Wiping existing primary data before full refresh. [2025-11-04T17:21:50.599192] [INFO ] [LOCAL ] ATGE: Starting deletion of primary data. [2025-11-04T17:21:50.599444] [DEBUG] [LOCAL ] ATGE: Deleted primary data file. [2025-11-04T17:21:50.599513] [INFO ] [MAIN ] ATGE: Starting full primary refresh. [2025-11-04T17:21:50.962886] [INFO ] [REMOTE ] ATGE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:21:51.367040] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-11. [2025-11-04T17:21:51.686471] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-11-04T17:21:51.805038] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T17:21:52.167054] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-10. [2025-11-04T17:21:52.418591] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T17:21:52.420403] [INFO ] [REMOTE ] ASH: Full fetch complete (55802 records). [2025-11-04T17:21:52.420501] [DEBUG] [VALIDATOR ] ASH: Validating primary data. [2025-11-04T17:21:52.802731] [DEBUG] [VALIDATOR ] ASH: All expected trading days are present. [2025-11-04T17:21:52.802822] [DEBUG] [VALIDATOR ] ASH: Primary data validation passed. [2025-11-04T17:21:52.803403] [INFO ] [MAIN ] ASH: Fetched primary data is valid. [2025-11-04T17:21:52.803444] [DEBUG] [LOCAL ] ASH: Saving primary data. [2025-11-04T17:21:52.804561] [DEBUG] [LOCAL ] ASH: Trimmed 55802 records to 37024 (keeping 200 days of data) [2025-11-04T17:21:52.970602] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-06. [2025-11-04T17:21:53.022475] [DEBUG] [LOCAL ] ASH: Primary data saved (37024 records). [2025-11-04T17:21:53.022596] [INFO ] [MAIN ] ASH: Full history saved. [2025-11-04T17:21:53.022694] [INFO ] [MAIN ] Update Progress: 164/2367 symbols completed. ASH took 47.2s. Estimated remaining: 4h 53m 13s. Rate: 7.5 symbols/min [2025-11-04T17:21:53.197636] [INFO ] [MAIN ] ATHM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:21:53.225630] [DEBUG] [LOCAL ] ATHM: Loaded 30148 existing records from primary storage. [2025-11-04T17:21:53.225907] [INFO ] [MAIN ] ATHM: Loaded 30148 existing records. [2025-11-04T17:21:53.399726] [WARN ] [MAIN ] ATHM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:21:53.399945] [INFO ] [MAIN ] ATHM: Wiping existing primary data before full refresh. [2025-11-04T17:21:53.399991] [INFO ] [LOCAL ] ATHM: Starting deletion of primary data. [2025-11-04T17:21:53.400181] [DEBUG] [LOCAL ] ATHM: Deleted primary data file. [2025-11-04T17:21:53.400244] [INFO ] [MAIN ] ATHM: Starting full primary refresh. [2025-11-04T17:21:53.750846] [INFO ] [REMOTE ] ATHM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:21:53.772144] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-01. [2025-11-04T17:21:53.803979] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-11-04T17:21:54.016185] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T17:21:54.567070] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-11. [2025-11-04T17:21:55.112398] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T17:21:55.367087] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-07. [2025-11-04T17:21:56.031204] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-11 (1.46 seconds). [2025-11-04T17:21:56.033351] [INFO ] [REMOTE ] ASX: Full fetch complete (76794 records). [2025-11-04T17:21:56.033468] [DEBUG] [VALIDATOR ] ASX: Validating primary data. [2025-11-04T17:21:56.172135] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-02. [2025-11-04T17:21:56.445070] [DEBUG] [VALIDATOR ] ASX: All expected trading days are present. [2025-11-04T17:21:56.445182] [DEBUG] [VALIDATOR ] ASX: Primary data validation passed. [2025-11-04T17:21:56.446165] [INFO ] [MAIN ] ASX: Fetched primary data is valid. [2025-11-04T17:21:56.446252] [DEBUG] [LOCAL ] ASX: Saving primary data. [2025-11-04T17:21:56.447950] [DEBUG] [LOCAL ] ASX: Trimmed 76794 records to 52847 (keeping 200 days of data) [2025-11-04T17:21:56.584413] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-11-04T17:21:56.785989] [DEBUG] [LOCAL ] ASX: Primary data saved (52847 records). [2025-11-04T17:21:56.786120] [INFO ] [MAIN ] ASX: Full history saved. [2025-11-04T17:21:56.786757] [INFO ] [MAIN ] Update Progress: 165/2367 symbols completed. ASX took 27.5s. Estimated remaining: 4h 52m 9s. Rate: 7.5 symbols/min [2025-11-04T17:21:56.957154] [INFO ] [MAIN ] ATHS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:21:56.966385] [DEBUG] [LOCAL ] ATHS: Loaded 6318 existing records from primary storage. [2025-11-04T17:21:56.966565] [INFO ] [MAIN ] ATHS: Loaded 6318 existing records. [2025-11-04T17:21:56.977706] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-01. [2025-11-04T17:21:57.146672] [WARN ] [MAIN ] ATHS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:21:57.146956] [INFO ] [MAIN ] ATHS: Wiping existing primary data before full refresh. [2025-11-04T17:21:57.147008] [INFO ] [LOCAL ] ATHS: Starting deletion of primary data. [2025-11-04T17:21:57.147172] [DEBUG] [LOCAL ] ATHS: Deleted primary data file. [2025-11-04T17:21:57.147227] [INFO ] [MAIN ] ATHS: Starting full primary refresh. [2025-11-04T17:21:57.503917] [INFO ] [REMOTE ] ATHS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:21:57.526175] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T17:21:57.767251] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-01. [2025-11-04T17:21:58.208606] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T17:21:58.272834] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-11-04T17:21:58.567122] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-08. [2025-11-04T17:21:59.367050] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-02. [2025-11-04T17:21:59.773891] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T17:22:00.167058] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-03. [2025-11-04T17:22:00.798821] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-11-04T17:22:00.967045] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-02. [2025-11-04T17:22:01.419801] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-11-04T17:22:01.519195] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-11-04T17:22:01.767088] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-09. [2025-11-04T17:22:02.567019] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-04. [2025-11-04T17:22:02.925929] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T17:22:03.367029] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-10. [2025-11-04T17:22:03.867941] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T17:22:04.167064] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-03. [2025-11-04T17:22:04.477493] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-11-04T17:22:04.967058] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-03. [2025-11-04T17:22:05.351539] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-11-04T17:22:05.519743] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-11-04T17:22:05.767062] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-04. [2025-11-04T17:22:06.567102] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-05. [2025-11-04T17:22:06.896448] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T17:22:07.367039] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-11. [2025-11-04T17:22:07.892564] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T17:22:08.167065] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-04. [2025-11-04T17:22:08.203682] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T17:22:08.206111] [INFO ] [REMOTE ] ATEN: Full fetch complete (56282 records). [2025-11-04T17:22:08.206365] [DEBUG] [VALIDATOR ] ATEN: Validating primary data. [2025-11-04T17:22:08.748686] [DEBUG] [VALIDATOR ] ATEN: All expected trading days are present. [2025-11-04T17:22:08.748849] [DEBUG] [VALIDATOR ] ATEN: Primary data validation passed. [2025-11-04T17:22:08.749683] [INFO ] [MAIN ] ATEN: Fetched primary data is valid. [2025-11-04T17:22:08.749771] [DEBUG] [LOCAL ] ATEN: Saving primary data. [2025-11-04T17:22:08.759235] [DEBUG] [LOCAL ] ATEN: Trimmed 56282 records to 36938 (keeping 200 days of data) [2025-11-04T17:22:08.758583] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-11-04T17:22:08.975059] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-06. [2025-11-04T17:22:09.071244] [DEBUG] [LOCAL ] ATEN: Primary data saved (36938 records). [2025-11-04T17:22:09.071427] [INFO ] [MAIN ] ATEN: Full history saved. [2025-11-04T17:22:09.071524] [INFO ] [MAIN ] Update Progress: 166/2367 symbols completed. ATEN took 36.3s. Estimated remaining: 4h 52m 59s. Rate: 7.5 symbols/min [2025-11-04T17:22:09.310363] [INFO ] [MAIN ] ATI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:22:09.310620] [DEBUG] [LOCAL ] ATI: No existing data file found in primary storage. [2025-11-04T17:22:09.310718] [WARN ] [MAIN ] ATI: No existing data found, performing full refresh instead. [2025-11-04T17:22:09.310855] [INFO ] [MAIN ] ATI: Wiping existing primary data before full refresh. [2025-11-04T17:22:09.310925] [INFO ] [LOCAL ] ATI: Starting deletion of primary data. [2025-11-04T17:22:09.311011] [DEBUG] [LOCAL ] ATI: No primary data file found to delete. [2025-11-04T17:22:09.311089] [INFO ] [MAIN ] ATI: Starting full primary refresh. [2025-11-04T17:22:09.743600] [INFO ] [REMOTE ] ATI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:22:09.767072] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-05. [2025-11-04T17:22:10.143123] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T17:22:10.264893] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-11-04T17:22:10.567056] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-01. [2025-11-04T17:22:11.367068] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-07. [2025-11-04T17:22:11.774385] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T17:22:12.174875] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-02. [2025-11-04T17:22:12.191399] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T17:22:12.967076] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-08. [2025-11-04T17:22:13.587526] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-11-04T17:22:13.767013] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-05. [2025-11-04T17:22:13.799672] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-11-04T17:22:14.567116] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-03. [2025-11-04T17:22:15.054186] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T17:22:15.367044] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-06. [2025-11-04T17:22:15.934681] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T17:22:16.167038] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-09. [2025-11-04T17:22:16.424022] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-11-04T17:22:16.965482] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-11-04T17:22:16.970464] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-04. [2025-11-04T17:22:17.767040] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-07. [2025-11-04T17:22:18.491938] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-11-04T17:22:18.567062] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-06. [2025-11-04T17:22:18.871463] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T17:22:19.191225] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-11-04T17:22:19.367066] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-10. [2025-11-04T17:22:20.167031] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-05. [2025-11-04T17:22:20.541554] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-11-04T17:22:20.967034] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-11. [2025-11-04T17:22:21.557078] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-11-04T17:22:21.767034] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-07. [2025-11-04T17:22:21.881169] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T17:22:21.882760] [INFO ] [REMOTE ] ATHM: Full fetch complete (48673 records). [2025-11-04T17:22:21.882855] [DEBUG] [VALIDATOR ] ATHM: Validating primary data. [2025-11-04T17:22:22.288310] [DEBUG] [VALIDATOR ] ATHM: All expected trading days are present. [2025-11-04T17:22:22.288530] [DEBUG] [VALIDATOR ] ATHM: Primary data validation passed. [2025-11-04T17:22:22.290002] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-11-04T17:22:22.289761] [INFO ] [MAIN ] ATHM: Fetched primary data is valid. [2025-11-04T17:22:22.292352] [DEBUG] [LOCAL ] ATHM: Saving primary data. [2025-11-04T17:22:22.293585] [DEBUG] [LOCAL ] ATHM: Trimmed 48673 records to 30062 (keeping 200 days of data) [2025-11-04T17:22:22.470268] [DEBUG] [LOCAL ] ATHM: Primary data saved (30062 records). [2025-11-04T17:22:22.470392] [INFO ] [MAIN ] ATHM: Full history saved. [2025-11-04T17:22:22.470466] [INFO ] [MAIN ] Update Progress: 167/2367 symbols completed. ATHM took 29.4s. Estimated remaining: 4h 54m 2s. Rate: 7.5 symbols/min [2025-11-04T17:22:22.569669] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-08. [2025-11-04T17:22:22.643666] [INFO ] [MAIN ] ATKR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:22:22.675134] [DEBUG] [LOCAL ] ATKR: Loaded 34077 existing records from primary storage. [2025-11-04T17:22:22.675388] [INFO ] [MAIN ] ATKR: Loaded 34077 existing records. [2025-11-04T17:22:22.856675] [WARN ] [MAIN ] ATKR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:22:22.856927] [INFO ] [MAIN ] ATKR: Wiping existing primary data before full refresh. [2025-11-04T17:22:22.856982] [INFO ] [LOCAL ] ATKR: Starting deletion of primary data. [2025-11-04T17:22:22.857160] [DEBUG] [LOCAL ] ATKR: Deleted primary data file. [2025-11-04T17:22:22.857198] [INFO ] [MAIN ] ATKR: Starting full primary refresh. [2025-11-04T17:22:23.160812] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-11-04T17:22:23.231703] [INFO ] [REMOTE ] ATKR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:22:23.367028] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-01. [2025-11-04T17:22:24.167023] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-08. [2025-11-04T17:22:24.571853] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T17:22:24.967132] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-06. [2025-11-04T17:22:25.280225] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T17:22:25.767061] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-09. [2025-11-04T17:22:26.253348] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-11-04T17:22:26.567047] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-02. [2025-11-04T17:22:26.735144] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-11-04T17:22:27.367071] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-07. [2025-11-04T17:22:27.820009] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-11-04T17:22:28.167100] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-03. [2025-11-04T17:22:28.807298] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:22:28.967043] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-10. [2025-11-04T17:22:29.105902] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-11-04T17:22:29.767071] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-04. [2025-11-04T17:22:30.007441] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-11-04T17:22:30.567057] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-08. [2025-11-04T17:22:30.960011] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T17:22:31.367075] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-11. [2025-11-04T17:22:31.953085] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T17:22:32.074030] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T17:22:32.075427] [INFO ] [REMOTE ] ATGE: Full fetch complete (45421 records). [2025-11-04T17:22:32.075520] [DEBUG] [VALIDATOR ] ATGE: Validating primary data. [2025-11-04T17:22:32.076145] [FATAL] [VALIDATOR ] ATGE: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:22:32.076246] [FATAL] [MAIN ] ATGE: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:22:32.076315] [WARN ] [MAIN ] ATGE: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:22:32.076411] [INFO ] [LOCAL ] ATGE: Starting deletion of primary data. [2025-11-04T17:22:32.076492] [DEBUG] [LOCAL ] ATGE: No primary data file found to delete. [2025-11-04T17:22:32.076530] [INFO ] [LOCAL ] ATGE: Starting deletion of derivative data. [2025-11-04T17:22:32.076707] [DEBUG] [LOCAL ] ATGE: Deleted derivative data file. [2025-11-04T17:22:32.077596] [INFO ] [MAIN ] Update Progress: 168/2367 symbols completed. ATGE took 41.9s. Estimated remaining: 4h 54m 15s. Rate: 7.5 symbols/min [2025-11-04T17:22:32.172158] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-09. [2025-11-04T17:22:32.253591] [INFO ] [MAIN ] ATMU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:22:32.290895] [DEBUG] [LOCAL ] ATMU: Loaded 38044 existing records from primary storage. [2025-11-04T17:22:32.291215] [INFO ] [MAIN ] ATMU: Loaded 38044 existing records. [2025-11-04T17:22:32.473183] [WARN ] [MAIN ] ATMU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:22:32.473426] [INFO ] [MAIN ] ATMU: Wiping existing primary data before full refresh. [2025-11-04T17:22:32.473502] [INFO ] [LOCAL ] ATMU: Starting deletion of primary data. [2025-11-04T17:22:32.473755] [DEBUG] [LOCAL ] ATMU: Deleted primary data file. [2025-11-04T17:22:32.473811] [INFO ] [MAIN ] ATMU: Starting full primary refresh. [2025-11-04T17:22:32.835810] [INFO ] [REMOTE ] ATMU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:22:32.967031] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-01. [2025-11-04T17:22:33.721585] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-11-04T17:22:33.767075] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-10. [2025-11-04T17:22:34.205145] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-11-04T17:22:34.567041] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-02. [2025-11-04T17:22:35.164332] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T17:22:35.367066] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-09. [2025-11-04T17:22:35.707776] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T17:22:35.798854] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-11-04T17:22:36.167053] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-10. [2025-11-04T17:22:36.624750] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-11-04T17:22:36.967052] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-11. [2025-11-04T17:22:37.490334] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-11 (0.52 seconds). [2025-11-04T17:22:37.491268] [INFO ] [REMOTE ] ATHS: Full fetch complete (9595 records). [2025-11-04T17:22:37.491339] [DEBUG] [VALIDATOR ] ATHS: Validating primary data. [2025-11-04T17:22:37.772306] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-05. [2025-11-04T17:22:37.869241] [DEBUG] [VALIDATOR ] ATHS: All expected trading days are present. [2025-11-04T17:22:37.869317] [DEBUG] [VALIDATOR ] ATHS: Primary data validation passed. [2025-11-04T17:22:37.869585] [INFO ] [MAIN ] ATHS: Fetched primary data is valid. [2025-11-04T17:22:37.869623] [DEBUG] [LOCAL ] ATHS: Saving primary data. [2025-11-04T17:22:37.870268] [DEBUG] [LOCAL ] ATHS: Trimmed 9595 records to 6365 (keeping 200 days of data) [2025-11-04T17:22:37.909718] [DEBUG] [LOCAL ] ATHS: Primary data saved (6365 records). [2025-11-04T17:22:37.909864] [INFO ] [MAIN ] ATHS: Full history saved. [2025-11-04T17:22:37.909944] [INFO ] [MAIN ] Update Progress: 169/2367 symbols completed. ATHS took 41.1s. Estimated remaining: 4h 53m 38s. Rate: 7.5 symbols/min [2025-11-04T17:22:38.082136] [INFO ] [MAIN ] ATO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:22:38.120839] [DEBUG] [LOCAL ] ATO: Loaded 41155 existing records from primary storage. [2025-11-04T17:22:38.121170] [INFO ] [MAIN ] ATO: Loaded 41155 existing records. [2025-11-04T17:22:38.303688] [WARN ] [MAIN ] ATO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:22:38.303939] [INFO ] [MAIN ] ATO: Wiping existing primary data before full refresh. [2025-11-04T17:22:38.304182] [INFO ] [LOCAL ] ATO: Starting deletion of primary data. [2025-11-04T17:22:38.304553] [DEBUG] [LOCAL ] ATO: Deleted primary data file. [2025-11-04T17:22:38.304590] [INFO ] [MAIN ] ATO: Starting full primary refresh. [2025-11-04T17:22:38.572200] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-11. [2025-11-04T17:22:38.670089] [INFO ] [REMOTE ] ATO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:22:38.907179] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T17:22:39.367033] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-06. [2025-11-04T17:22:39.854232] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-11 (1.28 seconds). [2025-11-04T17:22:39.856225] [INFO ] [REMOTE ] ATI: Full fetch complete (74374 records). [2025-11-04T17:22:39.856347] [DEBUG] [VALIDATOR ] ATI: Validating primary data. [2025-11-04T17:22:40.172156] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-03. [2025-11-04T17:22:40.267222] [DEBUG] [VALIDATOR ] ATI: All expected trading days are present. [2025-11-04T17:22:40.267385] [DEBUG] [VALIDATOR ] ATI: Primary data validation passed. [2025-11-04T17:22:40.268236] [INFO ] [MAIN ] ATI: Fetched primary data is valid. [2025-11-04T17:22:40.268391] [DEBUG] [LOCAL ] ATI: Saving primary data. [2025-11-04T17:22:40.270113] [DEBUG] [LOCAL ] ATI: Trimmed 74374 records to 51279 (keeping 200 days of data) [2025-11-04T17:22:40.381715] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-11-04T17:22:40.595320] [DEBUG] [LOCAL ] ATI: Primary data saved (51279 records). [2025-11-04T17:22:40.595528] [INFO ] [MAIN ] ATI: Full history saved. [2025-11-04T17:22:40.595619] [INFO ] [MAIN ] Update Progress: 170/2367 symbols completed. ATI took 31.5s. Estimated remaining: 4h 52m 21s. Rate: 7.5 symbols/min [2025-11-04T17:22:40.765007] [INFO ] [MAIN ] ATR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:22:40.765167] [DEBUG] [LOCAL ] ATR: No existing data file found in primary storage. [2025-11-04T17:22:40.765204] [WARN ] [MAIN ] ATR: No existing data found, performing full refresh instead. [2025-11-04T17:22:40.765311] [INFO ] [MAIN ] ATR: Wiping existing primary data before full refresh. [2025-11-04T17:22:40.765352] [INFO ] [LOCAL ] ATR: Starting deletion of primary data. [2025-11-04T17:22:40.765394] [DEBUG] [LOCAL ] ATR: No primary data file found to delete. [2025-11-04T17:22:40.765466] [INFO ] [MAIN ] ATR: Starting full primary refresh. [2025-11-04T17:22:40.974321] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-07. [2025-11-04T17:22:41.125071] [INFO ] [REMOTE ] ATR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:22:41.577725] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T17:22:41.767073] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-01. [2025-11-04T17:22:42.205349] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T17:22:42.567271] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-01. [2025-11-04T17:22:42.785753] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T17:22:43.367042] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-04. [2025-11-04T17:22:43.769247] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T17:22:44.167015] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-08. [2025-11-04T17:22:44.713720] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T17:22:44.967279] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-02. [2025-11-04T17:22:45.323189] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T17:22:45.767050] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-05. [2025-11-04T17:22:46.085021] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-11-04T17:22:46.567060] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-02. [2025-11-04T17:22:46.823078] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T17:22:47.367084] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-03. [2025-11-04T17:22:47.784201] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T17:22:48.167069] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-03. [2025-11-04T17:22:48.376179] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T17:22:48.967045] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-06. [2025-11-04T17:22:49.491670] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T17:22:49.767046] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-04. [2025-11-04T17:22:49.960895] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-11-04T17:22:50.567083] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-04. [2025-11-04T17:22:51.071803] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T17:22:51.367040] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-05. [2025-11-04T17:22:51.702146] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T17:22:52.167046] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-09. [2025-11-04T17:22:52.852021] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T17:22:52.967007] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-06. [2025-11-04T17:22:53.476858] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T17:22:53.767054] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-10. [2025-11-04T17:22:54.089381] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T17:22:54.567050] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-07. [2025-11-04T17:22:54.982891] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T17:22:55.367058] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-07. [2025-11-04T17:22:55.626145] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-11-04T17:22:56.167126] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-05. [2025-11-04T17:22:56.465337] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T17:22:56.967044] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-08. [2025-11-04T17:22:57.104940] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T17:22:57.767092] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-06. [2025-11-04T17:22:58.247386] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-11-04T17:22:58.567057] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-08. [2025-11-04T17:22:58.720115] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-11-04T17:22:59.367061] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-11. [2025-11-04T17:22:59.625274] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-11-04T17:23:00.167044] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-07. [2025-11-04T17:23:00.967071] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-09. [2025-11-04T17:23:01.767109] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-09. [2025-11-04T17:23:05.316243] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-07 (5.15 seconds). [2025-11-04T17:23:05.326607] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-08. [2025-11-04T17:23:05.869673] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-11 (6.50 seconds). [2025-11-04T17:23:05.870840] [INFO ] [REMOTE ] ATKR: Full fetch complete (52492 records). [2025-11-04T17:23:05.870929] [DEBUG] [VALIDATOR ] ATKR: Validating primary data. [2025-11-04T17:23:05.871469] [FATAL] [VALIDATOR ] ATKR: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:23:05.871552] [FATAL] [MAIN ] ATKR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:23:05.871594] [WARN ] [MAIN ] ATKR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:23:05.871684] [INFO ] [LOCAL ] ATKR: Starting deletion of primary data. [2025-11-04T17:23:05.871758] [DEBUG] [LOCAL ] ATKR: No primary data file found to delete. [2025-11-04T17:23:05.871792] [INFO ] [LOCAL ] ATKR: Starting deletion of derivative data. [2025-11-04T17:23:05.871944] [DEBUG] [LOCAL ] ATKR: Deleted derivative data file. [2025-11-04T17:23:05.872034] [INFO ] [MAIN ] Update Progress: 171/2367 symbols completed. ATKR took 43.4s. Estimated remaining: 4h 55m 55s. Rate: 7.4 symbols/min [2025-11-04T17:23:06.060479] [INFO ] [MAIN ] ATS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:23:06.060700] [DEBUG] [LOCAL ] ATS: No existing data file found in primary storage. [2025-11-04T17:23:06.060750] [WARN ] [MAIN ] ATS: No existing data found, performing full refresh instead. [2025-11-04T17:23:06.060858] [INFO ] [MAIN ] ATS: Wiping existing primary data before full refresh. [2025-11-04T17:23:06.060898] [INFO ] [LOCAL ] ATS: Starting deletion of primary data. [2025-11-04T17:23:06.060938] [DEBUG] [LOCAL ] ATS: No primary data file found to delete. [2025-11-04T17:23:06.060969] [INFO ] [MAIN ] ATS: Starting full primary refresh. [2025-11-04T17:23:06.428174] [INFO ] [REMOTE ] ATS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:23:06.428532] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-01. [2025-11-04T17:23:06.539508] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-09 (5.57 seconds). [2025-11-04T17:23:06.549305] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-10. [2025-11-04T17:23:06.571321] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T17:23:06.578828] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-09. [2025-11-04T17:23:07.075526] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-09 (5.31 seconds). [2025-11-04T17:23:07.085764] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-10. [2025-11-04T17:23:07.138931] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T17:23:07.143288] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-02. [2025-11-04T17:23:07.759173] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-11-04T17:23:07.768552] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-11. [2025-11-04T17:23:07.854978] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T17:23:07.936175] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T17:23:08.167055] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-10. [2025-11-04T17:23:08.364907] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-11-04T17:23:08.756409] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T17:23:08.758165] [INFO ] [REMOTE ] ATMU: Full fetch complete (56629 records). [2025-11-04T17:23:08.758283] [DEBUG] [VALIDATOR ] ATMU: Validating primary data. [2025-11-04T17:23:08.978212] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-03. [2025-11-04T17:23:09.190108] [DEBUG] [VALIDATOR ] ATMU: All expected trading days are present. [2025-11-04T17:23:09.190246] [DEBUG] [VALIDATOR ] ATMU: Primary data validation passed. [2025-11-04T17:23:09.191100] [INFO ] [MAIN ] ATMU: Fetched primary data is valid. [2025-11-04T17:23:09.191225] [DEBUG] [LOCAL ] ATMU: Saving primary data. [2025-11-04T17:23:09.192733] [DEBUG] [LOCAL ] ATMU: Trimmed 56629 records to 38070 (keeping 200 days of data) [2025-11-04T17:23:09.322141] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T17:23:09.467371] [DEBUG] [LOCAL ] ATMU: Primary data saved (38070 records). [2025-11-04T17:23:09.467524] [INFO ] [MAIN ] ATMU: Full history saved. [2025-11-04T17:23:09.468521] [INFO ] [MAIN ] Update Progress: 172/2367 symbols completed. ATMU took 37.4s. Estimated remaining: 4h 54m 50s. Rate: 7.4 symbols/min [2025-11-04T17:23:09.655177] [INFO ] [MAIN ] ATUS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:23:09.655349] [DEBUG] [LOCAL ] ATUS: No existing data file found in primary storage. [2025-11-04T17:23:09.655393] [WARN ] [MAIN ] ATUS: No existing data found, performing full refresh instead. [2025-11-04T17:23:09.655500] [INFO ] [MAIN ] ATUS: Wiping existing primary data before full refresh. [2025-11-04T17:23:09.655541] [INFO ] [LOCAL ] ATUS: Starting deletion of primary data. [2025-11-04T17:23:09.655581] [DEBUG] [LOCAL ] ATUS: No primary data file found to delete. [2025-11-04T17:23:09.655613] [INFO ] [MAIN ] ATUS: Starting full primary refresh. [2025-11-04T17:23:09.768605] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-11. [2025-11-04T17:23:10.014809] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T17:23:10.113678] [INFO ] [REMOTE ] ATUS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:23:10.567053] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-04. [2025-11-04T17:23:10.650488] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-11 (0.88 seconds). [2025-11-04T17:23:10.651840] [INFO ] [REMOTE ] ATR: Full fetch complete (44412 records). [2025-11-04T17:23:10.651924] [DEBUG] [VALIDATOR ] ATR: Validating primary data. [2025-11-04T17:23:10.652470] [FATAL] [VALIDATOR ] ATR: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:23:10.652562] [FATAL] [MAIN ] ATR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:23:10.652636] [WARN ] [MAIN ] ATR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:23:10.652751] [INFO ] [LOCAL ] ATR: Starting deletion of primary data. [2025-11-04T17:23:10.652832] [DEBUG] [LOCAL ] ATR: No primary data file found to delete. [2025-11-04T17:23:10.652868] [INFO ] [LOCAL ] ATR: Starting deletion of derivative data. [2025-11-04T17:23:10.652906] [DEBUG] [LOCAL ] ATR: No derivative data file found to delete. [2025-11-04T17:23:10.652958] [INFO ] [MAIN ] Update Progress: 173/2367 symbols completed. ATR took 30.1s. Estimated remaining: 4h 53m 15s. Rate: 7.5 symbols/min [2025-11-04T17:23:10.826462] [INFO ] [MAIN ] AU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:23:10.826639] [DEBUG] [LOCAL ] AU: No existing data file found in primary storage. [2025-11-04T17:23:10.826716] [WARN ] [MAIN ] AU: No existing data found, performing full refresh instead. [2025-11-04T17:23:10.826867] [INFO ] [MAIN ] AU: Wiping existing primary data before full refresh. [2025-11-04T17:23:10.826924] [INFO ] [LOCAL ] AU: Starting deletion of primary data. [2025-11-04T17:23:10.826972] [DEBUG] [LOCAL ] AU: No primary data file found to delete. [2025-11-04T17:23:10.827004] [INFO ] [MAIN ] AU: Starting full primary refresh. [2025-11-04T17:23:11.197857] [INFO ] [REMOTE ] AU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:23:11.367018] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-01. [2025-11-04T17:23:11.405588] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-11-04T17:23:12.167271] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-05. [2025-11-04T17:23:12.870340] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T17:23:12.969677] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-01. [2025-11-04T17:23:12.973717] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T17:23:13.767078] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-11. [2025-11-04T17:23:14.284557] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T17:23:14.567060] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-02. [2025-11-04T17:23:14.962523] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T17:23:14.964287] [INFO ] [REMOTE ] ATO: Full fetch complete (60520 records). [2025-11-04T17:23:14.964372] [DEBUG] [VALIDATOR ] ATO: Validating primary data. [2025-11-04T17:23:15.373697] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-06. [2025-11-04T17:23:15.375154] [DEBUG] [VALIDATOR ] ATO: All expected trading days are present. [2025-11-04T17:23:15.375211] [DEBUG] [VALIDATOR ] ATO: Primary data validation passed. [2025-11-04T17:23:15.375858] [INFO ] [MAIN ] ATO: Fetched primary data is valid. [2025-11-04T17:23:15.376006] [DEBUG] [LOCAL ] ATO: Saving primary data. [2025-11-04T17:23:15.377250] [DEBUG] [LOCAL ] ATO: Trimmed 60520 records to 41104 (keeping 200 days of data) [2025-11-04T17:23:15.635336] [DEBUG] [LOCAL ] ATO: Primary data saved (41104 records). [2025-11-04T17:23:15.635456] [INFO ] [MAIN ] ATO: Full history saved. [2025-11-04T17:23:15.636181] [INFO ] [MAIN ] Update Progress: 174/2367 symbols completed. ATO took 37.7s. Estimated remaining: 4h 52m 28s. Rate: 7.5 symbols/min [2025-11-04T17:23:15.819959] [INFO ] [MAIN ] AUB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:23:15.820149] [DEBUG] [LOCAL ] AUB: No existing data file found in primary storage. [2025-11-04T17:23:15.820186] [WARN ] [MAIN ] AUB: No existing data found, performing full refresh instead. [2025-11-04T17:23:15.820299] [INFO ] [MAIN ] AUB: Wiping existing primary data before full refresh. [2025-11-04T17:23:15.820357] [INFO ] [LOCAL ] AUB: Starting deletion of primary data. [2025-11-04T17:23:15.820433] [DEBUG] [LOCAL ] AUB: No primary data file found to delete. [2025-11-04T17:23:15.820520] [INFO ] [MAIN ] AUB: Starting full primary refresh. [2025-11-04T17:23:16.171965] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-11-04T17:23:16.172123] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-02. [2025-11-04T17:23:16.225468] [INFO ] [REMOTE ] AUB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:23:16.255947] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-11-04T17:23:16.967106] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-01. [2025-11-04T17:23:17.325160] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T17:23:17.767044] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-03. [2025-11-04T17:23:18.216212] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T17:23:18.567016] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-03. [2025-11-04T17:23:19.323558] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T17:23:19.367001] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-04. [2025-11-04T17:23:19.832236] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T17:23:20.167040] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-02. [2025-11-04T17:23:20.967020] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-04. [2025-11-04T17:23:21.395046] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-11-04T17:23:21.598343] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-11-04T17:23:21.767030] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-05. [2025-11-04T17:23:22.064620] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T17:23:22.567126] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-07. [2025-11-04T17:23:23.367084] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-05. [2025-11-04T17:23:23.415904] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T17:23:23.502245] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-11-04T17:23:24.167062] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-06. [2025-11-04T17:23:24.820761] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T17:23:24.967057] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-06. [2025-11-04T17:23:25.672341] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-11-04T17:23:25.767036] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-07. [2025-11-04T17:23:26.074279] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T17:23:26.567096] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-07. [2025-11-04T17:23:27.271588] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-11-04T17:23:27.367049] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-08. [2025-11-04T17:23:27.954011] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-11-04T17:23:28.167034] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-08. [2025-11-04T17:23:28.967027] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-03. [2025-11-04T17:23:29.216544] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-11-04T17:23:29.594548] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-11-04T17:23:29.767087] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-09. [2025-11-04T17:23:30.387206] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T17:23:30.567080] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-04. [2025-11-04T17:23:31.372289] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-09. [2025-11-04T17:23:31.386382] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-11-04T17:23:31.905502] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T17:23:32.167055] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-08. [2025-11-04T17:23:32.898977] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-11-04T17:23:32.967012] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-10. [2025-11-04T17:23:33.102165] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-11-04T17:23:33.767046] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-09. [2025-11-04T17:23:34.576972] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-05. [2025-11-04T17:23:34.599098] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-11-04T17:23:34.638161] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-11-04T17:23:35.367081] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-11. [2025-11-04T17:23:35.838092] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T17:23:36.167057] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-06. [2025-11-04T17:23:36.424355] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T17:23:36.426398] [INFO ] [REMOTE ] ATUS: Full fetch complete (71474 records). [2025-11-04T17:23:36.426560] [DEBUG] [VALIDATOR ] ATUS: Validating primary data. [2025-11-04T17:23:36.848546] [DEBUG] [VALIDATOR ] ATUS: All expected trading days are present. [2025-11-04T17:23:36.848638] [DEBUG] [VALIDATOR ] ATUS: Primary data validation passed. [2025-11-04T17:23:36.849463] [INFO ] [MAIN ] ATUS: Fetched primary data is valid. [2025-11-04T17:23:36.849524] [DEBUG] [LOCAL ] ATUS: Saving primary data. [2025-11-04T17:23:36.850994] [DEBUG] [LOCAL ] ATUS: Trimmed 71474 records to 49962 (keeping 200 days of data) [2025-11-04T17:23:36.969084] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-10. [2025-11-04T17:23:37.137291] [DEBUG] [LOCAL ] ATUS: Primary data saved (49962 records). [2025-11-04T17:23:37.137434] [INFO ] [MAIN ] ATUS: Full history saved. [2025-11-04T17:23:37.137533] [INFO ] [MAIN ] Update Progress: 175/2367 symbols completed. ATUS took 27.7s. Estimated remaining: 4h 55m 9s. Rate: 7.4 symbols/min [2025-11-04T17:23:37.322561] [INFO ] [MAIN ] AUNA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:23:37.330249] [DEBUG] [LOCAL ] AUNA: Loaded 5516 existing records from primary storage. [2025-11-04T17:23:37.330454] [INFO ] [MAIN ] AUNA: Loaded 5516 existing records. [2025-11-04T17:23:37.521467] [WARN ] [MAIN ] AUNA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:23:37.521928] [INFO ] [MAIN ] AUNA: Wiping existing primary data before full refresh. [2025-11-04T17:23:37.522038] [INFO ] [LOCAL ] AUNA: Starting deletion of primary data. [2025-11-04T17:23:37.522322] [DEBUG] [LOCAL ] AUNA: Deleted primary data file. [2025-11-04T17:23:37.522385] [INFO ] [MAIN ] AUNA: Starting full primary refresh. [2025-11-04T17:23:37.616980] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T17:23:37.723567] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-11-04T17:23:37.773052] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-07. [2025-11-04T17:23:37.978422] [INFO ] [REMOTE ] AUNA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:23:38.567035] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-01. [2025-11-04T17:23:39.072926] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-11-04T17:23:39.112292] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T17:23:39.367044] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-02. [2025-11-04T17:23:39.743546] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T17:23:40.167063] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-08. [2025-11-04T17:23:40.967055] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-03. [2025-11-04T17:23:41.350073] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-11-04T17:23:41.630861] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-11-04T17:23:41.767058] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-09. [2025-11-04T17:23:42.567103] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-10. [2025-11-04T17:23:43.048841] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T17:23:43.367065] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-11. [2025-11-04T17:23:43.955323] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-11 (0.59 seconds). [2025-11-04T17:23:43.956341] [INFO ] [REMOTE ] ATS: Full fetch complete (24999 records). [2025-11-04T17:23:43.956417] [DEBUG] [VALIDATOR ] ATS: Validating primary data. [2025-11-04T17:23:43.956948] [FATAL] [VALIDATOR ] ATS: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:23:43.957038] [FATAL] [MAIN ] ATS: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:23:43.957081] [WARN ] [MAIN ] ATS: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:23:43.957119] [INFO ] [LOCAL ] ATS: Starting deletion of primary data. [2025-11-04T17:23:43.957191] [DEBUG] [LOCAL ] ATS: No primary data file found to delete. [2025-11-04T17:23:43.957247] [INFO ] [LOCAL ] ATS: Starting deletion of derivative data. [2025-11-04T17:23:43.957310] [DEBUG] [LOCAL ] ATS: No derivative data file found to delete. [2025-11-04T17:23:43.957365] [INFO ] [MAIN ] Update Progress: 176/2367 symbols completed. ATS took 38.1s. Estimated remaining: 4h 54m 46s. Rate: 7.4 symbols/min [2025-11-04T17:23:44.129832] [INFO ] [MAIN ] AVA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:23:44.167490] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-04. [2025-11-04T17:23:44.167385] [DEBUG] [LOCAL ] AVA: Loaded 37952 existing records from primary storage. [2025-11-04T17:23:44.168855] [INFO ] [MAIN ] AVA: Loaded 37952 existing records. [2025-11-04T17:23:44.364639] [WARN ] [MAIN ] AVA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:23:44.364976] [INFO ] [MAIN ] AVA: Wiping existing primary data before full refresh. [2025-11-04T17:23:44.365027] [INFO ] [LOCAL ] AVA: Starting deletion of primary data. [2025-11-04T17:23:44.365351] [DEBUG] [LOCAL ] AVA: Deleted primary data file. [2025-11-04T17:23:44.365408] [INFO ] [MAIN ] AVA: Starting full primary refresh. [2025-11-04T17:23:44.662323] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-11-04T17:23:44.764797] [INFO ] [REMOTE ] AVA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:23:44.807006] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-10 (2.24 seconds). [2025-11-04T17:23:44.967039] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-11. [2025-11-04T17:23:45.767017] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-05. [2025-11-04T17:23:46.132689] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-11-04T17:23:46.494357] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-11 (1.53 seconds). [2025-11-04T17:23:46.495851] [INFO ] [REMOTE ] AU: Full fetch complete (76280 records). [2025-11-04T17:23:46.495939] [DEBUG] [VALIDATOR ] AU: Validating primary data. [2025-11-04T17:23:46.496619] [FATAL] [VALIDATOR ] AU: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:23:46.496725] [FATAL] [MAIN ] AU: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:23:46.496767] [WARN ] [MAIN ] AU: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:23:46.496804] [INFO ] [LOCAL ] AU: Starting deletion of primary data. [2025-11-04T17:23:46.496885] [DEBUG] [LOCAL ] AU: No primary data file found to delete. [2025-11-04T17:23:46.496938] [INFO ] [LOCAL ] AU: Starting deletion of derivative data. [2025-11-04T17:23:46.496980] [DEBUG] [LOCAL ] AU: No derivative data file found to delete. [2025-11-04T17:23:46.497034] [INFO ] [MAIN ] Update Progress: 177/2367 symbols completed. AU took 35.8s. Estimated remaining: 4h 53m 29s. Rate: 7.5 symbols/min [2025-11-04T17:23:46.572328] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-01. [2025-11-04T17:23:46.672135] [INFO ] [MAIN ] AVAL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:23:46.672313] [DEBUG] [LOCAL ] AVAL: No existing data file found in primary storage. [2025-11-04T17:23:46.672350] [WARN ] [MAIN ] AVAL: No existing data found, performing full refresh instead. [2025-11-04T17:23:46.672460] [INFO ] [MAIN ] AVAL: Wiping existing primary data before full refresh. [2025-11-04T17:23:46.672501] [INFO ] [LOCAL ] AVAL: Starting deletion of primary data. [2025-11-04T17:23:46.672541] [DEBUG] [LOCAL ] AVAL: No primary data file found to delete. [2025-11-04T17:23:46.672572] [INFO ] [MAIN ] AVAL: Starting full primary refresh. [2025-11-04T17:23:47.062259] [INFO ] [REMOTE ] AVAL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:23:47.367050] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-06. [2025-11-04T17:23:47.654067] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T17:23:47.959341] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-11-04T17:23:48.167108] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-02. [2025-11-04T17:23:48.967016] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-01. [2025-11-04T17:23:49.077845] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T17:23:49.660003] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T17:23:49.767055] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-10. [2025-11-04T17:23:50.567070] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-02. [2025-11-04T17:23:51.180546] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-11-04T17:23:51.350326] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-11-04T17:23:51.366992] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-11. [2025-11-04T17:23:52.167076] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-07. [2025-11-04T17:23:52.582849] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T17:23:52.585005] [INFO ] [REMOTE ] AUB: Full fetch complete (66622 records). [2025-11-04T17:23:52.585127] [DEBUG] [VALIDATOR ] AUB: Validating primary data. [2025-11-04T17:23:52.704107] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-11-04T17:23:52.970031] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-03. [2025-11-04T17:23:52.989804] [DEBUG] [VALIDATOR ] AUB: All expected trading days are present. [2025-11-04T17:23:52.989882] [DEBUG] [VALIDATOR ] AUB: Primary data validation passed. [2025-11-04T17:23:52.990556] [INFO ] [MAIN ] AUB: Fetched primary data is valid. [2025-11-04T17:23:52.990745] [DEBUG] [LOCAL ] AUB: Saving primary data. [2025-11-04T17:23:52.991977] [DEBUG] [LOCAL ] AUB: Trimmed 66622 records to 45477 (keeping 200 days of data) [2025-11-04T17:23:53.264092] [DEBUG] [LOCAL ] AUB: Primary data saved (45477 records). [2025-11-04T17:23:53.264218] [INFO ] [MAIN ] AUB: Full history saved. [2025-11-04T17:23:53.264285] [INFO ] [MAIN ] Update Progress: 178/2367 symbols completed. AUB took 37.6s. Estimated remaining: 4h 53m 5s. Rate: 7.5 symbols/min [2025-11-04T17:23:53.434887] [INFO ] [MAIN ] AVB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:23:53.474005] [DEBUG] [LOCAL ] AVB: Loaded 41960 existing records from primary storage. [2025-11-04T17:23:53.474279] [INFO ] [MAIN ] AVB: Loaded 41960 existing records. [2025-11-04T17:23:53.655959] [WARN ] [MAIN ] AVB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:23:53.656245] [INFO ] [MAIN ] AVB: Wiping existing primary data before full refresh. [2025-11-04T17:23:53.656323] [INFO ] [LOCAL ] AVB: Starting deletion of primary data. [2025-11-04T17:23:53.656631] [DEBUG] [LOCAL ] AVB: Deleted primary data file. [2025-11-04T17:23:53.656725] [INFO ] [MAIN ] AVB: Starting full primary refresh. [2025-11-04T17:23:53.771899] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-03. [2025-11-04T17:23:53.792865] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-11-04T17:23:54.041052] [INFO ] [REMOTE ] AVB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:23:54.567042] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-08. [2025-11-04T17:23:54.968143] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-11-04T17:23:54.969734] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T17:23:55.367089] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-01. [2025-11-04T17:23:56.167035] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-04. [2025-11-04T17:23:56.598372] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T17:23:56.738790] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-11-04T17:23:56.967075] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-05. [2025-11-04T17:23:57.454960] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-11-04T17:23:57.767055] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-02. [2025-11-04T17:23:58.567094] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-04. [2025-11-04T17:23:58.862742] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T17:23:59.367085] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-06. [2025-11-04T17:23:59.805423] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T17:23:59.863561] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-11-04T17:24:00.167104] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-03. [2025-11-04T17:24:00.967050] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-07. [2025-11-04T17:24:01.361554] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T17:24:01.593311] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-11-04T17:24:01.767052] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-08. [2025-11-04T17:24:02.362579] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-11-04T17:24:02.567122] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-09. [2025-11-04T17:24:03.137313] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-11-04T17:24:03.367039] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-09. [2025-11-04T17:24:03.963811] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-11-04T17:24:04.167101] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-10. [2025-11-04T17:24:04.657871] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-11-04T17:24:04.967173] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-05. [2025-11-04T17:24:05.767086] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-11. [2025-11-04T17:24:06.172951] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T17:24:06.339443] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T17:24:06.340320] [INFO ] [REMOTE ] AUNA: Full fetch complete (7844 records). [2025-11-04T17:24:06.340392] [DEBUG] [VALIDATOR ] AUNA: Validating primary data. [2025-11-04T17:24:06.572316] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-10. [2025-11-04T17:24:06.737700] [DEBUG] [VALIDATOR ] AUNA: All expected trading days are present. [2025-11-04T17:24:06.737789] [DEBUG] [VALIDATOR ] AUNA: Primary data validation passed. [2025-11-04T17:24:06.737993] [INFO ] [MAIN ] AUNA: Fetched primary data is valid. [2025-11-04T17:24:06.738032] [DEBUG] [LOCAL ] AUNA: Saving primary data. [2025-11-04T17:24:06.738536] [DEBUG] [LOCAL ] AUNA: Trimmed 7844 records to 5587 (keeping 200 days of data) [2025-11-04T17:24:06.774268] [DEBUG] [LOCAL ] AUNA: Primary data saved (5587 records). [2025-11-04T17:24:06.774419] [INFO ] [MAIN ] AUNA: Full history saved. [2025-11-04T17:24:06.774506] [INFO ] [MAIN ] Update Progress: 179/2367 symbols completed. AUNA took 29.6s. Estimated remaining: 4h 54m 4s. Rate: 7.4 symbols/min [2025-11-04T17:24:06.953797] [INFO ] [MAIN ] AVBC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:24:06.954003] [DEBUG] [LOCAL ] AVBC: No existing data file found in primary storage. [2025-11-04T17:24:06.954061] [WARN ] [MAIN ] AVBC: No existing data found, performing full refresh instead. [2025-11-04T17:24:06.954177] [INFO ] [MAIN ] AVBC: Wiping existing primary data before full refresh. [2025-11-04T17:24:06.954222] [INFO ] [LOCAL ] AVBC: Starting deletion of primary data. [2025-11-04T17:24:06.954297] [DEBUG] [LOCAL ] AVBC: No primary data file found to delete. [2025-11-04T17:24:06.954344] [INFO ] [MAIN ] AVBC: Starting full primary refresh. [2025-11-04T17:24:07.382020] [INFO ] [REMOTE ] AVBC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:24:07.376260] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-06. [2025-11-04T17:24:07.397622] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-11-04T17:24:08.167112] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-11. [2025-11-04T17:24:08.523485] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T17:24:08.702129] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T17:24:08.703297] [INFO ] [REMOTE ] AVAL: Full fetch complete (15272 records). [2025-11-04T17:24:08.703401] [DEBUG] [VALIDATOR ] AVAL: Validating primary data. [2025-11-04T17:24:08.975148] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-04. [2025-11-04T17:24:09.115990] [DEBUG] [VALIDATOR ] AVAL: All expected trading days are present. [2025-11-04T17:24:09.116090] [DEBUG] [VALIDATOR ] AVAL: Primary data validation passed. [2025-11-04T17:24:09.116463] [INFO ] [MAIN ] AVAL: Fetched primary data is valid. [2025-11-04T17:24:09.116562] [DEBUG] [LOCAL ] AVAL: Saving primary data. [2025-11-04T17:24:09.117260] [DEBUG] [LOCAL ] AVAL: Trimmed 15272 records to 8260 (keeping 200 days of data) [2025-11-04T17:24:09.170860] [DEBUG] [LOCAL ] AVAL: Primary data saved (8260 records). [2025-11-04T17:24:09.171066] [INFO ] [MAIN ] AVAL: Full history saved. [2025-11-04T17:24:09.171160] [INFO ] [MAIN ] Update Progress: 180/2367 symbols completed. AVAL took 22.7s. Estimated remaining: 4h 52m 47s. Rate: 7.5 symbols/min [2025-11-04T17:24:09.378329] [INFO ] [MAIN ] AVD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:24:09.378599] [DEBUG] [LOCAL ] AVD: No existing data file found in primary storage. [2025-11-04T17:24:09.378677] [WARN ] [MAIN ] AVD: No existing data found, performing full refresh instead. [2025-11-04T17:24:09.378815] [INFO ] [MAIN ] AVD: Wiping existing primary data before full refresh. [2025-11-04T17:24:09.378858] [INFO ] [LOCAL ] AVD: Starting deletion of primary data. [2025-11-04T17:24:09.378900] [DEBUG] [LOCAL ] AVD: No primary data file found to delete. [2025-11-04T17:24:09.378933] [INFO ] [MAIN ] AVD: Starting full primary refresh. [2025-11-04T17:24:09.757109] [INFO ] [REMOTE ] AVD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:24:09.767023] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-01. [2025-11-04T17:24:10.390592] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T17:24:10.567087] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-07. [2025-11-04T17:24:10.834562] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-11-04T17:24:11.367091] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-05. [2025-11-04T17:24:11.842962] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T17:24:12.167046] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-01. [2025-11-04T17:24:12.413202] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T17:24:12.448752] [DEBUG] [REMOTE ] AVBC: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:24:12.449071] [WARN ] [REMOTE ] AVBC: No data returned for month 2025-01 [2025-11-04T17:24:12.449157] [FATAL] [REMOTE ] AVBC: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:24:12.449226] [FATAL] [MAIN ] AVBC: Failed to fetch primary data. [2025-11-04T17:24:12.449292] [WARN ] [MAIN ] AVBC: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:24:12.449362] [INFO ] [LOCAL ] AVBC: Starting deletion of primary data. [2025-11-04T17:24:12.449423] [DEBUG] [LOCAL ] AVBC: No primary data file found to delete. [2025-11-04T17:24:12.449459] [INFO ] [LOCAL ] AVBC: Starting deletion of derivative data. [2025-11-04T17:24:12.449498] [DEBUG] [LOCAL ] AVBC: No derivative data file found to delete. [2025-11-04T17:24:12.449542] [INFO ] [MAIN ] Update Progress: 181/2367 symbols completed. AVBC took 5.7s. Estimated remaining: 4h 51m 42s. Rate: 7.5 symbols/min [2025-11-04T17:24:12.633538] [INFO ] [MAIN ] AVK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:24:12.653108] [DEBUG] [LOCAL ] AVK: Loaded 18425 existing records from primary storage. [2025-11-04T17:24:12.653363] [INFO ] [MAIN ] AVK: Loaded 18425 existing records. [2025-11-04T17:24:12.848081] [WARN ] [MAIN ] AVK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:24:12.848334] [INFO ] [MAIN ] AVK: Wiping existing primary data before full refresh. [2025-11-04T17:24:12.848384] [INFO ] [LOCAL ] AVK: Starting deletion of primary data. [2025-11-04T17:24:12.848584] [DEBUG] [LOCAL ] AVK: Deleted primary data file. [2025-11-04T17:24:12.848675] [INFO ] [MAIN ] AVK: Starting full primary refresh. [2025-11-04T17:24:12.972108] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-02. [2025-11-04T17:24:13.213795] [INFO ] [REMOTE ] AVK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:24:13.767016] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-06. [2025-11-04T17:24:13.800519] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T17:24:14.567046] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-08. [2025-11-04T17:24:15.125459] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T17:24:15.367089] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-01. [2025-11-04T17:24:15.834325] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T17:24:16.167047] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-07. [2025-11-04T17:24:16.367157] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T17:24:16.967044] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-02. [2025-11-04T17:24:17.369143] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T17:24:17.608011] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T17:24:17.767044] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-03. [2025-11-04T17:24:18.511391] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T17:24:18.566999] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-04. [2025-11-04T17:24:19.305884] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-11-04T17:24:19.367032] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-09. [2025-11-04T17:24:20.167036] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-08. [2025-11-04T17:24:20.518838] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T17:24:20.967102] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-10. [2025-11-04T17:24:21.668845] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-11-04T17:24:21.767052] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-05. [2025-11-04T17:24:22.039223] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T17:24:22.493132] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T17:24:22.567043] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-03. [2025-11-04T17:24:23.367033] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-11. [2025-11-04T17:24:23.797677] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T17:24:24.167064] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-04. [2025-11-04T17:24:24.273062] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T17:24:24.274697] [INFO ] [REMOTE ] AVA: Full fetch complete (55653 records). [2025-11-04T17:24:24.274869] [DEBUG] [VALIDATOR ] AVA: Validating primary data. [2025-11-04T17:24:24.657406] [DEBUG] [VALIDATOR ] AVA: All expected trading days are present. [2025-11-04T17:24:24.657485] [DEBUG] [VALIDATOR ] AVA: Primary data validation passed. [2025-11-04T17:24:24.658107] [INFO ] [MAIN ] AVA: Fetched primary data is valid. [2025-11-04T17:24:24.658172] [DEBUG] [LOCAL ] AVA: Saving primary data. [2025-11-04T17:24:24.659367] [DEBUG] [LOCAL ] AVA: Trimmed 55653 records to 38014 (keeping 200 days of data) [2025-11-04T17:24:24.886779] [DEBUG] [LOCAL ] AVA: Primary data saved (38014 records). [2025-11-04T17:24:24.887013] [INFO ] [MAIN ] AVA: Full history saved. [2025-11-04T17:24:24.887180] [INFO ] [MAIN ] Update Progress: 182/2367 symbols completed. AVA took 40.9s. Estimated remaining: 4h 52m 27s. Rate: 7.5 symbols/min [2025-11-04T17:24:24.971305] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-06. [2025-11-04T17:24:25.069037] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-11-04T17:24:25.090733] [INFO ] [MAIN ] AVNS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:24:25.130145] [DEBUG] [LOCAL ] AVNS: Loaded 33140 existing records from primary storage. [2025-11-04T17:24:25.130606] [INFO ] [MAIN ] AVNS: Loaded 33140 existing records. [2025-11-04T17:24:25.347524] [WARN ] [MAIN ] AVNS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:24:25.347752] [INFO ] [MAIN ] AVNS: Wiping existing primary data before full refresh. [2025-11-04T17:24:25.347805] [INFO ] [LOCAL ] AVNS: Starting deletion of primary data. [2025-11-04T17:24:25.348115] [DEBUG] [LOCAL ] AVNS: Deleted primary data file. [2025-11-04T17:24:25.348154] [INFO ] [MAIN ] AVNS: Starting full primary refresh. [2025-11-04T17:24:25.703409] [INFO ] [REMOTE ] AVNS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:24:25.772192] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-09. [2025-11-04T17:24:25.779743] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T17:24:26.567060] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-05. [2025-11-04T17:24:27.110517] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T17:24:27.301876] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T17:24:27.367049] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-10. [2025-11-04T17:24:28.167053] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-07. [2025-11-04T17:24:28.649239] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-11-04T17:24:28.967035] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-01. [2025-11-04T17:24:29.096410] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-11-04T17:24:29.767075] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-06. [2025-11-04T17:24:30.372334] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T17:24:30.550790] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T17:24:30.566999] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-07. [2025-11-04T17:24:31.367032] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-11. [2025-11-04T17:24:31.396328] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T17:24:31.832819] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T17:24:31.834577] [INFO ] [REMOTE ] AVB: Full fetch complete (60013 records). [2025-11-04T17:24:31.834701] [DEBUG] [VALIDATOR ] AVB: Validating primary data. [2025-11-04T17:24:32.174014] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-08. [2025-11-04T17:24:32.218380] [DEBUG] [VALIDATOR ] AVB: All expected trading days are present. [2025-11-04T17:24:32.218461] [DEBUG] [VALIDATOR ] AVB: Primary data validation passed. [2025-11-04T17:24:32.219095] [INFO ] [MAIN ] AVB: Fetched primary data is valid. [2025-11-04T17:24:32.219160] [DEBUG] [LOCAL ] AVB: Saving primary data. [2025-11-04T17:24:32.220398] [DEBUG] [LOCAL ] AVB: Trimmed 60013 records to 41985 (keeping 200 days of data) [2025-11-04T17:24:32.479927] [DEBUG] [LOCAL ] AVB: Primary data saved (41985 records). [2025-11-04T17:24:32.480092] [INFO ] [MAIN ] AVB: Full history saved. [2025-11-04T17:24:32.480769] [INFO ] [MAIN ] Update Progress: 183/2367 symbols completed. AVB took 39.2s. Estimated remaining: 4h 52m 14s. Rate: 7.5 symbols/min [2025-11-04T17:24:32.650052] [INFO ] [MAIN ] AVNT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:24:32.686913] [DEBUG] [LOCAL ] AVNT: Loaded 37880 existing records from primary storage. [2025-11-04T17:24:32.687174] [INFO ] [MAIN ] AVNT: Loaded 37880 existing records. [2025-11-04T17:24:32.869463] [WARN ] [MAIN ] AVNT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:24:32.869692] [INFO ] [MAIN ] AVNT: Wiping existing primary data before full refresh. [2025-11-04T17:24:32.869749] [INFO ] [LOCAL ] AVNT: Starting deletion of primary data. [2025-11-04T17:24:32.870007] [DEBUG] [LOCAL ] AVNT: Deleted primary data file. [2025-11-04T17:24:32.870045] [INFO ] [MAIN ] AVNT: Starting full primary refresh. [2025-11-04T17:24:32.967197] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-02. [2025-11-04T17:24:33.163770] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-11-04T17:24:33.246668] [INFO ] [REMOTE ] AVNT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:24:33.767068] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-01. [2025-11-04T17:24:33.883442] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-11-04T17:24:34.567054] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-09. [2025-11-04T17:24:34.835836] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-11-04T17:24:35.271130] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-11-04T17:24:35.367042] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-10. [2025-11-04T17:24:36.170524] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-02. [2025-11-04T17:24:36.178518] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-11-04T17:24:36.967033] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-08. [2025-11-04T17:24:37.157336] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T17:24:37.767011] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-03. [2025-11-04T17:24:37.880097] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-11-04T17:24:38.567064] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-09. [2025-11-04T17:24:38.836159] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T17:24:39.360067] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T17:24:39.367003] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-10. [2025-11-04T17:24:40.167016] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-04. [2025-11-04T17:24:40.187507] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T17:24:40.967091] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-11. [2025-11-04T17:24:41.389052] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T17:24:41.678549] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T17:24:41.680168] [INFO ] [REMOTE ] AVD: Full fetch complete (32027 records). [2025-11-04T17:24:41.680268] [DEBUG] [VALIDATOR ] AVD: Validating primary data. [2025-11-04T17:24:41.772294] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-03. [2025-11-04T17:24:42.067160] [DEBUG] [VALIDATOR ] AVD: All expected trading days are present. [2025-11-04T17:24:42.067242] [DEBUG] [VALIDATOR ] AVD: Primary data validation passed. [2025-11-04T17:24:42.067669] [INFO ] [MAIN ] AVD: Fetched primary data is valid. [2025-11-04T17:24:42.067760] [DEBUG] [LOCAL ] AVD: Saving primary data. [2025-11-04T17:24:42.068460] [DEBUG] [LOCAL ] AVD: Trimmed 32027 records to 20822 (keeping 200 days of data) [2025-11-04T17:24:42.179279] [DEBUG] [LOCAL ] AVD: Primary data saved (20822 records). [2025-11-04T17:24:42.179405] [INFO ] [MAIN ] AVD: Full history saved. [2025-11-04T17:24:42.179475] [INFO ] [MAIN ] Update Progress: 184/2367 symbols completed. AVD took 33.0s. Estimated remaining: 4h 52m 26s. Rate: 7.5 symbols/min [2025-11-04T17:24:42.344083] [INFO ] [MAIN ] AVTR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:24:42.392509] [DEBUG] [LOCAL ] AVTR: Loaded 53570 existing records from primary storage. [2025-11-04T17:24:42.393004] [INFO ] [MAIN ] AVTR: Loaded 53570 existing records. [2025-11-04T17:24:42.565815] [WARN ] [MAIN ] AVTR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:24:42.566017] [INFO ] [MAIN ] AVTR: Wiping existing primary data before full refresh. [2025-11-04T17:24:42.566063] [INFO ] [LOCAL ] AVTR: Starting deletion of primary data. [2025-11-04T17:24:42.566460] [DEBUG] [LOCAL ] AVTR: Deleted primary data file. [2025-11-04T17:24:42.566515] [INFO ] [MAIN ] AVTR: Starting full primary refresh. [2025-11-04T17:24:42.577523] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-05. [2025-11-04T17:24:42.926900] [INFO ] [REMOTE ] AVTR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:24:43.015072] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T17:24:43.367050] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-01. [2025-11-04T17:24:43.798868] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T17:24:44.167060] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-04. [2025-11-04T17:24:44.613832] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T17:24:44.967153] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-02. [2025-11-04T17:24:45.570451] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T17:24:45.767102] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-05. [2025-11-04T17:24:46.266762] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T17:24:46.567088] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-11. [2025-11-04T17:24:46.795054] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-11-04T17:24:47.366137] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-11 (0.80 seconds). [2025-11-04T17:24:47.367041] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-06. [2025-11-04T17:24:47.368083] [INFO ] [REMOTE ] AVK: Full fetch complete (26688 records). [2025-11-04T17:24:47.368167] [DEBUG] [VALIDATOR ] AVK: Validating primary data. [2025-11-04T17:24:47.749803] [DEBUG] [VALIDATOR ] AVK: All expected trading days are present. [2025-11-04T17:24:47.749882] [DEBUG] [VALIDATOR ] AVK: Primary data validation passed. [2025-11-04T17:24:47.750240] [INFO ] [MAIN ] AVK: Fetched primary data is valid. [2025-11-04T17:24:47.750278] [DEBUG] [LOCAL ] AVK: Saving primary data. [2025-11-04T17:24:47.750954] [DEBUG] [LOCAL ] AVK: Trimmed 26688 records to 18454 (keeping 200 days of data) [2025-11-04T17:24:47.861526] [DEBUG] [LOCAL ] AVK: Primary data saved (18454 records). [2025-11-04T17:24:47.861675] [INFO ] [MAIN ] AVK: Full history saved. [2025-11-04T17:24:47.861785] [INFO ] [MAIN ] Update Progress: 185/2367 symbols completed. AVK took 35.4s. Estimated remaining: 4h 51m 50s. Rate: 7.5 symbols/min [2025-11-04T17:24:48.037044] [INFO ] [MAIN ] AVY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:24:48.078509] [DEBUG] [LOCAL ] AVY: Loaded 37660 existing records from primary storage. [2025-11-04T17:24:48.078821] [INFO ] [MAIN ] AVY: Loaded 37660 existing records. [2025-11-04T17:24:48.172273] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-06. [2025-11-04T17:24:48.273904] [WARN ] [MAIN ] AVY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:24:48.274127] [INFO ] [MAIN ] AVY: Wiping existing primary data before full refresh. [2025-11-04T17:24:48.274175] [INFO ] [LOCAL ] AVY: Starting deletion of primary data. [2025-11-04T17:24:48.274349] [DEBUG] [LOCAL ] AVY: Deleted primary data file. [2025-11-04T17:24:48.274387] [INFO ] [MAIN ] AVY: Starting full primary refresh. [2025-11-04T17:24:48.653113] [INFO ] [REMOTE ] AVY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:24:48.967077] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-01. [2025-11-04T17:24:49.195397] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T17:24:49.710697] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-06 (2.34 seconds). [2025-11-04T17:24:49.767007] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-07. [2025-11-04T17:24:50.352944] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-11-04T17:24:50.567036] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-03. [2025-11-04T17:24:50.992492] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T17:24:51.367039] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-08. [2025-11-04T17:24:52.124630] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T17:24:52.167117] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-02. [2025-11-04T17:24:52.446028] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-11-04T17:24:52.967110] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-07. [2025-11-04T17:24:53.364018] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T17:24:53.767107] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-09. [2025-11-04T17:24:54.173336] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-11-04T17:24:54.567080] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-04. [2025-11-04T17:24:54.816686] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T17:24:55.367094] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-10. [2025-11-04T17:24:56.081228] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T17:24:56.167085] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-05. [2025-11-04T17:24:56.602456] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-11-04T17:24:56.967056] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-11. [2025-11-04T17:24:57.591042] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-11-04T17:24:57.767094] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-03. [2025-11-04T17:24:57.972123] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-11 (1.01 seconds). [2025-11-04T17:24:57.973797] [INFO ] [REMOTE ] AVNT: Full fetch complete (55949 records). [2025-11-04T17:24:57.973900] [DEBUG] [VALIDATOR ] AVNT: Validating primary data. [2025-11-04T17:24:58.359763] [DEBUG] [VALIDATOR ] AVNT: All expected trading days are present. [2025-11-04T17:24:58.359859] [DEBUG] [VALIDATOR ] AVNT: Primary data validation passed. [2025-11-04T17:24:58.360553] [INFO ] [MAIN ] AVNT: Fetched primary data is valid. [2025-11-04T17:24:58.360603] [DEBUG] [LOCAL ] AVNT: Saving primary data. [2025-11-04T17:24:58.362136] [DEBUG] [LOCAL ] AVNT: Trimmed 55949 records to 37919 (keeping 200 days of data) [2025-11-04T17:24:58.569888] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-08. [2025-11-04T17:24:58.591789] [DEBUG] [LOCAL ] AVNT: Primary data saved (37919 records). [2025-11-04T17:24:58.591958] [INFO ] [MAIN ] AVNT: Full history saved. [2025-11-04T17:24:58.592520] [INFO ] [MAIN ] Update Progress: 186/2367 symbols completed. AVNT took 26.1s. Estimated remaining: 4h 52m 14s. Rate: 7.5 symbols/min [2025-11-04T17:24:58.773688] [INFO ] [MAIN ] AWF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:24:58.799295] [DEBUG] [LOCAL ] AWF: Loaded 23582 existing records from primary storage. [2025-11-04T17:24:58.799558] [INFO ] [MAIN ] AWF: Loaded 23582 existing records. [2025-11-04T17:24:58.984729] [WARN ] [MAIN ] AWF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:24:58.984972] [INFO ] [MAIN ] AWF: Wiping existing primary data before full refresh. [2025-11-04T17:24:58.985095] [INFO ] [LOCAL ] AWF: Starting deletion of primary data. [2025-11-04T17:24:58.985304] [DEBUG] [LOCAL ] AWF: Deleted primary data file. [2025-11-04T17:24:58.985361] [INFO ] [MAIN ] AWF: Starting full primary refresh. [2025-11-04T17:24:59.387317] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-06. [2025-11-04T17:24:59.414949] [INFO ] [REMOTE ] AWF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:24:59.418757] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-11-04T17:24:59.746930] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-11-04T17:25:00.167075] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-04. [2025-11-04T17:25:00.950460] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-11-04T17:25:00.967004] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-07. [2025-11-04T17:25:01.767111] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-01. [2025-11-04T17:25:02.030790] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-11-04T17:25:02.478776] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T17:25:02.567094] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-08. [2025-11-04T17:25:02.915489] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T17:25:03.367048] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-09. [2025-11-04T17:25:04.167070] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-05. [2025-11-04T17:25:04.249768] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-11-04T17:25:04.698571] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T17:25:04.967060] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-09. [2025-11-04T17:25:05.195266] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-11-04T17:25:05.767062] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-02. [2025-11-04T17:25:06.552940] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T17:25:06.575975] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-10. [2025-11-04T17:25:06.580062] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-11-04T17:25:07.367032] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-03. [2025-11-04T17:25:08.176791] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-10. [2025-11-04T17:25:08.183397] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-11-04T17:25:08.217598] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-11-04T17:25:08.967063] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-04. [2025-11-04T17:25:09.269605] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-11-04T17:25:09.767015] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-11. [2025-11-04T17:25:09.882806] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T17:25:10.567043] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-06. [2025-11-04T17:25:10.966491] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T17:25:10.968378] [INFO ] [REMOTE ] AVTR: Full fetch complete (77289 records). [2025-11-04T17:25:10.968499] [DEBUG] [VALIDATOR ] AVTR: Validating primary data. [2025-11-04T17:25:10.969429] [FATAL] [VALIDATOR ] AVTR: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:25:10.969522] [FATAL] [MAIN ] AVTR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:25:10.969603] [WARN ] [MAIN ] AVTR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:25:10.969847] [INFO ] [LOCAL ] AVTR: Starting deletion of primary data. [2025-11-04T17:25:10.969935] [DEBUG] [LOCAL ] AVTR: No primary data file found to delete. [2025-11-04T17:25:10.969982] [INFO ] [LOCAL ] AVTR: Starting deletion of derivative data. [2025-11-04T17:25:10.970126] [DEBUG] [LOCAL ] AVTR: Deleted derivative data file. [2025-11-04T17:25:10.971120] [INFO ] [MAIN ] Update Progress: 187/2367 symbols completed. AVTR took 28.8s. Estimated remaining: 4h 52m 56s. Rate: 7.4 symbols/min [2025-11-04T17:25:11.190801] [INFO ] [MAIN ] AWI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:25:11.223814] [DEBUG] [LOCAL ] AWI: Loaded 25025 existing records from primary storage. [2025-11-04T17:25:11.224127] [INFO ] [MAIN ] AWI: Loaded 25025 existing records. [2025-11-04T17:25:11.372127] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-05. [2025-11-04T17:25:11.414680] [WARN ] [MAIN ] AWI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:25:11.415214] [INFO ] [MAIN ] AWI: Wiping existing primary data before full refresh. [2025-11-04T17:25:11.415350] [INFO ] [LOCAL ] AWI: Starting deletion of primary data. [2025-11-04T17:25:11.415623] [DEBUG] [LOCAL ] AWI: Deleted primary data file. [2025-11-04T17:25:11.415710] [INFO ] [MAIN ] AWI: Starting full primary refresh. [2025-11-04T17:25:11.805987] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T17:25:11.838084] [INFO ] [REMOTE ] AWI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:25:12.167010] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-01. [2025-11-04T17:25:12.286516] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-11-04T17:25:12.967026] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-07. [2025-11-04T17:25:13.095100] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-11-04T17:25:13.767062] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-06. [2025-11-04T17:25:14.198559] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T17:25:14.567086] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-11. [2025-11-04T17:25:14.625767] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T17:25:15.367109] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-08. [2025-11-04T17:25:15.591932] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T17:25:15.593671] [INFO ] [REMOTE ] AVNS: Full fetch complete (47660 records). [2025-11-04T17:25:15.593857] [DEBUG] [VALIDATOR ] AVNS: Validating primary data. [2025-11-04T17:25:15.595421] [FATAL] [VALIDATOR ] AVNS: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:25:15.595517] [FATAL] [MAIN ] AVNS: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:25:15.595586] [WARN ] [MAIN ] AVNS: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:25:15.595893] [INFO ] [LOCAL ] AVNS: Starting deletion of primary data. [2025-11-04T17:25:15.595980] [DEBUG] [LOCAL ] AVNS: No primary data file found to delete. [2025-11-04T17:25:15.596023] [INFO ] [LOCAL ] AVNS: Starting deletion of derivative data. [2025-11-04T17:25:15.596169] [DEBUG] [LOCAL ] AVNS: Deleted derivative data file. [2025-11-04T17:25:15.596277] [INFO ] [MAIN ] Update Progress: 188/2367 symbols completed. AVNS took 50.7s. Estimated remaining: 4h 52m 8s. Rate: 7.5 symbols/min [2025-11-04T17:25:15.796817] [INFO ] [MAIN ] AWK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:25:15.797132] [DEBUG] [LOCAL ] AWK: No existing data file found in primary storage. [2025-11-04T17:25:15.797176] [WARN ] [MAIN ] AWK: No existing data found, performing full refresh instead. [2025-11-04T17:25:15.797313] [INFO ] [MAIN ] AWK: Wiping existing primary data before full refresh. [2025-11-04T17:25:15.797356] [INFO ] [LOCAL ] AWK: Starting deletion of primary data. [2025-11-04T17:25:15.797399] [DEBUG] [LOCAL ] AWK: No primary data file found to delete. [2025-11-04T17:25:15.797434] [INFO ] [MAIN ] AWK: Starting full primary refresh. [2025-11-04T17:25:16.172278] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-02. [2025-11-04T17:25:16.236971] [INFO ] [REMOTE ] AWK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:25:16.511055] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T17:25:16.967007] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-01. [2025-11-04T17:25:16.998221] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T17:25:17.767042] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-07. [2025-11-04T17:25:18.403999] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T17:25:18.567023] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-02. [2025-11-04T17:25:18.613533] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T17:25:19.367043] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-08. [2025-11-04T17:25:19.778669] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T17:25:19.985518] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-11-04T17:25:20.167057] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-09. [2025-11-04T17:25:20.967016] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-03. [2025-11-04T17:25:21.014960] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-11-04T17:25:21.767033] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-09. [2025-11-04T17:25:21.793718] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-11-04T17:25:22.567047] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-10. [2025-11-04T17:25:22.981296] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T17:25:23.367074] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-10. [2025-11-04T17:25:23.458199] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-11-04T17:25:24.167044] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-11. [2025-11-04T17:25:24.861207] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-11-04T17:25:24.967056] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-11. [2025-11-04T17:25:25.081364] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T17:25:25.082724] [INFO ] [REMOTE ] AWF: Full fetch complete (35589 records). [2025-11-04T17:25:25.082823] [DEBUG] [VALIDATOR ] AWF: Validating primary data. [2025-11-04T17:25:25.498450] [DEBUG] [VALIDATOR ] AWF: All expected trading days are present. [2025-11-04T17:25:25.498520] [DEBUG] [VALIDATOR ] AWF: Primary data validation passed. [2025-11-04T17:25:25.499110] [INFO ] [MAIN ] AWF: Fetched primary data is valid. [2025-11-04T17:25:25.499193] [DEBUG] [LOCAL ] AWF: Saving primary data. [2025-11-04T17:25:25.500222] [DEBUG] [LOCAL ] AWF: Trimmed 35589 records to 23535 (keeping 200 days of data) [2025-11-04T17:25:25.647382] [DEBUG] [LOCAL ] AWF: Primary data saved (23535 records). [2025-11-04T17:25:25.647504] [INFO ] [MAIN ] AWF: Full history saved. [2025-11-04T17:25:25.647581] [INFO ] [MAIN ] Update Progress: 189/2367 symbols completed. AWF took 27.1s. Estimated remaining: 4h 52m 23s. Rate: 7.4 symbols/min [2025-11-04T17:25:25.772135] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-04. [2025-11-04T17:25:25.865830] [INFO ] [MAIN ] AWP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:25:25.885704] [DEBUG] [LOCAL ] AWP: Loaded 21342 existing records from primary storage. [2025-11-04T17:25:25.885955] [INFO ] [MAIN ] AWP: Loaded 21342 existing records. [2025-11-04T17:25:26.051163] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-11 (1.08 seconds). [2025-11-04T17:25:26.070430] [WARN ] [MAIN ] AWP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:25:26.072044] [INFO ] [MAIN ] AWP: Wiping existing primary data before full refresh. [2025-11-04T17:25:26.072102] [INFO ] [LOCAL ] AWP: Starting deletion of primary data. [2025-11-04T17:25:26.072292] [DEBUG] [LOCAL ] AWP: Deleted primary data file. [2025-11-04T17:25:26.072363] [INFO ] [MAIN ] AWP: Starting full primary refresh. [2025-11-04T17:25:26.072256] [INFO ] [REMOTE ] AVY: Full fetch complete (56605 records). [2025-11-04T17:25:26.103924] [DEBUG] [VALIDATOR ] AVY: Validating primary data. [2025-11-04T17:25:26.573952] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-03. [2025-11-04T17:25:26.796869] [INFO ] [REMOTE ] AWP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:25:26.840524] [DEBUG] [VALIDATOR ] AVY: All expected trading days are present. [2025-11-04T17:25:26.840754] [DEBUG] [VALIDATOR ] AVY: Primary data validation passed. [2025-11-04T17:25:26.841559] [INFO ] [MAIN ] AVY: Fetched primary data is valid. [2025-11-04T17:25:26.841675] [DEBUG] [LOCAL ] AVY: Saving primary data. [2025-11-04T17:25:26.842985] [DEBUG] [LOCAL ] AVY: Trimmed 56605 records to 37630 (keeping 200 days of data) [2025-11-04T17:25:27.075515] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T17:25:27.116729] [DEBUG] [LOCAL ] AVY: Primary data saved (37630 records). [2025-11-04T17:25:27.117738] [INFO ] [MAIN ] AVY: Full history saved. [2025-11-04T17:25:27.118887] [INFO ] [MAIN ] Update Progress: 190/2367 symbols completed. AVY took 39.3s. Estimated remaining: 4h 50m 60s. Rate: 7.5 symbols/min [2025-11-04T17:25:27.295782] [INFO ] [MAIN ] AWR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:25:27.318895] [DEBUG] [LOCAL ] AWR: Loaded 22506 existing records from primary storage. [2025-11-04T17:25:27.319160] [INFO ] [MAIN ] AWR: Loaded 22506 existing records. [2025-11-04T17:25:27.372889] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-05. [2025-11-04T17:25:27.496327] [WARN ] [MAIN ] AWR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:25:27.496536] [INFO ] [MAIN ] AWR: Wiping existing primary data before full refresh. [2025-11-04T17:25:27.496593] [INFO ] [LOCAL ] AWR: Starting deletion of primary data. [2025-11-04T17:25:27.496787] [DEBUG] [LOCAL ] AWR: Deleted primary data file. [2025-11-04T17:25:27.496892] [INFO ] [MAIN ] AWR: Starting full primary refresh. [2025-11-04T17:25:27.888443] [INFO ] [REMOTE ] AWR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:25:28.099393] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T17:25:28.167041] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-01. [2025-11-04T17:25:28.258127] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T17:25:28.967074] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-06. [2025-11-04T17:25:29.168535] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T17:25:29.767017] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-02. [2025-11-04T17:25:29.827633] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T17:25:30.544990] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-11-04T17:25:30.567009] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-01. [2025-11-04T17:25:31.367026] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-07. [2025-11-04T17:25:31.544107] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T17:25:32.140573] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T17:25:32.167006] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-02. [2025-11-04T17:25:32.908297] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-11-04T17:25:32.967020] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-03. [2025-11-04T17:25:33.696975] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-11-04T17:25:33.767036] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-04. [2025-11-04T17:25:34.455786] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-11-04T17:25:34.567007] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-08. [2025-11-04T17:25:35.367101] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-04. [2025-11-04T17:25:35.514769] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T17:25:36.167106] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-03. [2025-11-04T17:25:36.630672] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T17:25:36.835628] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-11-04T17:25:36.967077] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-09. [2025-11-04T17:25:37.767008] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-05. [2025-11-04T17:25:37.931852] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T17:25:38.470144] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-11-04T17:25:38.567056] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-06. [2025-11-04T17:25:39.250879] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-11-04T17:25:39.367104] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-07. [2025-11-04T17:25:40.167042] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-05. [2025-11-04T17:25:40.442509] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T17:25:40.967077] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-08. [2025-11-04T17:25:41.447013] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T17:25:41.767122] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-10. [2025-11-04T17:25:41.851925] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-11-04T17:25:42.567033] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-06. [2025-11-04T17:25:42.714267] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T17:25:43.367085] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-09. [2025-11-04T17:25:43.762914] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T17:25:43.938960] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-11-04T17:25:44.167117] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-11. [2025-11-04T17:25:44.967091] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-04. [2025-11-04T17:25:45.009035] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T17:25:45.010834] [INFO ] [REMOTE ] AWI: Full fetch complete (36714 records). [2025-11-04T17:25:45.010936] [DEBUG] [VALIDATOR ] AWI: Validating primary data. [2025-11-04T17:25:45.387114] [DEBUG] [VALIDATOR ] AWI: All expected trading days are present. [2025-11-04T17:25:45.387193] [DEBUG] [VALIDATOR ] AWI: Primary data validation passed. [2025-11-04T17:25:45.387627] [INFO ] [MAIN ] AWI: Fetched primary data is valid. [2025-11-04T17:25:45.387702] [DEBUG] [LOCAL ] AWI: Saving primary data. [2025-11-04T17:25:45.388465] [DEBUG] [LOCAL ] AWI: Trimmed 36714 records to 24986 (keeping 200 days of data) [2025-11-04T17:25:45.543343] [DEBUG] [LOCAL ] AWI: Primary data saved (24986 records). [2025-11-04T17:25:45.543473] [INFO ] [MAIN ] AWI: Full history saved. [2025-11-04T17:25:45.543553] [INFO ] [MAIN ] Update Progress: 191/2367 symbols completed. AWI took 34.6s. Estimated remaining: 4h 52m 50s. Rate: 7.4 symbols/min [2025-11-04T17:25:45.709408] [INFO ] [MAIN ] AX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:25:45.735715] [DEBUG] [LOCAL ] AX: Loaded 27158 existing records from primary storage. [2025-11-04T17:25:45.735979] [INFO ] [MAIN ] AX: Loaded 27158 existing records. [2025-11-04T17:25:45.772229] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-07. [2025-11-04T17:25:45.918564] [WARN ] [MAIN ] AX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:25:45.919123] [INFO ] [MAIN ] AX: Wiping existing primary data before full refresh. [2025-11-04T17:25:45.919397] [INFO ] [LOCAL ] AX: Starting deletion of primary data. [2025-11-04T17:25:45.919766] [DEBUG] [LOCAL ] AX: Deleted primary data file. [2025-11-04T17:25:45.919870] [INFO ] [MAIN ] AX: Starting full primary refresh. [2025-11-04T17:25:45.988811] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T17:25:46.332141] [INFO ] [REMOTE ] AX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:25:46.567053] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-01. [2025-11-04T17:25:47.325424] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-11-04T17:25:47.367105] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-08. [2025-11-04T17:25:47.741943] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T17:25:48.167074] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-02. [2025-11-04T17:25:48.654296] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T17:25:48.967011] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-09. [2025-11-04T17:25:49.022677] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-11-04T17:25:49.767032] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-05. [2025-11-04T17:25:50.345952] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T17:25:50.570284] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-10. [2025-11-04T17:25:50.573997] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T17:25:51.316274] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-11-04T17:25:51.367029] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-06. [2025-11-04T17:25:52.065876] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T17:25:52.167015] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-10. [2025-11-04T17:25:52.967093] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-07. [2025-11-04T17:25:53.596490] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T17:25:53.707192] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T17:25:53.767132] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-08. [2025-11-04T17:25:54.567063] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-11. [2025-11-04T17:25:54.582482] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-11-04T17:25:55.041328] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T17:25:55.044152] [INFO ] [REMOTE ] AWK: Full fetch complete (71008 records). [2025-11-04T17:25:55.044411] [DEBUG] [VALIDATOR ] AWK: Validating primary data. [2025-11-04T17:25:55.367308] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-11. [2025-11-04T17:25:55.465509] [DEBUG] [VALIDATOR ] AWK: All expected trading days are present. [2025-11-04T17:25:55.465608] [DEBUG] [VALIDATOR ] AWK: Primary data validation passed. [2025-11-04T17:25:55.466392] [INFO ] [MAIN ] AWK: Fetched primary data is valid. [2025-11-04T17:25:55.466453] [DEBUG] [LOCAL ] AWK: Saving primary data. [2025-11-04T17:25:55.467830] [DEBUG] [LOCAL ] AWK: Trimmed 71008 records to 48371 (keeping 200 days of data) [2025-11-04T17:25:55.754343] [DEBUG] [LOCAL ] AWK: Primary data saved (48371 records). [2025-11-04T17:25:55.754473] [INFO ] [MAIN ] AWK: Full history saved. [2025-11-04T17:25:55.754541] [INFO ] [MAIN ] Update Progress: 192/2367 symbols completed. AWK took 40.2s. Estimated remaining: 4h 53m 6s. Rate: 7.4 symbols/min [2025-11-04T17:25:55.921242] [INFO ] [MAIN ] AXL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:25:55.921426] [DEBUG] [LOCAL ] AXL: No existing data file found in primary storage. [2025-11-04T17:25:55.921481] [WARN ] [MAIN ] AXL: No existing data found, performing full refresh instead. [2025-11-04T17:25:55.921587] [INFO ] [MAIN ] AXL: Wiping existing primary data before full refresh. [2025-11-04T17:25:55.921627] [INFO ] [LOCAL ] AXL: Starting deletion of primary data. [2025-11-04T17:25:55.921708] [DEBUG] [LOCAL ] AXL: No primary data file found to delete. [2025-11-04T17:25:55.921750] [INFO ] [MAIN ] AXL: Starting full primary refresh. [2025-11-04T17:25:56.172206] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-09. [2025-11-04T17:25:56.270109] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T17:25:56.294037] [INFO ] [REMOTE ] AXL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:25:56.296429] [INFO ] [REMOTE ] AWP: Full fetch complete (30971 records). [2025-11-04T17:25:56.296616] [DEBUG] [VALIDATOR ] AWP: Validating primary data. [2025-11-04T17:25:56.683494] [DEBUG] [VALIDATOR ] AWP: All expected trading days are present. [2025-11-04T17:25:56.683573] [DEBUG] [VALIDATOR ] AWP: Primary data validation passed. [2025-11-04T17:25:56.684005] [INFO ] [MAIN ] AWP: Fetched primary data is valid. [2025-11-04T17:25:56.684065] [DEBUG] [LOCAL ] AWP: Saving primary data. [2025-11-04T17:25:56.684809] [DEBUG] [LOCAL ] AWP: Trimmed 30971 records to 21316 (keeping 200 days of data) [2025-11-04T17:25:56.810201] [DEBUG] [LOCAL ] AWP: Primary data saved (21316 records). [2025-11-04T17:25:56.810350] [INFO ] [MAIN ] AWP: Full history saved. [2025-11-04T17:25:56.810454] [INFO ] [MAIN ] Update Progress: 193/2367 symbols completed. AWP took 31.2s. Estimated remaining: 4h 51m 39s. Rate: 7.5 symbols/min [2025-11-04T17:25:56.987757] [INFO ] [MAIN ] AXP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:25:56.981469] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-01. [2025-11-04T17:25:57.036482] [DEBUG] [LOCAL ] AXP: Loaded 53081 existing records from primary storage. [2025-11-04T17:25:57.036885] [INFO ] [MAIN ] AXP: Loaded 53081 existing records. [2025-11-04T17:25:57.216346] [WARN ] [MAIN ] AXP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:25:57.216677] [INFO ] [MAIN ] AXP: Wiping existing primary data before full refresh. [2025-11-04T17:25:57.216748] [INFO ] [LOCAL ] AXP: Starting deletion of primary data. [2025-11-04T17:25:57.217196] [DEBUG] [LOCAL ] AXP: Deleted primary data file. [2025-11-04T17:25:57.224073] [INFO ] [MAIN ] AXP: Starting full primary refresh. [2025-11-04T17:25:57.315385] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T17:25:57.599607] [INFO ] [REMOTE ] AXP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:25:57.767029] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-01. [2025-11-04T17:25:58.320396] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T17:25:58.567064] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-02. [2025-11-04T17:25:59.367048] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-10. [2025-11-04T17:25:59.663615] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-11-04T17:25:59.860933] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T17:26:00.167071] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-03. [2025-11-04T17:26:00.207316] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-11-04T17:26:00.967075] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-11. [2025-11-04T17:26:01.767033] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-03. [2025-11-04T17:26:01.910345] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T17:26:01.914339] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-11-04T17:26:01.925572] [INFO ] [REMOTE ] AWR: Full fetch complete (32741 records). [2025-11-04T17:26:01.926383] [DEBUG] [VALIDATOR ] AWR: Validating primary data. [2025-11-04T17:26:02.294916] [DEBUG] [VALIDATOR ] AWR: All expected trading days are present. [2025-11-04T17:26:02.295015] [DEBUG] [VALIDATOR ] AWR: Primary data validation passed. [2025-11-04T17:26:02.295472] [INFO ] [MAIN ] AWR: Fetched primary data is valid. [2025-11-04T17:26:02.295524] [DEBUG] [LOCAL ] AWR: Saving primary data. [2025-11-04T17:26:02.296399] [DEBUG] [LOCAL ] AWR: Trimmed 32741 records to 22611 (keeping 200 days of data) [2025-11-04T17:26:02.429621] [DEBUG] [LOCAL ] AWR: Primary data saved (22611 records). [2025-11-04T17:26:02.429776] [INFO ] [MAIN ] AWR: Full history saved. [2025-11-04T17:26:02.429899] [INFO ] [MAIN ] Update Progress: 194/2367 symbols completed. AWR took 35.3s. Estimated remaining: 4h 51m 4s. Rate: 7.5 symbols/min [2025-11-04T17:26:02.567884] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-02. [2025-11-04T17:26:02.598535] [INFO ] [MAIN ] AXR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:26:02.603259] [DEBUG] [LOCAL ] AXR: Loaded 2720 existing records from primary storage. [2025-11-04T17:26:02.603403] [INFO ] [MAIN ] AXR: Loaded 2720 existing records. [2025-11-04T17:26:02.779733] [WARN ] [MAIN ] AXR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:26:02.779961] [INFO ] [MAIN ] AXR: Wiping existing primary data before full refresh. [2025-11-04T17:26:02.780010] [INFO ] [LOCAL ] AXR: Starting deletion of primary data. [2025-11-04T17:26:02.780164] [DEBUG] [LOCAL ] AXR: Deleted primary data file. [2025-11-04T17:26:02.780225] [INFO ] [MAIN ] AXR: Starting full primary refresh. [2025-11-04T17:26:03.153754] [INFO ] [REMOTE ] AXR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:26:03.172463] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T17:26:03.367032] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-04. [2025-11-04T17:26:04.167036] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-01. [2025-11-04T17:26:04.355178] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-11-04T17:26:04.581388] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T17:26:04.716111] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-11-04T17:26:04.967105] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-02. [2025-11-04T17:26:05.291224] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T17:26:05.767128] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-03. [2025-11-04T17:26:06.567045] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-05. [2025-11-04T17:26:07.370558] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-03. [2025-11-04T17:26:07.382338] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-11-04T17:26:07.782961] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-11-04T17:26:07.916266] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-11-04T17:26:08.167083] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-06. [2025-11-04T17:26:08.967051] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-04. [2025-11-04T17:26:09.471324] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T17:26:09.767070] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-04. [2025-11-04T17:26:10.052948] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T17:26:10.146667] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T17:26:10.567074] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-07. [2025-11-04T17:26:11.367090] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-05. [2025-11-04T17:26:11.731306] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-11-04T17:26:11.746155] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-11-04T17:26:12.167043] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-08. [2025-11-04T17:26:12.967036] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-06. [2025-11-04T17:26:13.292756] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-11-04T17:26:13.738176] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-11-04T17:26:13.767018] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-09. [2025-11-04T17:26:14.567049] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-04. [2025-11-04T17:26:15.104780] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T17:26:15.367041] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-07. [2025-11-04T17:26:15.914744] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-11-04T17:26:16.167018] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-08. [2025-11-04T17:26:16.394229] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-11-04T17:26:16.456275] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-11-04T17:26:16.967057] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-09. [2025-11-04T17:26:17.332418] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-11-04T17:26:17.767065] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-05. [2025-11-04T17:26:17.825128] [DEBUG] [REMOTE ] AXP: Error fetching primary data for 2025-05: 'API call returned an empty response.' [2025-11-04T17:26:17.825385] [WARN ] [REMOTE ] AXP: No data returned for month 2025-05 [2025-11-04T17:26:17.825475] [FATAL] [REMOTE ] AXP: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:26:17.825550] [FATAL] [MAIN ] AXP: Failed to fetch primary data. [2025-11-04T17:26:17.825587] [WARN ] [MAIN ] AXP: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:26:17.825674] [INFO ] [LOCAL ] AXP: Starting deletion of primary data. [2025-11-04T17:26:17.825751] [DEBUG] [LOCAL ] AXP: No primary data file found to delete. [2025-11-04T17:26:17.825789] [INFO ] [LOCAL ] AXP: Starting deletion of derivative data. [2025-11-04T17:26:17.825930] [DEBUG] [LOCAL ] AXP: Deleted derivative data file. [2025-11-04T17:26:17.826499] [INFO ] [MAIN ] Update Progress: 195/2367 symbols completed. AXP took 21.0s. Estimated remaining: 4h 52m 18s. Rate: 7.4 symbols/min [2025-11-04T17:26:17.999981] [INFO ] [MAIN ] AXS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:26:18.039777] [DEBUG] [LOCAL ] AXS: Loaded 38961 existing records from primary storage. [2025-11-04T17:26:18.040059] [INFO ] [MAIN ] AXS: Loaded 38961 existing records. [2025-11-04T17:26:18.216659] [WARN ] [MAIN ] AXS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:26:18.216919] [INFO ] [MAIN ] AXS: Wiping existing primary data before full refresh. [2025-11-04T17:26:18.216977] [INFO ] [LOCAL ] AXS: Starting deletion of primary data. [2025-11-04T17:26:18.217191] [DEBUG] [LOCAL ] AXS: Deleted primary data file. [2025-11-04T17:26:18.217248] [INFO ] [MAIN ] AXS: Starting full primary refresh. [2025-11-04T17:26:18.574386] [INFO ] [REMOTE ] AXS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:26:18.570698] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-05. [2025-11-04T17:26:19.367029] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-10. [2025-11-04T17:26:19.686735] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-10 (0.32 seconds). [2025-11-04T17:26:19.868991] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T17:26:20.167035] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-06. [2025-11-04T17:26:20.894212] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-11-04T17:26:20.967018] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-11. [2025-11-04T17:26:21.356430] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-11 (0.39 seconds). [2025-11-04T17:26:21.357149] [INFO ] [REMOTE ] AXR: Full fetch complete (4153 records). [2025-11-04T17:26:21.357216] [DEBUG] [VALIDATOR ] AXR: Validating primary data. [2025-11-04T17:26:21.732064] [DEBUG] [VALIDATOR ] AXR: All expected trading days are present. [2025-11-04T17:26:21.732223] [DEBUG] [VALIDATOR ] AXR: Primary data validation passed. [2025-11-04T17:26:21.732357] [INFO ] [MAIN ] AXR: Fetched primary data is valid. [2025-11-04T17:26:21.732396] [DEBUG] [LOCAL ] AXR: Saving primary data. [2025-11-04T17:26:21.732775] [DEBUG] [LOCAL ] AXR: Trimmed 4153 records to 2709 (keeping 200 days of data) [2025-11-04T17:26:21.748300] [DEBUG] [LOCAL ] AXR: Primary data saved (2709 records). [2025-11-04T17:26:21.748376] [INFO ] [MAIN ] AXR: Full history saved. [2025-11-04T17:26:21.748448] [INFO ] [MAIN ] Update Progress: 196/2367 symbols completed. AXR took 19.3s. Estimated remaining: 4h 51m 24s. Rate: 7.5 symbols/min [2025-11-04T17:26:21.777020] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-01. [2025-11-04T17:26:21.917213] [INFO ] [MAIN ] AXTA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:26:21.969537] [DEBUG] [LOCAL ] AXTA: Loaded 52358 existing records from primary storage. [2025-11-04T17:26:21.969861] [INFO ] [MAIN ] AXTA: Loaded 52358 existing records. [2025-11-04T17:26:22.142910] [WARN ] [MAIN ] AXTA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:26:22.143135] [INFO ] [MAIN ] AXTA: Wiping existing primary data before full refresh. [2025-11-04T17:26:22.143181] [INFO ] [LOCAL ] AXTA: Starting deletion of primary data. [2025-11-04T17:26:22.143367] [DEBUG] [LOCAL ] AXTA: Deleted primary data file. [2025-11-04T17:26:22.143404] [INFO ] [MAIN ] AXTA: Starting full primary refresh. [2025-11-04T17:26:22.494520] [INFO ] [REMOTE ] AXTA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:26:22.567034] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-01. [2025-11-04T17:26:23.000441] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-11-04T17:26:23.367044] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-02. [2025-11-04T17:26:23.881437] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T17:26:24.167058] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-10. [2025-11-04T17:26:24.428034] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T17:26:24.967109] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-07. [2025-11-04T17:26:25.627124] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T17:26:25.767023] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-03. [2025-11-04T17:26:25.838808] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-11-04T17:26:26.567038] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-02. [2025-11-04T17:26:26.956890] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T17:26:27.367085] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-04. [2025-11-04T17:26:27.729811] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T17:26:28.167032] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-08. [2025-11-04T17:26:28.512772] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T17:26:28.918810] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T17:26:28.967026] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-03. [2025-11-04T17:26:29.767052] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-09. [2025-11-04T17:26:30.133427] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-11-04T17:26:30.567039] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-11. [2025-11-04T17:26:30.716041] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-11-04T17:26:31.367099] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-04. [2025-11-04T17:26:31.665083] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-11 (1.10 seconds). [2025-11-04T17:26:31.670839] [INFO ] [REMOTE ] AXL: Full fetch complete (73386 records). [2025-11-04T17:26:31.671181] [DEBUG] [VALIDATOR ] AXL: Validating primary data. [2025-11-04T17:26:32.201536] [DEBUG] [VALIDATOR ] AXL: All expected trading days are present. [2025-11-04T17:26:32.178145] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-10. [2025-11-04T17:26:32.201750] [DEBUG] [VALIDATOR ] AXL: Primary data validation passed. [2025-11-04T17:26:32.203549] [INFO ] [MAIN ] AXL: Fetched primary data is valid. [2025-11-04T17:26:32.203759] [DEBUG] [LOCAL ] AXL: Saving primary data. [2025-11-04T17:26:32.205296] [DEBUG] [LOCAL ] AXL: Trimmed 73386 records to 50502 (keeping 200 days of data) [2025-11-04T17:26:32.504039] [DEBUG] [LOCAL ] AXL: Primary data saved (50502 records). [2025-11-04T17:26:32.504309] [INFO ] [MAIN ] AXL: Full history saved. [2025-11-04T17:26:32.504473] [INFO ] [MAIN ] Update Progress: 197/2367 symbols completed. AXL took 36.7s. Estimated remaining: 4h 51m 46s. Rate: 7.4 symbols/min [2025-11-04T17:26:32.545264] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T17:26:32.967030] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-05. [2025-11-04T17:26:33.105944] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-11-04T17:26:33.767089] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-11. [2025-11-04T17:26:34.135952] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T17:26:34.509074] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T17:26:34.510573] [INFO ] [REMOTE ] AX: Full fetch complete (41673 records). [2025-11-04T17:26:34.510707] [DEBUG] [VALIDATOR ] AX: Validating primary data. [2025-11-04T17:26:34.573286] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-06. [2025-11-04T17:26:34.894373] [DEBUG] [VALIDATOR ] AX: All expected trading days are present. [2025-11-04T17:26:34.894469] [DEBUG] [VALIDATOR ] AX: Primary data validation passed. [2025-11-04T17:26:34.895095] [INFO ] [MAIN ] AX: Fetched primary data is valid. [2025-11-04T17:26:34.895158] [DEBUG] [LOCAL ] AX: Saving primary data. [2025-11-04T17:26:34.896060] [DEBUG] [LOCAL ] AX: Trimmed 41673 records to 27085 (keeping 200 days of data) [2025-11-04T17:26:35.050479] [DEBUG] [LOCAL ] AX: Primary data saved (27085 records). [2025-11-04T17:26:35.050604] [INFO ] [MAIN ] AX: Full history saved. [2025-11-04T17:26:35.050701] [INFO ] [MAIN ] Update Progress: 198/2367 symbols completed. AX took 49.5s. Estimated remaining: 4h 50m 37s. Rate: 7.5 symbols/min [2025-11-04T17:26:35.367082] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-05. [2025-11-04T17:26:35.770573] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T17:26:36.167072] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-07. [2025-11-04T17:26:36.930155] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-11-04T17:26:36.967081] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-06. [2025-11-04T17:26:37.265483] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T17:26:37.767078] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-08. [2025-11-04T17:26:38.404113] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-11-04T17:26:38.567050] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-07. [2025-11-04T17:26:38.827155] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-11-04T17:26:39.367045] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-09. [2025-11-04T17:26:40.003904] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:26:40.167047] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-08. [2025-11-04T17:26:40.529800] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T17:26:40.967071] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-10. [2025-11-04T17:26:41.565957] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T17:26:41.767016] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-09. [2025-11-04T17:26:42.050719] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-11-04T17:26:42.567126] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-11. [2025-11-04T17:26:43.078600] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T17:26:43.367048] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-10. [2025-11-04T17:26:43.619116] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T17:26:43.621507] [INFO ] [REMOTE ] AXS: Full fetch complete (56613 records). [2025-11-04T17:26:43.621685] [DEBUG] [VALIDATOR ] AXS: Validating primary data. [2025-11-04T17:26:44.013984] [DEBUG] [VALIDATOR ] AXS: All expected trading days are present. [2025-11-04T17:26:44.014062] [DEBUG] [VALIDATOR ] AXS: Primary data validation passed. [2025-11-04T17:26:44.014679] [INFO ] [MAIN ] AXS: Fetched primary data is valid. [2025-11-04T17:26:44.014763] [DEBUG] [LOCAL ] AXS: Saving primary data. [2025-11-04T17:26:44.015994] [DEBUG] [LOCAL ] AXS: Trimmed 56613 records to 39080 (keeping 200 days of data) [2025-11-04T17:26:44.256468] [DEBUG] [LOCAL ] AXS: Primary data saved (39080 records). [2025-11-04T17:26:44.256616] [INFO ] [MAIN ] AXS: Full history saved. [2025-11-04T17:26:44.257288] [INFO ] [MAIN ] Update Progress: 199/2367 symbols completed. AXS took 26.4s. Estimated remaining: 4h 50m 42s. Rate: 7.5 symbols/min [2025-11-04T17:26:44.804126] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T17:26:44.817009] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-11. [2025-11-04T17:26:46.047692] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T17:26:46.049691] [INFO ] [REMOTE ] AXTA: Full fetch complete (75692 records). [2025-11-04T17:26:46.049816] [DEBUG] [VALIDATOR ] AXTA: Validating primary data. [2025-11-04T17:26:46.446523] [DEBUG] [VALIDATOR ] AXTA: All expected trading days are present. [2025-11-04T17:26:46.446622] [DEBUG] [VALIDATOR ] AXTA: Primary data validation passed. [2025-11-04T17:26:46.447478] [INFO ] [MAIN ] AXTA: Fetched primary data is valid. [2025-11-04T17:26:46.447544] [DEBUG] [LOCAL ] AXTA: Saving primary data. [2025-11-04T17:26:46.449051] [DEBUG] [LOCAL ] AXTA: Trimmed 75692 records to 52382 (keeping 200 days of data) [2025-11-04T17:26:46.745715] [DEBUG] [LOCAL ] AXTA: Primary data saved (52382 records). [2025-11-04T17:26:46.745841] [INFO ] [MAIN ] AXTA: Full history saved. [2025-11-04T17:26:46.746985] [INFO ] [MAIN ] Update Progress: 200/2367 symbols completed. AXTA took 25.0s. Estimated remaining: 4h 49m 33s. Rate: 7.5 symbols/min [2025-11-04T17:26:47.457750] [INFO ] [MAIN ] AZO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:26:47.460705] [INFO ] [MAIN ] AYI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:26:47.437474] [INFO ] [MAIN ] B: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:26:47.461383] [DEBUG] [LOCAL ] B: No existing data file found in primary storage. [2025-11-04T17:26:47.462223] [INFO ] [MAIN ] AZZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:26:47.462460] [WARN ] [MAIN ] B: No existing data found, performing full refresh instead. [2025-11-04T17:26:47.463286] [INFO ] [MAIN ] B: Wiping existing primary data before full refresh. [2025-11-04T17:26:47.463334] [INFO ] [LOCAL ] B: Starting deletion of primary data. [2025-11-04T17:26:47.463471] [DEBUG] [LOCAL ] B: No primary data file found to delete. [2025-11-04T17:26:47.463634] [INFO ] [MAIN ] B: Starting full primary refresh. [2025-11-04T17:26:47.576943] [DEBUG] [LOCAL ] AZO: Loaded 10956 existing records from primary storage. [2025-11-04T17:26:47.601268] [INFO ] [MAIN ] AZO: Loaded 10956 existing records. [2025-11-04T17:26:47.702061] [DEBUG] [LOCAL ] AZZ: Loaded 22960 existing records from primary storage. [2025-11-04T17:26:47.694437] [DEBUG] [LOCAL ] AYI: Loaded 23197 existing records from primary storage. [2025-11-04T17:26:47.757175] [INFO ] [MAIN ] AZZ: Loaded 22960 existing records. [2025-11-04T17:26:47.757066] [INFO ] [MAIN ] AYI: Loaded 23197 existing records. [2025-11-04T17:26:48.229093] [WARN ] [MAIN ] AZO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:26:48.302022] [INFO ] [MAIN ] AZO: Wiping existing primary data before full refresh. [2025-11-04T17:26:48.342953] [INFO ] [LOCAL ] AZO: Starting deletion of primary data. [2025-11-04T17:26:48.421909] [INFO ] [REMOTE ] B: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:26:48.437514] [INFO ] [REMOTE ] B: Fetching primary data for 2025-01. [2025-11-04T17:26:48.416361] [WARN ] [MAIN ] AZZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:26:48.443188] [WARN ] [MAIN ] AYI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:26:48.443440] [DEBUG] [LOCAL ] AZO: Deleted primary data file. [2025-11-04T17:26:48.443545] [INFO ] [MAIN ] AZO: Starting full primary refresh. [2025-11-04T17:26:48.443352] [INFO ] [MAIN ] AZZ: Wiping existing primary data before full refresh. [2025-11-04T17:26:48.466423] [INFO ] [LOCAL ] AZZ: Starting deletion of primary data. [2025-11-04T17:26:48.444483] [INFO ] [MAIN ] AYI: Wiping existing primary data before full refresh. [2025-11-04T17:26:48.476272] [INFO ] [LOCAL ] AYI: Starting deletion of primary data. [2025-11-04T17:26:48.497890] [DEBUG] [LOCAL ] AYI: Deleted primary data file. [2025-11-04T17:26:48.503914] [INFO ] [MAIN ] AYI: Starting full primary refresh. [2025-11-04T17:26:48.497848] [DEBUG] [LOCAL ] AZZ: Deleted primary data file. [2025-11-04T17:26:48.523561] [INFO ] [MAIN ] AZZ: Starting full primary refresh. [2025-11-04T17:26:49.427895] [INFO ] [REMOTE ] AZO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:26:49.529621] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-01. [2025-11-04T17:26:49.532617] [INFO ] [REMOTE ] AYI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:26:49.587410] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-01. [2025-11-04T17:26:49.624989] [INFO ] [REMOTE ] AZZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:26:49.625570] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-01. [2025-11-04T17:26:49.627397] [INFO ] [REMOTE ] B: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T17:26:49.633762] [INFO ] [REMOTE ] B: Fetching primary data for 2025-02. [2025-11-04T17:26:49.783248] [DEBUG] [REMOTE ] B: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:26:49.783553] [WARN ] [REMOTE ] B: No data returned for month 2025-02 [2025-11-04T17:26:49.783623] [FATAL] [REMOTE ] B: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:26:49.783729] [FATAL] [MAIN ] B: Failed to fetch primary data. [2025-11-04T17:26:49.783776] [WARN ] [MAIN ] B: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:26:49.783814] [INFO ] [LOCAL ] B: Starting deletion of primary data. [2025-11-04T17:26:49.783866] [DEBUG] [LOCAL ] B: No primary data file found to delete. [2025-11-04T17:26:49.783900] [INFO ] [LOCAL ] B: Starting deletion of derivative data. [2025-11-04T17:26:49.783939] [DEBUG] [LOCAL ] B: No derivative data file found to delete. [2025-11-04T17:26:49.783983] [INFO ] [MAIN ] Update Progress: 201/2367 symbols completed. B took 3.0s. Estimated remaining: 4h 48m 32s. Rate: 7.5 symbols/min [2025-11-04T17:26:49.954102] [INFO ] [MAIN ] BA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:26:50.007790] [DEBUG] [LOCAL ] BA: Loaded 53790 existing records from primary storage. [2025-11-04T17:26:50.008141] [INFO ] [MAIN ] BA: Loaded 53790 existing records. [2025-11-04T17:26:50.188220] [WARN ] [MAIN ] BA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:26:50.188431] [INFO ] [MAIN ] BA: Wiping existing primary data before full refresh. [2025-11-04T17:26:50.188482] [INFO ] [LOCAL ] BA: Starting deletion of primary data. [2025-11-04T17:26:50.188705] [DEBUG] [LOCAL ] BA: Deleted primary data file. [2025-11-04T17:26:50.188762] [INFO ] [MAIN ] BA: Starting full primary refresh. [2025-11-04T17:26:50.459197] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-11-04T17:26:50.484156] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-02. [2025-11-04T17:26:50.569451] [INFO ] [REMOTE ] BA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:26:50.569849] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-01. [2025-11-04T17:26:50.583337] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T17:26:50.587465] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-02. [2025-11-04T17:26:50.649508] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T17:26:51.044922] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-11-04T17:26:51.303510] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T17:26:51.367054] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-03. [2025-11-04T17:26:51.968798] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-11-04T17:26:52.167090] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-04. [2025-11-04T17:26:52.617529] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-11-04T17:26:52.967058] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-02. [2025-11-04T17:26:53.030161] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T17:26:53.767072] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-03. [2025-11-04T17:26:54.477966] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T17:26:54.575315] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-02. [2025-11-04T17:26:54.593513] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-11-04T17:26:55.277763] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T17:26:55.367046] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-03. [2025-11-04T17:26:56.167018] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-05. [2025-11-04T17:26:56.937542] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-11-04T17:26:56.967008] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-06. [2025-11-04T17:26:57.048599] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-11-04T17:26:57.687077] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-11-04T17:26:57.767026] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-03. [2025-11-04T17:26:58.567097] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-04. [2025-11-04T17:26:58.793075] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T17:26:59.286513] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-11-04T17:26:59.367032] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-05. [2025-11-04T17:27:00.108593] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-11-04T17:27:00.167104] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-06. [2025-11-04T17:27:00.967011] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-04. [2025-11-04T17:27:01.003215] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T17:27:01.767082] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-04. [2025-11-04T17:27:02.567064] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-07. [2025-11-04T17:27:02.856872] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-11-04T17:27:02.900253] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T17:27:03.367040] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-07. [2025-11-04T17:27:03.631040] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-11-04T17:27:03.908338] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-11-04T17:27:04.167075] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-08. [2025-11-04T17:27:04.800867] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-11-04T17:27:04.967130] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-09. [2025-11-04T17:27:05.618921] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-11-04T17:27:05.767062] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-05. [2025-11-04T17:27:06.567083] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-05. [2025-11-04T17:27:07.367071] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-10. [2025-11-04T17:27:07.452945] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T17:27:07.675325] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-11-04T17:27:08.107345] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-11-04T17:27:08.166997] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-06. [2025-11-04T17:27:08.967041] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-08. [2025-11-04T17:27:09.014753] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T17:27:09.767025] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-06. [2025-11-04T17:27:09.848302] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-11-04T17:27:10.567088] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-09. [2025-11-04T17:27:11.264723] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-11-04T17:27:11.367014] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-10. [2025-11-04T17:27:11.473806] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-11-04T17:27:12.167028] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-11. [2025-11-04T17:27:12.233424] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-11-04T17:27:12.967091] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-07. [2025-11-04T17:27:13.246703] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-11 (1.08 seconds). [2025-11-04T17:27:13.248430] [INFO ] [REMOTE ] AZO: Full fetch complete (16691 records). [2025-11-04T17:27:13.248518] [DEBUG] [VALIDATOR ] AZO: Validating primary data. [2025-11-04T17:27:13.609684] [DEBUG] [VALIDATOR ] AZO: All expected trading days are present. [2025-11-04T17:27:13.609796] [DEBUG] [VALIDATOR ] AZO: Primary data validation passed. [2025-11-04T17:27:13.610142] [INFO ] [MAIN ] AZO: Fetched primary data is valid. [2025-11-04T17:27:13.610200] [DEBUG] [LOCAL ] AZO: Saving primary data. [2025-11-04T17:27:13.610895] [DEBUG] [LOCAL ] AZO: Trimmed 16691 records to 11276 (keeping 200 days of data) [2025-11-04T17:27:13.681467] [DEBUG] [LOCAL ] AZO: Primary data saved (11276 records). [2025-11-04T17:27:13.681589] [INFO ] [MAIN ] AZO: Full history saved. [2025-11-04T17:27:13.681690] [INFO ] [MAIN ] Update Progress: 202/2367 symbols completed. AZO took 26.9s. Estimated remaining: 4h 51m 14s. Rate: 7.4 symbols/min [2025-11-04T17:27:13.772705] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-07. [2025-11-04T17:27:13.853048] [INFO ] [MAIN ] BABA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:27:13.853231] [DEBUG] [LOCAL ] BABA: No existing data file found in primary storage. [2025-11-04T17:27:13.853275] [WARN ] [MAIN ] BABA: No existing data found, performing full refresh instead. [2025-11-04T17:27:13.853379] [INFO ] [MAIN ] BABA: Wiping existing primary data before full refresh. [2025-11-04T17:27:13.853419] [INFO ] [LOCAL ] BABA: Starting deletion of primary data. [2025-11-04T17:27:13.853458] [DEBUG] [LOCAL ] BABA: No primary data file found to delete. [2025-11-04T17:27:13.853488] [INFO ] [MAIN ] BABA: Starting full primary refresh. [2025-11-04T17:27:14.196943] [INFO ] [REMOTE ] BABA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:27:14.567030] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-01. [2025-11-04T17:27:14.848894] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T17:27:14.901934] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-11-04T17:27:15.367042] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-08. [2025-11-04T17:27:16.167022] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-11. [2025-11-04T17:27:16.250246] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-11-04T17:27:16.479510] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-11-04T17:27:16.967133] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-02. [2025-11-04T17:27:17.539758] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-11 (1.37 seconds). [2025-11-04T17:27:17.541020] [INFO ] [REMOTE ] AZZ: Full fetch complete (32253 records). [2025-11-04T17:27:17.541097] [DEBUG] [VALIDATOR ] AZZ: Validating primary data. [2025-11-04T17:27:17.772826] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-09. [2025-11-04T17:27:17.941832] [DEBUG] [VALIDATOR ] AZZ: All expected trading days are present. [2025-11-04T17:27:17.941908] [DEBUG] [VALIDATOR ] AZZ: Primary data validation passed. [2025-11-04T17:27:17.942328] [INFO ] [MAIN ] AZZ: Fetched primary data is valid. [2025-11-04T17:27:17.942366] [DEBUG] [LOCAL ] AZZ: Saving primary data. [2025-11-04T17:27:17.943087] [DEBUG] [LOCAL ] AZZ: Trimmed 32253 records to 22943 (keeping 200 days of data) [2025-11-04T17:27:18.088857] [DEBUG] [LOCAL ] AZZ: Primary data saved (22943 records). [2025-11-04T17:27:18.088994] [INFO ] [MAIN ] AZZ: Full history saved. [2025-11-04T17:27:18.089073] [INFO ] [MAIN ] Update Progress: 203/2367 symbols completed. AZZ took 31.3s. Estimated remaining: 4h 50m 27s. Rate: 7.5 symbols/min [2025-11-04T17:27:18.257404] [INFO ] [MAIN ] BAC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:27:18.311413] [DEBUG] [LOCAL ] BAC: Loaded 53808 existing records from primary storage. [2025-11-04T17:27:18.311778] [INFO ] [MAIN ] BAC: Loaded 53808 existing records. [2025-11-04T17:27:18.489777] [WARN ] [MAIN ] BAC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:27:18.489976] [INFO ] [MAIN ] BAC: Wiping existing primary data before full refresh. [2025-11-04T17:27:18.490022] [INFO ] [LOCAL ] BAC: Starting deletion of primary data. [2025-11-04T17:27:18.490204] [DEBUG] [LOCAL ] BAC: Deleted primary data file. [2025-11-04T17:27:18.490241] [INFO ] [MAIN ] BAC: Starting full primary refresh. [2025-11-04T17:27:18.574407] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-08. [2025-11-04T17:27:18.786564] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T17:27:18.868415] [INFO ] [REMOTE ] BAC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:27:19.336037] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-02 (2.37 seconds). [2025-11-04T17:27:19.367003] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-03. [2025-11-04T17:27:20.167045] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-01. [2025-11-04T17:27:20.813587] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-08 (2.24 seconds). [2025-11-04T17:27:20.967038] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-09. [2025-11-04T17:27:21.380282] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-11-04T17:27:21.767014] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-10. [2025-11-04T17:27:22.421208] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-01 (2.25 seconds). [2025-11-04T17:27:22.534027] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-11-04T17:27:22.567025] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-11. [2025-11-04T17:27:22.932277] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-11-04T17:27:23.318547] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-11 (0.75 seconds). [2025-11-04T17:27:23.319919] [INFO ] [REMOTE ] AYI: Full fetch complete (35891 records). [2025-11-04T17:27:23.320003] [DEBUG] [VALIDATOR ] AYI: Validating primary data. [2025-11-04T17:27:23.367085] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-04. [2025-11-04T17:27:23.685891] [DEBUG] [VALIDATOR ] AYI: All expected trading days are present. [2025-11-04T17:27:23.685962] [DEBUG] [VALIDATOR ] AYI: Primary data validation passed. [2025-11-04T17:27:23.686363] [INFO ] [MAIN ] AYI: Fetched primary data is valid. [2025-11-04T17:27:23.686400] [DEBUG] [LOCAL ] AYI: Saving primary data. [2025-11-04T17:27:23.687175] [DEBUG] [LOCAL ] AYI: Trimmed 35891 records to 23262 (keeping 200 days of data) [2025-11-04T17:27:23.826086] [DEBUG] [LOCAL ] AYI: Primary data saved (23262 records). [2025-11-04T17:27:23.826210] [INFO ] [MAIN ] AYI: Full history saved. [2025-11-04T17:27:23.826283] [INFO ] [MAIN ] Update Progress: 204/2367 symbols completed. AYI took 37.1s. Estimated remaining: 4h 49m 54s. Rate: 7.5 symbols/min [2025-11-04T17:27:23.991190] [INFO ] [MAIN ] BAH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:27:24.039096] [DEBUG] [LOCAL ] BAH: Loaded 51681 existing records from primary storage. [2025-11-04T17:27:24.039444] [INFO ] [MAIN ] BAH: Loaded 51681 existing records. [2025-11-04T17:27:24.176468] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-10. [2025-11-04T17:27:24.215250] [WARN ] [MAIN ] BAH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:27:24.215518] [INFO ] [MAIN ] BAH: Wiping existing primary data before full refresh. [2025-11-04T17:27:24.215564] [INFO ] [LOCAL ] BAH: Starting deletion of primary data. [2025-11-04T17:27:24.215772] [DEBUG] [LOCAL ] BAH: Deleted primary data file. [2025-11-04T17:27:24.215827] [INFO ] [MAIN ] BAH: Starting full primary refresh. [2025-11-04T17:27:24.565142] [INFO ] [REMOTE ] BAH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:27:24.967029] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-02. [2025-11-04T17:27:25.545439] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-11-04T17:27:25.767053] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-05. [2025-11-04T17:27:26.578743] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-01. [2025-11-04T17:27:26.591201] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-11-04T17:27:26.635688] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-10 (2.46 seconds). [2025-11-04T17:27:27.367032] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-11. [2025-11-04T17:27:27.775896] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T17:27:27.777954] [INFO ] [REMOTE ] BA: Full fetch complete (77969 records). [2025-11-04T17:27:27.778072] [DEBUG] [VALIDATOR ] BA: Validating primary data. [2025-11-04T17:27:28.201715] [DEBUG] [VALIDATOR ] BA: All expected trading days are present. [2025-11-04T17:27:28.201880] [DEBUG] [VALIDATOR ] BA: Primary data validation passed. [2025-11-04T17:27:28.179680] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-03. [2025-11-04T17:27:28.203887] [INFO ] [MAIN ] BA: Fetched primary data is valid. [2025-11-04T17:27:28.204413] [DEBUG] [LOCAL ] BA: Saving primary data. [2025-11-04T17:27:28.222491] [DEBUG] [LOCAL ] BA: Trimmed 77969 records to 53790 (keeping 200 days of data) [2025-11-04T17:27:28.236264] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-11-04T17:27:28.389764] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-05 (2.62 seconds). [2025-11-04T17:27:28.609568] [DEBUG] [LOCAL ] BA: Primary data saved (53790 records). [2025-11-04T17:27:28.609728] [INFO ] [MAIN ] BA: Full history saved. [2025-11-04T17:27:28.610466] [INFO ] [MAIN ] Update Progress: 205/2367 symbols completed. BA took 38.8s. Estimated remaining: 4h 49m 12s. Rate: 7.5 symbols/min [2025-11-04T17:27:28.773491] [INFO ] [MAIN ] BAK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:27:28.811845] [DEBUG] [LOCAL ] BAK: Loaded 39069 existing records from primary storage. [2025-11-04T17:27:28.812118] [INFO ] [MAIN ] BAK: Loaded 39069 existing records. [2025-11-04T17:27:28.984784] [WARN ] [MAIN ] BAK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:27:28.980122] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-02. [2025-11-04T17:27:28.985665] [INFO ] [MAIN ] BAK: Wiping existing primary data before full refresh. [2025-11-04T17:27:28.985741] [INFO ] [LOCAL ] BAK: Starting deletion of primary data. [2025-11-04T17:27:28.985928] [DEBUG] [LOCAL ] BAK: Deleted primary data file. [2025-11-04T17:27:28.985983] [INFO ] [MAIN ] BAK: Starting full primary refresh. [2025-11-04T17:27:29.330091] [INFO ] [REMOTE ] BAK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:27:29.767121] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-01. [2025-11-04T17:27:30.032850] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-11-04T17:27:30.352439] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-11-04T17:27:30.567097] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-06. [2025-11-04T17:27:30.919949] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T17:27:31.367050] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-04. [2025-11-04T17:27:32.167103] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-03. [2025-11-04T17:27:32.453758] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-11-04T17:27:32.967044] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-07. [2025-11-04T17:27:33.510244] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-11-04T17:27:33.569086] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T17:27:33.767049] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-04. [2025-11-04T17:27:34.567089] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-02. [2025-11-04T17:27:35.014527] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-07 (2.05 seconds). [2025-11-04T17:27:35.334052] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-11-04T17:27:35.367037] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-05. [2025-11-04T17:27:35.673999] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T17:27:36.167086] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-05. [2025-11-04T17:27:36.851422] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T17:27:36.967090] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-06. [2025-11-04T17:27:37.767170] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-03. [2025-11-04T17:27:38.234381] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-11-04T17:27:38.492957] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-11-04T17:27:38.567065] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-06. [2025-11-04T17:27:38.800733] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T17:27:39.367131] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-08. [2025-11-04T17:27:40.167073] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-04. [2025-11-04T17:27:40.911259] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-06 (2.34 seconds). [2025-11-04T17:27:40.967070] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-07. [2025-11-04T17:27:41.768050] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-07. [2025-11-04T17:27:41.775169] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-11-04T17:27:41.821874] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-08 (2.45 seconds). [2025-11-04T17:27:42.567080] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-09. [2025-11-04T17:27:43.024836] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-11-04T17:27:43.367032] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-08. [2025-11-04T17:27:43.508765] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-11-04T17:27:44.167052] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-08. [2025-11-04T17:27:44.712926] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-09 (2.15 seconds). [2025-11-04T17:27:44.967045] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-10. [2025-11-04T17:27:45.368353] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T17:27:45.581220] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-11-04T17:27:45.767043] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-09. [2025-11-04T17:27:46.567044] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-09. [2025-11-04T17:27:47.196876] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-10 (2.23 seconds). [2025-11-04T17:27:47.367026] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-05. [2025-11-04T17:27:47.587449] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-11-04T17:27:47.937704] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-11-04T17:27:48.167040] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-11. [2025-11-04T17:27:48.524696] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T17:27:48.967054] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-10. [2025-11-04T17:27:49.767107] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-10. [2025-11-04T17:27:50.060059] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-11 (1.89 seconds). [2025-11-04T17:27:50.062465] [INFO ] [REMOTE ] BABA: Full fetch complete (78000 records). [2025-11-04T17:27:50.062589] [DEBUG] [VALIDATOR ] BABA: Validating primary data. [2025-11-04T17:27:50.476293] [DEBUG] [VALIDATOR ] BABA: All expected trading days are present. [2025-11-04T17:27:50.476400] [DEBUG] [VALIDATOR ] BABA: Primary data validation passed. [2025-11-04T17:27:50.477340] [INFO ] [MAIN ] BABA: Fetched primary data is valid. [2025-11-04T17:27:50.477437] [DEBUG] [LOCAL ] BABA: Saving primary data. [2025-11-04T17:27:50.479023] [DEBUG] [LOCAL ] BABA: Trimmed 78000 records to 53820 (keeping 200 days of data) [2025-11-04T17:27:50.575358] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-06. [2025-11-04T17:27:50.637081] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-11-04T17:27:50.856770] [DEBUG] [LOCAL ] BABA: Primary data saved (53820 records). [2025-11-04T17:27:50.856928] [INFO ] [MAIN ] BABA: Full history saved. [2025-11-04T17:27:50.857007] [INFO ] [MAIN ] Update Progress: 206/2367 symbols completed. BABA took 37.2s. Estimated remaining: 4h 51m 33s. Rate: 7.4 symbols/min [2025-11-04T17:27:51.026988] [INFO ] [MAIN ] BALL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:27:51.074274] [DEBUG] [LOCAL ] BALL: Loaded 51934 existing records from primary storage. [2025-11-04T17:27:51.074563] [INFO ] [MAIN ] BALL: Loaded 51934 existing records. [2025-11-04T17:27:51.251303] [WARN ] [MAIN ] BALL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:27:51.251538] [INFO ] [MAIN ] BALL: Wiping existing primary data before full refresh. [2025-11-04T17:27:51.251599] [INFO ] [LOCAL ] BALL: Starting deletion of primary data. [2025-11-04T17:27:51.251848] [DEBUG] [LOCAL ] BALL: Deleted primary data file. [2025-11-04T17:27:51.251905] [INFO ] [MAIN ] BALL: Starting full primary refresh. [2025-11-04T17:27:51.372113] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-11. [2025-11-04T17:27:51.635283] [INFO ] [REMOTE ] BALL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:27:51.663147] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T17:27:51.755767] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-11-04T17:27:52.167082] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-07. [2025-11-04T17:27:52.944740] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-11 (1.57 seconds). [2025-11-04T17:27:52.946727] [INFO ] [REMOTE ] BAH: Full fetch complete (75399 records). [2025-11-04T17:27:52.946825] [DEBUG] [VALIDATOR ] BAH: Validating primary data. [2025-11-04T17:27:52.977625] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-01. [2025-11-04T17:27:53.338912] [DEBUG] [VALIDATOR ] BAH: All expected trading days are present. [2025-11-04T17:27:53.339083] [DEBUG] [VALIDATOR ] BAH: Primary data validation passed. [2025-11-04T17:27:53.339867] [INFO ] [MAIN ] BAH: Fetched primary data is valid. [2025-11-04T17:27:53.339941] [DEBUG] [LOCAL ] BAH: Saving primary data. [2025-11-04T17:27:53.341338] [DEBUG] [LOCAL ] BAH: Trimmed 75399 records to 51706 (keeping 200 days of data) [2025-11-04T17:27:53.468938] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:27:53.683913] [DEBUG] [LOCAL ] BAH: Primary data saved (51706 records). [2025-11-04T17:27:53.684049] [INFO ] [MAIN ] BAH: Full history saved. [2025-11-04T17:27:53.684140] [INFO ] [MAIN ] Update Progress: 207/2367 symbols completed. BAH took 29.9s. Estimated remaining: 4h 50m 30s. Rate: 7.4 symbols/min [2025-11-04T17:27:53.772366] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-08. [2025-11-04T17:27:53.850217] [INFO ] [MAIN ] BALY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:27:53.850365] [DEBUG] [LOCAL ] BALY: No existing data file found in primary storage. [2025-11-04T17:27:53.850400] [WARN ] [MAIN ] BALY: No existing data found, performing full refresh instead. [2025-11-04T17:27:53.850506] [INFO ] [MAIN ] BALY: Wiping existing primary data before full refresh. [2025-11-04T17:27:53.850549] [INFO ] [LOCAL ] BALY: Starting deletion of primary data. [2025-11-04T17:27:53.850588] [DEBUG] [LOCAL ] BALY: No primary data file found to delete. [2025-11-04T17:27:53.850618] [INFO ] [MAIN ] BALY: Starting full primary refresh. [2025-11-04T17:27:54.196954] [INFO ] [REMOTE ] BALY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:27:54.575725] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-01. [2025-11-04T17:27:54.580815] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-11-04T17:27:55.107504] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T17:27:55.367098] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-11. [2025-11-04T17:27:55.512021] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-11-04T17:27:55.701393] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-11 (0.33 seconds). [2025-11-04T17:27:55.703431] [INFO ] [REMOTE ] BAC: Full fetch complete (77988 records). [2025-11-04T17:27:55.703549] [DEBUG] [VALIDATOR ] BAC: Validating primary data. [2025-11-04T17:27:56.090032] [DEBUG] [VALIDATOR ] BAC: All expected trading days are present. [2025-11-04T17:27:56.090355] [DEBUG] [VALIDATOR ] BAC: Primary data validation passed. [2025-11-04T17:27:56.091152] [INFO ] [MAIN ] BAC: Fetched primary data is valid. [2025-11-04T17:27:56.091218] [DEBUG] [LOCAL ] BAC: Saving primary data. [2025-11-04T17:27:56.092548] [DEBUG] [LOCAL ] BAC: Trimmed 77988 records to 53808 (keeping 200 days of data) [2025-11-04T17:27:56.181816] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-02. [2025-11-04T17:27:56.415452] [DEBUG] [LOCAL ] BAC: Primary data saved (53808 records). [2025-11-04T17:27:56.415576] [INFO ] [MAIN ] BAC: Full history saved. [2025-11-04T17:27:56.417141] [INFO ] [MAIN ] Update Progress: 208/2367 symbols completed. BAC took 38.3s. Estimated remaining: 4h 49m 27s. Rate: 7.5 symbols/min [2025-11-04T17:27:56.586997] [INFO ] [MAIN ] BAM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:27:56.639177] [DEBUG] [LOCAL ] BAM: Loaded 52094 existing records from primary storage. [2025-11-04T17:27:56.639479] [INFO ] [MAIN ] BAM: Loaded 52094 existing records. [2025-11-04T17:27:56.748917] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-11-04T17:27:56.825511] [WARN ] [MAIN ] BAM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:27:56.825704] [INFO ] [MAIN ] BAM: Wiping existing primary data before full refresh. [2025-11-04T17:27:56.825772] [INFO ] [LOCAL ] BAM: Starting deletion of primary data. [2025-11-04T17:27:56.826306] [DEBUG] [LOCAL ] BAM: Deleted primary data file. [2025-11-04T17:27:56.826369] [INFO ] [MAIN ] BAM: Starting full primary refresh. [2025-11-04T17:27:56.973696] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-02. [2025-11-04T17:27:57.178707] [INFO ] [REMOTE ] BAM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:27:57.767033] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-09. [2025-11-04T17:27:58.240695] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T17:27:58.567022] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-03. [2025-11-04T17:27:58.612082] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-11-04T17:27:59.079891] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-11-04T17:27:59.367049] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-01. [2025-11-04T17:28:00.167038] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-04. [2025-11-04T17:28:00.602056] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-11-04T17:28:00.767906] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T17:28:00.967011] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-02. [2025-11-04T17:28:01.767077] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-10. [2025-11-04T17:28:02.395567] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-11-04T17:28:02.567077] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-03. [2025-11-04T17:28:02.947431] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T17:28:03.367036] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-11. [2025-11-04T17:28:03.951767] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T17:28:04.167056] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-04. [2025-11-04T17:28:04.250382] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-11 (0.88 seconds). [2025-11-04T17:28:04.252005] [INFO ] [REMOTE ] BAK: Full fetch complete (56437 records). [2025-11-04T17:28:04.252117] [DEBUG] [VALIDATOR ] BAK: Validating primary data. [2025-11-04T17:28:04.253023] [FATAL] [VALIDATOR ] BAK: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:28:04.253113] [FATAL] [MAIN ] BAK: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:28:04.253170] [WARN ] [MAIN ] BAK: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:28:04.253230] [INFO ] [LOCAL ] BAK: Starting deletion of primary data. [2025-11-04T17:28:04.253307] [DEBUG] [LOCAL ] BAK: No primary data file found to delete. [2025-11-04T17:28:04.253367] [INFO ] [LOCAL ] BAK: Starting deletion of derivative data. [2025-11-04T17:28:04.253551] [DEBUG] [LOCAL ] BAK: Deleted derivative data file. [2025-11-04T17:28:04.254089] [INFO ] [MAIN ] Update Progress: 209/2367 symbols completed. BAK took 35.6s. Estimated remaining: 4h 49m 17s. Rate: 7.5 symbols/min [2025-11-04T17:28:04.425696] [INFO ] [MAIN ] BANC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:28:04.476831] [DEBUG] [LOCAL ] BANC: Loaded 51372 existing records from primary storage. [2025-11-04T17:28:04.477161] [INFO ] [MAIN ] BANC: Loaded 51372 existing records. [2025-11-04T17:28:04.649672] [WARN ] [MAIN ] BANC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:28:04.649883] [INFO ] [MAIN ] BANC: Wiping existing primary data before full refresh. [2025-11-04T17:28:04.649937] [INFO ] [LOCAL ] BANC: Starting deletion of primary data. [2025-11-04T17:28:04.650112] [DEBUG] [LOCAL ] BANC: Deleted primary data file. [2025-11-04T17:28:04.650159] [INFO ] [MAIN ] BANC: Starting full primary refresh. [2025-11-04T17:28:04.976784] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-03. [2025-11-04T17:28:05.006801] [INFO ] [REMOTE ] BANC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:28:05.434352] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T17:28:05.767044] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-01. [2025-11-04T17:28:06.501436] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T17:28:06.567020] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-04. [2025-11-04T17:28:07.324138] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-11-04T17:28:07.367003] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-02. [2025-11-04T17:28:07.877337] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T17:28:08.167069] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-05. [2025-11-04T17:28:08.515945] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T17:28:08.967047] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-05. [2025-11-04T17:28:09.702452] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T17:28:09.767040] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-06. [2025-11-04T17:28:10.423294] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T17:28:10.567036] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-05. [2025-11-04T17:28:11.036999] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T17:28:11.168379] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-11-04T17:28:11.367050] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-06. [2025-11-04T17:28:12.167069] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-07. [2025-11-04T17:28:12.668194] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T17:28:12.967074] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-03. [2025-11-04T17:28:13.464208] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:28:13.767034] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-08. [2025-11-04T17:28:14.316066] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-11-04T17:28:14.567037] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-04. [2025-11-04T17:28:15.090169] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-11-04T17:28:15.367043] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-06. [2025-11-04T17:28:15.840480] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T17:28:15.956523] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-11-04T17:28:16.167040] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-07. [2025-11-04T17:28:16.967073] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-09. [2025-11-04T17:28:17.573531] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T17:28:17.767042] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-08. [2025-11-04T17:28:18.272318] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T17:28:18.567070] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-05. [2025-11-04T17:28:18.991373] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T17:28:19.367022] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-09. [2025-11-04T17:28:19.847510] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T17:28:20.167105] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-10. [2025-11-04T17:28:20.620167] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T17:28:20.967059] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-06. [2025-11-04T17:28:21.662906] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-11-04T17:28:21.767076] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-10. [2025-11-04T17:28:22.183238] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-11-04T17:28:22.567039] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-07. [2025-11-04T17:28:23.321932] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-11-04T17:28:23.367009] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-11. [2025-11-04T17:28:23.411784] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-11-04T17:28:23.672996] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-11 (0.31 seconds). [2025-11-04T17:28:23.675937] [INFO ] [REMOTE ] BALL: Full fetch complete (75640 records). [2025-11-04T17:28:23.676187] [DEBUG] [VALIDATOR ] BALL: Validating primary data. [2025-11-04T17:28:24.090839] [DEBUG] [VALIDATOR ] BALL: All expected trading days are present. [2025-11-04T17:28:24.090927] [DEBUG] [VALIDATOR ] BALL: Primary data validation passed. [2025-11-04T17:28:24.091684] [INFO ] [MAIN ] BALL: Fetched primary data is valid. [2025-11-04T17:28:24.091749] [DEBUG] [LOCAL ] BALL: Saving primary data. [2025-11-04T17:28:24.093161] [DEBUG] [LOCAL ] BALL: Trimmed 75640 records to 51937 (keeping 200 days of data) [2025-11-04T17:28:24.181689] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-07. [2025-11-04T17:28:24.446758] [DEBUG] [LOCAL ] BALL: Primary data saved (51937 records). [2025-11-04T17:28:24.446959] [INFO ] [MAIN ] BALL: Full history saved. [2025-11-04T17:28:24.448825] [INFO ] [MAIN ] Update Progress: 210/2367 symbols completed. BALL took 33.6s. Estimated remaining: 4h 51m 13s. Rate: 7.4 symbols/min [2025-11-04T17:28:24.657729] [INFO ] [MAIN ] BAP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:28:24.685184] [DEBUG] [LOCAL ] BAP: Loaded 25020 existing records from primary storage. [2025-11-04T17:28:24.685475] [INFO ] [MAIN ] BAP: Loaded 25020 existing records. [2025-11-04T17:28:24.906670] [WARN ] [MAIN ] BAP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:28:24.906915] [INFO ] [MAIN ] BAP: Wiping existing primary data before full refresh. [2025-11-04T17:28:24.906982] [INFO ] [LOCAL ] BAP: Starting deletion of primary data. [2025-11-04T17:28:24.907241] [DEBUG] [LOCAL ] BAP: Deleted primary data file. [2025-11-04T17:28:24.907319] [INFO ] [MAIN ] BAP: Starting full primary refresh. [2025-11-04T17:28:24.973304] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-11. [2025-11-04T17:28:25.356446] [INFO ] [REMOTE ] BAP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:28:25.551337] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T17:28:25.767057] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-08. [2025-11-04T17:28:26.227677] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T17:28:26.230207] [INFO ] [REMOTE ] BAM: Full fetch complete (75512 records). [2025-11-04T17:28:26.230384] [DEBUG] [VALIDATOR ] BAM: Validating primary data. [2025-11-04T17:28:26.572479] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-08. [2025-11-04T17:28:26.829820] [DEBUG] [VALIDATOR ] BAM: All expected trading days are present. [2025-11-04T17:28:26.830043] [DEBUG] [VALIDATOR ] BAM: Primary data validation passed. [2025-11-04T17:28:26.831307] [INFO ] [MAIN ] BAM: Fetched primary data is valid. [2025-11-04T17:28:26.831406] [DEBUG] [LOCAL ] BAM: Saving primary data. [2025-11-04T17:28:26.833524] [DEBUG] [LOCAL ] BAM: Trimmed 75512 records to 52101 (keeping 200 days of data) [2025-11-04T17:28:27.158995] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-11-04T17:28:27.256436] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T17:28:27.318440] [DEBUG] [LOCAL ] BAM: Primary data saved (52101 records). [2025-11-04T17:28:27.318609] [INFO ] [MAIN ] BAM: Full history saved. [2025-11-04T17:28:27.319882] [INFO ] [MAIN ] Update Progress: 211/2367 symbols completed. BAM took 30.9s. Estimated remaining: 4h 50m 12s. Rate: 7.4 symbols/min [2025-11-04T17:28:27.372120] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-01. [2025-11-04T17:28:27.526443] [INFO ] [MAIN ] BARK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:28:27.526606] [DEBUG] [LOCAL ] BARK: No existing data file found in primary storage. [2025-11-04T17:28:27.526668] [WARN ] [MAIN ] BARK: No existing data found, performing full refresh instead. [2025-11-04T17:28:27.526789] [INFO ] [MAIN ] BARK: Wiping existing primary data before full refresh. [2025-11-04T17:28:27.526832] [INFO ] [LOCAL ] BARK: Starting deletion of primary data. [2025-11-04T17:28:27.526914] [DEBUG] [LOCAL ] BARK: No primary data file found to delete. [2025-11-04T17:28:27.526952] [INFO ] [MAIN ] BARK: Starting full primary refresh. [2025-11-04T17:28:27.961754] [INFO ] [REMOTE ] BARK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:28:28.167042] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-01. [2025-11-04T17:28:28.438094] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-11-04T17:28:28.967038] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-09. [2025-11-04T17:28:29.341135] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T17:28:29.472891] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-11-04T17:28:29.767070] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-02. [2025-11-04T17:28:30.567063] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-02. [2025-11-04T17:28:30.807195] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-11-04T17:28:31.367096] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-10. [2025-11-04T17:28:31.519534] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-11-04T17:28:32.112692] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-11-04T17:28:32.167085] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-03. [2025-11-04T17:28:32.967012] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-11. [2025-11-04T17:28:33.122582] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T17:28:33.767029] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-04. [2025-11-04T17:28:33.789085] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-11 (0.82 seconds). [2025-11-04T17:28:33.790137] [INFO ] [REMOTE ] BALY: Full fetch complete (15466 records). [2025-11-04T17:28:33.790204] [DEBUG] [VALIDATOR ] BALY: Validating primary data. [2025-11-04T17:28:33.975193] [WARN ] [VALIDATOR ] BALY: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-11-04T17:28:33.975525] [FATAL] [MAIN ] BALY: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:28:33.975596] [WARN ] [MAIN ] BALY: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:28:33.975693] [INFO ] [LOCAL ] BALY: Starting deletion of primary data. [2025-11-04T17:28:33.975769] [DEBUG] [LOCAL ] BALY: No primary data file found to delete. [2025-11-04T17:28:33.975805] [INFO ] [LOCAL ] BALY: Starting deletion of derivative data. [2025-11-04T17:28:33.975843] [DEBUG] [LOCAL ] BALY: No derivative data file found to delete. [2025-11-04T17:28:33.975908] [INFO ] [MAIN ] Update Progress: 212/2367 symbols completed. BALY took 40.3s. Estimated remaining: 4h 49m 49s. Rate: 7.4 symbols/min [2025-11-04T17:28:34.142902] [INFO ] [MAIN ] BAX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:28:34.198291] [DEBUG] [LOCAL ] BAX: Loaded 53540 existing records from primary storage. [2025-11-04T17:28:34.198606] [INFO ] [MAIN ] BAX: Loaded 53540 existing records. [2025-11-04T17:28:34.376225] [WARN ] [MAIN ] BAX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:28:34.376443] [INFO ] [MAIN ] BAX: Wiping existing primary data before full refresh. [2025-11-04T17:28:34.376490] [INFO ] [LOCAL ] BAX: Starting deletion of primary data. [2025-11-04T17:28:34.376762] [DEBUG] [LOCAL ] BAX: Deleted primary data file. [2025-11-04T17:28:34.376819] [INFO ] [MAIN ] BAX: Starting full primary refresh. [2025-11-04T17:28:34.573980] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-03. [2025-11-04T17:28:34.750022] [INFO ] [REMOTE ] BAX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:28:34.772723] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-11-04T17:28:35.367055] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-09. [2025-11-04T17:28:35.724225] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T17:28:36.167054] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-04. [2025-11-04T17:28:36.780021] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-11-04T17:28:36.967075] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-05. [2025-11-04T17:28:37.143944] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-11-04T17:28:37.767059] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-01. [2025-11-04T17:28:37.885506] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T17:28:38.567087] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-05. [2025-11-04T17:28:39.222892] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-11-04T17:28:39.367064] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-02. [2025-11-04T17:28:39.469043] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T17:28:40.167114] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-06. [2025-11-04T17:28:40.800861] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-11-04T17:28:40.967021] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-03. [2025-11-04T17:28:41.349088] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T17:28:41.767047] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-07. [2025-11-04T17:28:42.282002] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T17:28:42.567087] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-10. [2025-11-04T17:28:43.262070] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-11-04T17:28:43.367058] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-08. [2025-11-04T17:28:43.968727] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T17:28:44.167082] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-11. [2025-11-04T17:28:44.343296] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-11-04T17:28:44.967037] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-06. [2025-11-04T17:28:45.540312] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-11 (1.37 seconds). [2025-11-04T17:28:45.542250] [INFO ] [REMOTE ] BANC: Full fetch complete (74664 records). [2025-11-04T17:28:45.542343] [DEBUG] [VALIDATOR ] BANC: Validating primary data. [2025-11-04T17:28:45.771418] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-04. [2025-11-04T17:28:45.961536] [DEBUG] [VALIDATOR ] BANC: All expected trading days are present. [2025-11-04T17:28:45.961894] [DEBUG] [VALIDATOR ] BANC: Primary data validation passed. [2025-11-04T17:28:45.963947] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T17:28:45.963179] [INFO ] [MAIN ] BANC: Fetched primary data is valid. [2025-11-04T17:28:45.969364] [DEBUG] [LOCAL ] BANC: Saving primary data. [2025-11-04T17:28:45.971427] [DEBUG] [LOCAL ] BANC: Trimmed 74664 records to 51346 (keeping 200 days of data) [2025-11-04T17:28:46.279309] [DEBUG] [LOCAL ] BANC: Primary data saved (51346 records). [2025-11-04T17:28:46.279455] [INFO ] [MAIN ] BANC: Full history saved. [2025-11-04T17:28:46.280142] [INFO ] [MAIN ] Update Progress: 213/2367 symbols completed. BANC took 42.0s. Estimated remaining: 4h 50m 24s. Rate: 7.4 symbols/min [2025-11-04T17:28:46.444127] [INFO ] [MAIN ] BB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:28:46.493548] [DEBUG] [LOCAL ] BB: Loaded 52846 existing records from primary storage. [2025-11-04T17:28:46.493900] [INFO ] [MAIN ] BB: Loaded 52846 existing records. [2025-11-04T17:28:46.572175] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-09. [2025-11-04T17:28:46.668826] [WARN ] [MAIN ] BB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:28:46.669026] [INFO ] [MAIN ] BB: Wiping existing primary data before full refresh. [2025-11-04T17:28:46.669074] [INFO ] [LOCAL ] BB: Starting deletion of primary data. [2025-11-04T17:28:46.669253] [DEBUG] [LOCAL ] BB: Deleted primary data file. [2025-11-04T17:28:46.669292] [INFO ] [MAIN ] BB: Starting full primary refresh. [2025-11-04T17:28:47.019455] [INFO ] [REMOTE ] BB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:28:47.369161] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-07. [2025-11-04T17:28:47.374526] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-11-04T17:28:47.635050] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T17:28:48.167025] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-01. [2025-11-04T17:28:48.240291] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-11-04T17:28:48.967091] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-08. [2025-11-04T17:28:49.767085] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-05. [2025-11-04T17:28:49.854939] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T17:28:49.908829] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-11-04T17:28:50.567106] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-10. [2025-11-04T17:28:51.127126] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T17:28:51.367055] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-09. [2025-11-04T17:28:51.653991] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-11-04T17:28:52.167073] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-11. [2025-11-04T17:28:52.476279] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T17:28:52.967084] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-02. [2025-11-04T17:28:53.171870] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T17:28:53.173395] [INFO ] [REMOTE ] BARK: Full fetch complete (55645 records). [2025-11-04T17:28:53.173498] [DEBUG] [VALIDATOR ] BARK: Validating primary data. [2025-11-04T17:28:53.559765] [DEBUG] [VALIDATOR ] BARK: All expected trading days are present. [2025-11-04T17:28:53.559853] [DEBUG] [VALIDATOR ] BARK: Primary data validation passed. [2025-11-04T17:28:53.560469] [INFO ] [MAIN ] BARK: Fetched primary data is valid. [2025-11-04T17:28:53.560509] [DEBUG] [LOCAL ] BARK: Saving primary data. [2025-11-04T17:28:53.561633] [DEBUG] [LOCAL ] BARK: Trimmed 55645 records to 38057 (keeping 200 days of data) [2025-11-04T17:28:53.767058] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-06. [2025-11-04T17:28:53.773766] [DEBUG] [LOCAL ] BARK: Primary data saved (38057 records). [2025-11-04T17:28:53.774608] [INFO ] [MAIN ] BARK: Full history saved. [2025-11-04T17:28:53.774729] [INFO ] [MAIN ] Update Progress: 214/2367 symbols completed. BARK took 26.5s. Estimated remaining: 4h 50m 10s. Rate: 7.4 symbols/min [2025-11-04T17:28:53.940984] [INFO ] [MAIN ] BBAI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:28:53.998319] [DEBUG] [LOCAL ] BBAI: Loaded 53820 existing records from primary storage. [2025-11-04T17:28:53.998619] [INFO ] [MAIN ] BBAI: Loaded 53820 existing records. [2025-11-04T17:28:54.191033] [WARN ] [MAIN ] BBAI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:28:54.191259] [INFO ] [MAIN ] BBAI: Wiping existing primary data before full refresh. [2025-11-04T17:28:54.191305] [INFO ] [LOCAL ] BBAI: Starting deletion of primary data. [2025-11-04T17:28:54.191518] [DEBUG] [LOCAL ] BBAI: Deleted primary data file. [2025-11-04T17:28:54.191574] [INFO ] [MAIN ] BBAI: Starting full primary refresh. [2025-11-04T17:28:54.548287] [INFO ] [REMOTE ] BBAI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:28:54.572291] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-10. [2025-11-04T17:28:54.587054] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-11-04T17:28:55.138548] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-11-04T17:28:55.367044] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-07. [2025-11-04T17:28:55.440166] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-11-04T17:28:56.167061] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-11. [2025-11-04T17:28:56.865259] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-11-04T17:28:56.896629] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T17:28:56.898229] [INFO ] [REMOTE ] BAP: Full fetch complete (37142 records). [2025-11-04T17:28:56.898332] [DEBUG] [VALIDATOR ] BAP: Validating primary data. [2025-11-04T17:28:56.972225] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-08. [2025-11-04T17:28:57.293909] [DEBUG] [VALIDATOR ] BAP: All expected trading days are present. [2025-11-04T17:28:57.294003] [DEBUG] [VALIDATOR ] BAP: Primary data validation passed. [2025-11-04T17:28:57.294475] [INFO ] [MAIN ] BAP: Fetched primary data is valid. [2025-11-04T17:28:57.294517] [DEBUG] [LOCAL ] BAP: Saving primary data. [2025-11-04T17:28:57.295410] [DEBUG] [LOCAL ] BAP: Trimmed 37142 records to 25084 (keeping 200 days of data) [2025-11-04T17:28:57.447713] [DEBUG] [LOCAL ] BAP: Primary data saved (25084 records). [2025-11-04T17:28:57.447852] [INFO ] [MAIN ] BAP: Full history saved. [2025-11-04T17:28:57.447935] [INFO ] [MAIN ] Update Progress: 215/2367 symbols completed. BAP took 33.0s. Estimated remaining: 4h 49m 18s. Rate: 7.4 symbols/min [2025-11-04T17:28:57.617513] [INFO ] [MAIN ] BBAR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:28:57.660921] [DEBUG] [LOCAL ] BBAR: Loaded 42918 existing records from primary storage. [2025-11-04T17:28:57.661225] [INFO ] [MAIN ] BBAR: Loaded 42918 existing records. [2025-11-04T17:28:57.772245] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-01. [2025-11-04T17:28:57.844697] [WARN ] [MAIN ] BBAR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:28:57.844878] [INFO ] [MAIN ] BBAR: Wiping existing primary data before full refresh. [2025-11-04T17:28:57.844927] [INFO ] [LOCAL ] BBAR: Starting deletion of primary data. [2025-11-04T17:28:57.845102] [DEBUG] [LOCAL ] BBAR: Deleted primary data file. [2025-11-04T17:28:57.845141] [INFO ] [MAIN ] BBAR: Starting full primary refresh. [2025-11-04T17:28:58.197230] [INFO ] [REMOTE ] BBAR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:28:58.572206] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-01. [2025-11-04T17:28:58.591029] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-11-04T17:28:59.367091] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-03. [2025-11-04T17:28:59.749940] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T17:28:59.882318] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-11-04T17:29:00.167048] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-02. [2025-11-04T17:29:00.967099] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-02. [2025-11-04T17:29:01.389904] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-11-04T17:29:01.532227] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-11-04T17:29:01.767148] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-04. [2025-11-04T17:29:02.567055] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-09. [2025-11-04T17:29:03.141011] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-02 (2.17 seconds). [2025-11-04T17:29:03.302335] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-11-04T17:29:03.367052] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-03. [2025-11-04T17:29:04.041003] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-11-04T17:29:04.167050] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-10. [2025-11-04T17:29:04.967076] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-05. [2025-11-04T17:29:05.635806] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T17:29:05.767004] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-11. [2025-11-04T17:29:05.819479] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-03 (2.45 seconds). [2025-11-04T17:29:06.202171] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-11 (0.44 seconds). [2025-11-04T17:29:06.204143] [INFO ] [REMOTE ] BAX: Full fetch complete (77623 records). [2025-11-04T17:29:06.204239] [DEBUG] [VALIDATOR ] BAX: Validating primary data. [2025-11-04T17:29:06.568827] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-04. [2025-11-04T17:29:06.612086] [DEBUG] [VALIDATOR ] BAX: All expected trading days are present. [2025-11-04T17:29:06.612179] [DEBUG] [VALIDATOR ] BAX: Primary data validation passed. [2025-11-04T17:29:06.613022] [INFO ] [MAIN ] BAX: Fetched primary data is valid. [2025-11-04T17:29:06.613283] [DEBUG] [LOCAL ] BAX: Saving primary data. [2025-11-04T17:29:06.644786] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-11-04T17:29:06.643304] [DEBUG] [LOCAL ] BAX: Trimmed 77623 records to 53540 (keeping 200 days of data) [2025-11-04T17:29:06.969971] [DEBUG] [LOCAL ] BAX: Primary data saved (53540 records). [2025-11-04T17:29:06.970097] [INFO ] [MAIN ] BAX: Full history saved. [2025-11-04T17:29:06.970173] [INFO ] [MAIN ] Update Progress: 216/2367 symbols completed. BAX took 33.0s. Estimated remaining: 4h 49m 24s. Rate: 7.4 symbols/min [2025-11-04T17:29:07.133599] [INFO ] [MAIN ] BBD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:29:07.180716] [DEBUG] [LOCAL ] BBD: Loaded 53296 existing records from primary storage. [2025-11-04T17:29:07.181016] [INFO ] [MAIN ] BBD: Loaded 53296 existing records. [2025-11-04T17:29:07.356891] [WARN ] [MAIN ] BBD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:29:07.357146] [INFO ] [MAIN ] BBD: Wiping existing primary data before full refresh. [2025-11-04T17:29:07.357193] [INFO ] [LOCAL ] BBD: Starting deletion of primary data. [2025-11-04T17:29:07.357396] [DEBUG] [LOCAL ] BBD: Deleted primary data file. [2025-11-04T17:29:07.357451] [INFO ] [MAIN ] BBD: Starting full primary refresh. [2025-11-04T17:29:07.377460] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-06. [2025-11-04T17:29:07.707221] [INFO ] [REMOTE ] BBD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:29:08.167039] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-03. [2025-11-04T17:29:08.682053] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-04 (2.11 seconds). [2025-11-04T17:29:08.967008] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-05. [2025-11-04T17:29:09.135604] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-11-04T17:29:09.213505] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-11-04T17:29:09.767050] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-07. [2025-11-04T17:29:10.567043] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-04. [2025-11-04T17:29:11.242503] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-05 (2.28 seconds). [2025-11-04T17:29:11.367015] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-06. [2025-11-04T17:29:11.440033] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-11-04T17:29:11.567935] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-11-04T17:29:12.167047] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-01. [2025-11-04T17:29:12.967039] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-08. [2025-11-04T17:29:13.643479] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-06 (2.28 seconds). [2025-11-04T17:29:13.741083] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-11-04T17:29:13.767000] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-02. [2025-11-04T17:29:14.345396] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T17:29:14.567054] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-09. [2025-11-04T17:29:15.323577] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-11-04T17:29:15.367004] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-03. [2025-11-04T17:29:16.167033] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-07. [2025-11-04T17:29:16.305415] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-11-04T17:29:16.771748] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T17:29:16.967040] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-10. [2025-11-04T17:29:17.767048] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-05. [2025-11-04T17:29:18.428461] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-11-04T17:29:18.560537] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T17:29:18.573537] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-08. [2025-11-04T17:29:18.901077] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T17:29:19.367104] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-11. [2025-11-04T17:29:20.167085] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-06. [2025-11-04T17:29:20.646830] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-11-04T17:29:20.815556] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-11 (1.45 seconds). [2025-11-04T17:29:20.817612] [INFO ] [REMOTE ] BB: Full fetch complete (76948 records). [2025-11-04T17:29:20.817725] [DEBUG] [VALIDATOR ] BB: Validating primary data. [2025-11-04T17:29:20.972215] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-09. [2025-11-04T17:29:21.216531] [DEBUG] [VALIDATOR ] BB: All expected trading days are present. [2025-11-04T17:29:21.216698] [DEBUG] [VALIDATOR ] BB: Primary data validation passed. [2025-11-04T17:29:21.217679] [INFO ] [MAIN ] BB: Fetched primary data is valid. [2025-11-04T17:29:21.217772] [DEBUG] [LOCAL ] BB: Saving primary data. [2025-11-04T17:29:21.219308] [DEBUG] [LOCAL ] BB: Trimmed 76948 records to 52863 (keeping 200 days of data) [2025-11-04T17:29:21.440421] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T17:29:21.549322] [DEBUG] [LOCAL ] BB: Primary data saved (52863 records). [2025-11-04T17:29:21.549803] [INFO ] [MAIN ] BB: Full history saved. [2025-11-04T17:29:21.550445] [INFO ] [MAIN ] Update Progress: 217/2367 symbols completed. BB took 35.3s. Estimated remaining: 4h 50m 21s. Rate: 7.4 symbols/min [2025-11-04T17:29:21.721861] [INFO ] [MAIN ] BBDC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:29:21.722052] [DEBUG] [LOCAL ] BBDC: No existing data file found in primary storage. [2025-11-04T17:29:21.722089] [WARN ] [MAIN ] BBDC: No existing data found, performing full refresh instead. [2025-11-04T17:29:21.722203] [INFO ] [MAIN ] BBDC: Wiping existing primary data before full refresh. [2025-11-04T17:29:21.722244] [INFO ] [LOCAL ] BBDC: Starting deletion of primary data. [2025-11-04T17:29:21.722285] [DEBUG] [LOCAL ] BBDC: No primary data file found to delete. [2025-11-04T17:29:21.722316] [INFO ] [MAIN ] BBDC: Starting full primary refresh. [2025-11-04T17:29:21.775594] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-04. [2025-11-04T17:29:22.079567] [INFO ] [REMOTE ] BBDC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:29:22.567042] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-07. [2025-11-04T17:29:23.080782] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T17:29:23.367036] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-05. [2025-11-04T17:29:23.553551] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-09 (2.58 seconds). [2025-11-04T17:29:23.887408] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T17:29:24.167105] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-01. [2025-11-04T17:29:24.645258] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T17:29:24.967044] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-06. [2025-11-04T17:29:25.163714] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T17:29:25.767019] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-08. [2025-11-04T17:29:26.280277] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T17:29:26.567103] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-10. [2025-11-04T17:29:26.874803] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T17:29:27.367039] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-09. [2025-11-04T17:29:28.167098] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-02. [2025-11-04T17:29:28.723156] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-10 (2.16 seconds). [2025-11-04T17:29:28.767221] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T17:29:28.967064] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-10. [2025-11-04T17:29:29.362826] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T17:29:29.767041] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-07. [2025-11-04T17:29:30.238339] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T17:29:30.567037] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-11. [2025-11-04T17:29:31.138622] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T17:29:31.367091] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-03. [2025-11-04T17:29:32.167037] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-08. [2025-11-04T17:29:32.421187] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-11 (1.85 seconds). [2025-11-04T17:29:32.423210] [INFO ] [REMOTE ] BBAI: Full fetch complete (78000 records). [2025-11-04T17:29:32.423323] [DEBUG] [VALIDATOR ] BBAI: Validating primary data. [2025-11-04T17:29:32.718678] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-11-04T17:29:32.860191] [DEBUG] [VALIDATOR ] BBAI: All expected trading days are present. [2025-11-04T17:29:32.860267] [DEBUG] [VALIDATOR ] BBAI: Primary data validation passed. [2025-11-04T17:29:32.861028] [INFO ] [MAIN ] BBAI: Fetched primary data is valid. [2025-11-04T17:29:32.861091] [DEBUG] [LOCAL ] BBAI: Saving primary data. [2025-11-04T17:29:32.862473] [DEBUG] [LOCAL ] BBAI: Trimmed 78000 records to 53820 (keeping 200 days of data) [2025-11-04T17:29:32.968771] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-04. [2025-11-04T17:29:33.173668] [DEBUG] [LOCAL ] BBAI: Primary data saved (53820 records). [2025-11-04T17:29:33.173796] [INFO ] [MAIN ] BBAI: Full history saved. [2025-11-04T17:29:33.175330] [INFO ] [MAIN ] Update Progress: 218/2367 symbols completed. BBAI took 39.4s. Estimated remaining: 4h 50m 47s. Rate: 7.4 symbols/min [2025-11-04T17:29:33.348332] [INFO ] [MAIN ] BBDO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:29:33.357582] [DEBUG] [LOCAL ] BBDO: Loaded 4925 existing records from primary storage. [2025-11-04T17:29:33.357845] [INFO ] [MAIN ] BBDO: Loaded 4925 existing records. [2025-11-04T17:29:33.568615] [WARN ] [MAIN ] BBDO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:29:33.568951] [INFO ] [MAIN ] BBDO: Wiping existing primary data before full refresh. [2025-11-04T17:29:33.569193] [INFO ] [LOCAL ] BBDO: Starting deletion of primary data. [2025-11-04T17:29:33.569402] [DEBUG] [LOCAL ] BBDO: Deleted primary data file. [2025-11-04T17:29:33.569528] [INFO ] [MAIN ] BBDO: Starting full primary refresh. [2025-11-04T17:29:33.771482] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-11. [2025-11-04T17:29:33.815002] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-11-04T17:29:33.978279] [INFO ] [REMOTE ] BBDO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:29:34.037854] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T17:29:34.567035] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-01. [2025-11-04T17:29:35.185946] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-11-04T17:29:35.238936] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-11 (1.47 seconds). [2025-11-04T17:29:35.240693] [INFO ] [REMOTE ] BBAR: Full fetch complete (61462 records). [2025-11-04T17:29:35.240773] [DEBUG] [VALIDATOR ] BBAR: Validating primary data. [2025-11-04T17:29:35.380978] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-05. [2025-11-04T17:29:35.630722] [DEBUG] [VALIDATOR ] BBAR: All expected trading days are present. [2025-11-04T17:29:35.630816] [DEBUG] [VALIDATOR ] BBAR: Primary data validation passed. [2025-11-04T17:29:35.631447] [INFO ] [MAIN ] BBAR: Fetched primary data is valid. [2025-11-04T17:29:35.631487] [DEBUG] [LOCAL ] BBAR: Saving primary data. [2025-11-04T17:29:35.632794] [DEBUG] [LOCAL ] BBAR: Trimmed 61462 records to 42969 (keeping 200 days of data) [2025-11-04T17:29:35.889049] [DEBUG] [LOCAL ] BBAR: Primary data saved (42969 records). [2025-11-04T17:29:35.889172] [INFO ] [MAIN ] BBAR: Full history saved. [2025-11-04T17:29:35.890707] [INFO ] [MAIN ] Update Progress: 219/2367 symbols completed. BBAR took 38.4s. Estimated remaining: 4h 49m 46s. Rate: 7.4 symbols/min [2025-11-04T17:29:36.057408] [INFO ] [MAIN ] BBN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:29:36.076147] [DEBUG] [LOCAL ] BBN: Loaded 17440 existing records from primary storage. [2025-11-04T17:29:36.076375] [INFO ] [MAIN ] BBN: Loaded 17440 existing records. [2025-11-04T17:29:36.174211] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-02. [2025-11-04T17:29:36.256467] [WARN ] [MAIN ] BBN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:29:36.256687] [INFO ] [MAIN ] BBN: Wiping existing primary data before full refresh. [2025-11-04T17:29:36.256747] [INFO ] [LOCAL ] BBN: Starting deletion of primary data. [2025-11-04T17:29:36.256944] [DEBUG] [LOCAL ] BBN: Deleted primary data file. [2025-11-04T17:29:36.257000] [INFO ] [MAIN ] BBN: Starting full primary refresh. [2025-11-04T17:29:36.610592] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-11-04T17:29:36.631812] [INFO ] [REMOTE ] BBN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:29:36.740504] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T17:29:36.967038] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-01. [2025-11-04T17:29:37.741585] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-11-04T17:29:37.767002] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-03. [2025-11-04T17:29:38.188103] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T17:29:38.567081] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-04. [2025-11-04T17:29:39.061362] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-11-04T17:29:39.367055] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-05. [2025-11-04T17:29:39.777775] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-11-04T17:29:40.167075] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-02. [2025-11-04T17:29:40.840313] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-11-04T17:29:40.967098] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-09. [2025-11-04T17:29:41.767150] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-06. [2025-11-04T17:29:42.480686] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-11-04T17:29:42.567064] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-10. [2025-11-04T17:29:42.870787] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T17:29:43.367084] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-06. [2025-11-04T17:29:43.756181] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-11-04T17:29:44.161347] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T17:29:44.173020] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-07. [2025-11-04T17:29:44.552401] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-11-04T17:29:44.967116] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-08. [2025-11-04T17:29:45.439472] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-11-04T17:29:45.767144] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-11. [2025-11-04T17:29:46.567057] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-03. [2025-11-04T17:29:47.113345] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-11 (1.35 seconds). [2025-11-04T17:29:47.115407] [INFO ] [REMOTE ] BBD: Full fetch complete (77262 records). [2025-11-04T17:29:47.115518] [DEBUG] [VALIDATOR ] BBD: Validating primary data. [2025-11-04T17:29:47.372417] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-07. [2025-11-04T17:29:47.477179] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T17:29:47.539197] [DEBUG] [VALIDATOR ] BBD: All expected trading days are present. [2025-11-04T17:29:47.539287] [DEBUG] [VALIDATOR ] BBD: Primary data validation passed. [2025-11-04T17:29:47.540115] [INFO ] [MAIN ] BBD: Fetched primary data is valid. [2025-11-04T17:29:47.540182] [DEBUG] [LOCAL ] BBD: Saving primary data. [2025-11-04T17:29:47.541705] [DEBUG] [LOCAL ] BBD: Trimmed 77262 records to 53305 (keeping 200 days of data) [2025-11-04T17:29:47.866834] [DEBUG] [LOCAL ] BBD: Primary data saved (53305 records). [2025-11-04T17:29:47.867029] [INFO ] [MAIN ] BBD: Full history saved. [2025-11-04T17:29:47.867119] [INFO ] [MAIN ] Update Progress: 220/2367 symbols completed. BBD took 40.9s. Estimated remaining: 4h 50m 16s. Rate: 7.4 symbols/min [2025-11-04T17:29:48.068088] [INFO ] [MAIN ] BBT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:29:48.068275] [DEBUG] [LOCAL ] BBT: No existing data file found in primary storage. [2025-11-04T17:29:48.068314] [WARN ] [MAIN ] BBT: No existing data found, performing full refresh instead. [2025-11-04T17:29:48.068457] [INFO ] [MAIN ] BBT: Wiping existing primary data before full refresh. [2025-11-04T17:29:48.068499] [INFO ] [LOCAL ] BBT: Starting deletion of primary data. [2025-11-04T17:29:48.068541] [DEBUG] [LOCAL ] BBT: No primary data file found to delete. [2025-11-04T17:29:48.068574] [INFO ] [MAIN ] BBT: Starting full primary refresh. [2025-11-04T17:29:48.167377] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-04. [2025-11-04T17:29:48.420593] [INFO ] [REMOTE ] BBT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:29:48.615540] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-11-04T17:29:48.967024] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-01. [2025-11-04T17:29:49.034272] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-11-04T17:29:49.123998] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:29:49.124328] [WARN ] [REMOTE ] BBT: No data returned for month 2025-01 [2025-11-04T17:29:49.124415] [FATAL] [REMOTE ] BBT: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:29:49.124484] [FATAL] [MAIN ] BBT: Failed to fetch primary data. [2025-11-04T17:29:49.124548] [WARN ] [MAIN ] BBT: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:29:49.124621] [INFO ] [LOCAL ] BBT: Starting deletion of primary data. [2025-11-04T17:29:49.124719] [DEBUG] [LOCAL ] BBT: No primary data file found to delete. [2025-11-04T17:29:49.124764] [INFO ] [LOCAL ] BBT: Starting deletion of derivative data. [2025-11-04T17:29:49.124806] [DEBUG] [LOCAL ] BBT: No derivative data file found to delete. [2025-11-04T17:29:49.124853] [INFO ] [MAIN ] Update Progress: 221/2367 symbols completed. BBT took 1.3s. Estimated remaining: 4h 49m 1s. Rate: 7.4 symbols/min [2025-11-04T17:29:49.294741] [INFO ] [MAIN ] BBU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:29:49.302994] [DEBUG] [LOCAL ] BBU: Loaded 6029 existing records from primary storage. [2025-11-04T17:29:49.303235] [INFO ] [MAIN ] BBU: Loaded 6029 existing records. [2025-11-04T17:29:49.475549] [WARN ] [MAIN ] BBU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:29:49.475803] [INFO ] [MAIN ] BBU: Wiping existing primary data before full refresh. [2025-11-04T17:29:49.475855] [INFO ] [LOCAL ] BBU: Starting deletion of primary data. [2025-11-04T17:29:49.476021] [DEBUG] [LOCAL ] BBU: Deleted primary data file. [2025-11-04T17:29:49.476075] [INFO ] [MAIN ] BBU: Starting full primary refresh. [2025-11-04T17:29:49.772182] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-08. [2025-11-04T17:29:49.828495] [INFO ] [REMOTE ] BBU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:29:50.567087] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-05. [2025-11-04T17:29:50.782390] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-11-04T17:29:51.249740] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-11-04T17:29:51.367016] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-06. [2025-11-04T17:29:51.994142] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-11-04T17:29:52.167095] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-01. [2025-11-04T17:29:52.644092] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-01 (0.48 seconds). [2025-11-04T17:29:52.967022] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-07. [2025-11-04T17:29:53.712195] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-11-04T17:29:53.767051] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-09. [2025-11-04T17:29:54.567050] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-08. [2025-11-04T17:29:55.020532] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T17:29:55.354504] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T17:29:55.366997] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-09. [2025-11-04T17:29:56.100603] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-11-04T17:29:56.167041] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-10. [2025-11-04T17:29:56.970001] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-02. [2025-11-04T17:29:56.971068] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T17:29:57.435589] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-11-04T17:29:57.767059] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-09. [2025-11-04T17:29:58.229693] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-11-04T17:29:58.567068] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-10. [2025-11-04T17:29:59.037580] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-11-04T17:29:59.367049] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-03. [2025-11-04T17:29:59.722785] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-11-04T17:30:00.167092] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-04. [2025-11-04T17:30:00.580835] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-11-04T17:30:00.967040] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-05. [2025-11-04T17:30:01.767049] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-11. [2025-11-04T17:30:01.889234] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T17:30:02.567097] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-06. [2025-11-04T17:30:02.848324] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-11 (1.08 seconds). [2025-11-04T17:30:02.849958] [INFO ] [REMOTE ] BBN: Full fetch complete (25123 records). [2025-11-04T17:30:02.850060] [DEBUG] [VALIDATOR ] BBN: Validating primary data. [2025-11-04T17:30:02.850838] [FATAL] [VALIDATOR ] BBN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:30:02.850953] [FATAL] [MAIN ] BBN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:30:02.851042] [WARN ] [MAIN ] BBN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:30:02.851141] [INFO ] [LOCAL ] BBN: Starting deletion of primary data. [2025-11-04T17:30:02.851254] [DEBUG] [LOCAL ] BBN: No primary data file found to delete. [2025-11-04T17:30:02.851333] [INFO ] [LOCAL ] BBN: Starting deletion of derivative data. [2025-11-04T17:30:02.851957] [DEBUG] [LOCAL ] BBN: Deleted derivative data file. [2025-11-04T17:30:02.852267] [INFO ] [MAIN ] Update Progress: 222/2367 symbols completed. BBN took 27.0s. Estimated remaining: 4h 49m 48s. Rate: 7.4 symbols/min [2025-11-04T17:30:03.030208] [INFO ] [MAIN ] BBUC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:30:03.048063] [DEBUG] [LOCAL ] BBUC: Loaded 13781 existing records from primary storage. [2025-11-04T17:30:03.048313] [INFO ] [MAIN ] BBUC: Loaded 13781 existing records. [2025-11-04T17:30:03.163116] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-11-04T17:30:03.235874] [WARN ] [MAIN ] BBUC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:30:03.236115] [INFO ] [MAIN ] BBUC: Wiping existing primary data before full refresh. [2025-11-04T17:30:03.236165] [INFO ] [LOCAL ] BBUC: Starting deletion of primary data. [2025-11-04T17:30:03.236344] [DEBUG] [LOCAL ] BBUC: Deleted primary data file. [2025-11-04T17:30:03.236409] [INFO ] [MAIN ] BBUC: Starting full primary refresh. [2025-11-04T17:30:03.376232] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-07. [2025-11-04T17:30:03.595821] [INFO ] [REMOTE ] BBUC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:30:03.990140] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-11-04T17:30:04.167085] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-11. [2025-11-04T17:30:04.944154] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T17:30:04.944960] [INFO ] [REMOTE ] BBDO: Full fetch complete (7444 records). [2025-11-04T17:30:04.945030] [DEBUG] [VALIDATOR ] BBDO: Validating primary data. [2025-11-04T17:30:04.945433] [FATAL] [VALIDATOR ] BBDO: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:30:04.945516] [FATAL] [MAIN ] BBDO: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:30:04.945559] [WARN ] [MAIN ] BBDO: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:30:04.945625] [INFO ] [LOCAL ] BBDO: Starting deletion of primary data. [2025-11-04T17:30:04.945725] [DEBUG] [LOCAL ] BBDO: No primary data file found to delete. [2025-11-04T17:30:04.945768] [INFO ] [LOCAL ] BBDO: Starting deletion of derivative data. [2025-11-04T17:30:04.945932] [DEBUG] [LOCAL ] BBDO: Deleted derivative data file. [2025-11-04T17:30:04.946023] [INFO ] [MAIN ] Update Progress: 223/2367 symbols completed. BBDO took 31.8s. Estimated remaining: 4h 48m 42s. Rate: 7.4 symbols/min [2025-11-04T17:30:04.972207] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-08. [2025-11-04T17:30:05.129569] [INFO ] [MAIN ] BBVA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:30:05.175536] [DEBUG] [LOCAL ] BBVA: Loaded 48940 existing records from primary storage. [2025-11-04T17:30:05.175851] [INFO ] [MAIN ] BBVA: Loaded 48940 existing records. [2025-11-04T17:30:05.361198] [WARN ] [MAIN ] BBVA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:30:05.361458] [INFO ] [MAIN ] BBVA: Wiping existing primary data before full refresh. [2025-11-04T17:30:05.361519] [INFO ] [LOCAL ] BBVA: Starting deletion of primary data. [2025-11-04T17:30:05.361764] [DEBUG] [LOCAL ] BBVA: Deleted primary data file. [2025-11-04T17:30:05.361824] [INFO ] [MAIN ] BBVA: Starting full primary refresh. [2025-11-04T17:30:05.526621] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-11-04T17:30:05.772216] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-01. [2025-11-04T17:30:05.924948] [INFO ] [REMOTE ] BBVA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:30:06.567036] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-01. [2025-11-04T17:30:06.596070] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-11-04T17:30:07.367052] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-02. [2025-11-04T17:30:07.977884] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-11-04T17:30:08.136025] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-11-04T17:30:08.167027] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-03. [2025-11-04T17:30:08.967053] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-10. [2025-11-04T17:30:08.981903] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-11-04T17:30:09.767039] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-04. [2025-11-04T17:30:10.443910] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-11-04T17:30:10.501805] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T17:30:10.567003] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-05. [2025-11-04T17:30:11.209899] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T17:30:11.367054] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-02. [2025-11-04T17:30:12.167089] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-06. [2025-11-04T17:30:12.515738] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T17:30:12.967048] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-03. [2025-11-04T17:30:13.108204] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-11-04T17:30:13.767052] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-07. [2025-11-04T17:30:14.370913] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T17:30:14.389901] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-11-04T17:30:14.567031] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-08. [2025-11-04T17:30:15.124845] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-11-04T17:30:15.367045] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-11. [2025-11-04T17:30:16.167031] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-04. [2025-11-04T17:30:16.266172] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T17:30:16.267349] [INFO ] [REMOTE ] BBDC: Full fetch complete (48784 records). [2025-11-04T17:30:16.267421] [DEBUG] [VALIDATOR ] BBDC: Validating primary data. [2025-11-04T17:30:16.268034] [FATAL] [VALIDATOR ] BBDC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:30:16.268118] [FATAL] [MAIN ] BBDC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:30:16.268159] [WARN ] [MAIN ] BBDC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:30:16.268196] [INFO ] [LOCAL ] BBDC: Starting deletion of primary data. [2025-11-04T17:30:16.268248] [DEBUG] [LOCAL ] BBDC: No primary data file found to delete. [2025-11-04T17:30:16.268281] [INFO ] [LOCAL ] BBDC: Starting deletion of derivative data. [2025-11-04T17:30:16.268319] [DEBUG] [LOCAL ] BBDC: No derivative data file found to delete. [2025-11-04T17:30:16.268370] [INFO ] [MAIN ] Update Progress: 224/2367 symbols completed. BBDC took 54.7s. Estimated remaining: 4h 49m 5s. Rate: 7.4 symbols/min [2025-11-04T17:30:16.436303] [INFO ] [MAIN ] BBW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:30:16.465199] [DEBUG] [LOCAL ] BBW: Loaded 26747 existing records from primary storage. [2025-11-04T17:30:16.465443] [INFO ] [MAIN ] BBW: Loaded 26747 existing records. [2025-11-04T17:30:16.636804] [WARN ] [MAIN ] BBW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:30:16.636982] [INFO ] [MAIN ] BBW: Wiping existing primary data before full refresh. [2025-11-04T17:30:16.637027] [INFO ] [LOCAL ] BBW: Starting deletion of primary data. [2025-11-04T17:30:16.637196] [DEBUG] [LOCAL ] BBW: Deleted primary data file. [2025-11-04T17:30:16.637233] [INFO ] [MAIN ] BBW: Starting full primary refresh. [2025-11-04T17:30:16.969563] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-09. [2025-11-04T17:30:16.981914] [INFO ] [REMOTE ] BBW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:30:17.617150] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T17:30:17.767036] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-09. [2025-11-04T17:30:17.891115] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T17:30:18.387733] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-11-04T17:30:18.567051] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-01. [2025-11-04T17:30:19.367077] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-10. [2025-11-04T17:30:19.472137] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-11-04T17:30:20.024698] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-11-04T17:30:20.167095] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-10. [2025-11-04T17:30:20.761581] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-11-04T17:30:20.967089] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-05. [2025-11-04T17:30:21.767044] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-02. [2025-11-04T17:30:22.572135] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-11. [2025-11-04T17:30:22.578737] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-11-04T17:30:22.655862] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T17:30:23.293243] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T17:30:23.294182] [INFO ] [REMOTE ] BBUC: Full fetch complete (19475 records). [2025-11-04T17:30:23.294273] [DEBUG] [VALIDATOR ] BBUC: Validating primary data. [2025-11-04T17:30:23.294850] [FATAL] [VALIDATOR ] BBUC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:30:23.294951] [FATAL] [MAIN ] BBUC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:30:23.295001] [WARN ] [MAIN ] BBUC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:30:23.295094] [INFO ] [LOCAL ] BBUC: Starting deletion of primary data. [2025-11-04T17:30:23.295188] [DEBUG] [LOCAL ] BBUC: No primary data file found to delete. [2025-11-04T17:30:23.295236] [INFO ] [LOCAL ] BBUC: Starting deletion of derivative data. [2025-11-04T17:30:23.295408] [DEBUG] [LOCAL ] BBUC: Deleted derivative data file. [2025-11-04T17:30:23.295513] [INFO ] [MAIN ] Update Progress: 225/2367 symbols completed. BBUC took 20.4s. Estimated remaining: 4h 48m 46s. Rate: 7.4 symbols/min [2025-11-04T17:30:23.372154] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-03. [2025-11-04T17:30:23.476065] [INFO ] [MAIN ] BBWI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:30:23.538486] [DEBUG] [LOCAL ] BBWI: Loaded 53591 existing records from primary storage. [2025-11-04T17:30:23.538837] [INFO ] [MAIN ] BBWI: Loaded 53591 existing records. [2025-11-04T17:30:23.789133] [WARN ] [MAIN ] BBWI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:30:23.789369] [INFO ] [MAIN ] BBWI: Wiping existing primary data before full refresh. [2025-11-04T17:30:23.789441] [INFO ] [LOCAL ] BBWI: Starting deletion of primary data. [2025-11-04T17:30:23.789732] [DEBUG] [LOCAL ] BBWI: Deleted primary data file. [2025-11-04T17:30:23.789791] [INFO ] [MAIN ] BBWI: Starting full primary refresh. [2025-11-04T17:30:24.171962] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-06. [2025-11-04T17:30:24.281289] [INFO ] [REMOTE ] BBWI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:30:24.300268] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T17:30:24.967055] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-04. [2025-11-04T17:30:25.517695] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-11-04T17:30:25.767068] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-11. [2025-11-04T17:30:26.049284] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T17:30:26.357814] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-11 (0.59 seconds). [2025-11-04T17:30:26.358511] [INFO ] [REMOTE ] BBU: Full fetch complete (7685 records). [2025-11-04T17:30:26.358578] [DEBUG] [VALIDATOR ] BBU: Validating primary data. [2025-11-04T17:30:26.358972] [FATAL] [VALIDATOR ] BBU: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:30:26.359056] [FATAL] [MAIN ] BBU: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:30:26.359099] [WARN ] [MAIN ] BBU: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:30:26.359190] [INFO ] [LOCAL ] BBU: Starting deletion of primary data. [2025-11-04T17:30:26.359266] [DEBUG] [LOCAL ] BBU: No primary data file found to delete. [2025-11-04T17:30:26.359301] [INFO ] [LOCAL ] BBU: Starting deletion of derivative data. [2025-11-04T17:30:26.359425] [DEBUG] [LOCAL ] BBU: Deleted derivative data file. [2025-11-04T17:30:26.359510] [INFO ] [MAIN ] Update Progress: 226/2367 symbols completed. BBU took 37.2s. Estimated remaining: 4h 47m 51s. Rate: 7.4 symbols/min [2025-11-04T17:30:26.529686] [INFO ] [MAIN ] BBY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:30:26.574283] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-07. [2025-11-04T17:30:26.577486] [DEBUG] [LOCAL ] BBY: Loaded 53417 existing records from primary storage. [2025-11-04T17:30:26.578039] [INFO ] [MAIN ] BBY: Loaded 53417 existing records. [2025-11-04T17:30:26.758951] [WARN ] [MAIN ] BBY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:30:26.759186] [INFO ] [MAIN ] BBY: Wiping existing primary data before full refresh. [2025-11-04T17:30:26.759233] [INFO ] [LOCAL ] BBY: Starting deletion of primary data. [2025-11-04T17:30:26.759408] [DEBUG] [LOCAL ] BBY: Deleted primary data file. [2025-11-04T17:30:26.759444] [INFO ] [MAIN ] BBY: Starting full primary refresh. [2025-11-04T17:30:27.101833] [INFO ] [REMOTE ] BBY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:30:27.367055] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-01. [2025-11-04T17:30:27.972152] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T17:30:28.167060] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-01. [2025-11-04T17:30:28.967088] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-08. [2025-11-04T17:30:29.181851] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-11-04T17:30:29.419093] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T17:30:29.767057] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-02. [2025-11-04T17:30:30.503915] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-11-04T17:30:30.567021] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-05. [2025-11-04T17:30:31.124393] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T17:30:31.367016] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-09. [2025-11-04T17:30:31.551700] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-11-04T17:30:32.167028] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-06. [2025-11-04T17:30:32.553215] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T17:30:32.967099] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-02. [2025-11-04T17:30:33.049087] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-11-04T17:30:33.767078] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-07. [2025-11-04T17:30:34.264428] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T17:30:34.567087] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-03. [2025-11-04T17:30:34.603388] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-11-04T17:30:35.367105] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-08. [2025-11-04T17:30:36.011248] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T17:30:36.106987] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-11-04T17:30:36.167052] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-09. [2025-11-04T17:30:36.967134] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-10. [2025-11-04T17:30:37.296333] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-11-04T17:30:37.767055] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-04. [2025-11-04T17:30:38.567023] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-03. [2025-11-04T17:30:38.743352] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-11-04T17:30:39.367033] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-10. [2025-11-04T17:30:39.425750] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-11-04T17:30:40.167051] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-05. [2025-11-04T17:30:40.329249] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-11-04T17:30:40.395164] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-11-04T17:30:40.967076] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-11. [2025-11-04T17:30:41.763265] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-11-04T17:30:41.773775] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-11. [2025-11-04T17:30:41.779189] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T17:30:41.781738] [INFO ] [REMOTE ] BBW: Full fetch complete (38990 records). [2025-11-04T17:30:41.781884] [DEBUG] [VALIDATOR ] BBW: Validating primary data. [2025-11-04T17:30:41.782978] [FATAL] [VALIDATOR ] BBW: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:30:41.783083] [FATAL] [MAIN ] BBW: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:30:41.783143] [WARN ] [MAIN ] BBW: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:30:41.783225] [INFO ] [LOCAL ] BBW: Starting deletion of primary data. [2025-11-04T17:30:41.783329] [DEBUG] [LOCAL ] BBW: No primary data file found to delete. [2025-11-04T17:30:41.783403] [INFO ] [LOCAL ] BBW: Starting deletion of derivative data. [2025-11-04T17:30:41.783614] [DEBUG] [LOCAL ] BBW: Deleted derivative data file. [2025-11-04T17:30:41.783760] [INFO ] [MAIN ] Update Progress: 227/2367 symbols completed. BBW took 25.5s. Estimated remaining: 4h 48m 52s. Rate: 7.4 symbols/min [2025-11-04T17:30:41.979988] [INFO ] [MAIN ] BC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:30:42.024965] [DEBUG] [LOCAL ] BC: Loaded 43526 existing records from primary storage. [2025-11-04T17:30:42.025279] [INFO ] [MAIN ] BC: Loaded 43526 existing records. [2025-11-04T17:30:42.200985] [WARN ] [MAIN ] BC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:30:42.201422] [INFO ] [MAIN ] BC: Wiping existing primary data before full refresh. [2025-11-04T17:30:42.201469] [INFO ] [LOCAL ] BC: Starting deletion of primary data. [2025-11-04T17:30:42.201691] [DEBUG] [LOCAL ] BC: Deleted primary data file. [2025-11-04T17:30:42.201746] [INFO ] [MAIN ] BC: Starting full primary refresh. [2025-11-04T17:30:42.551094] [INFO ] [REMOTE ] BC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:30:42.566999] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-01. [2025-11-04T17:30:42.843622] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T17:30:42.845130] [INFO ] [REMOTE ] BBVA: Full fetch complete (71561 records). [2025-11-04T17:30:42.845244] [DEBUG] [VALIDATOR ] BBVA: Validating primary data. [2025-11-04T17:30:42.846032] [FATAL] [VALIDATOR ] BBVA: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:30:42.846145] [FATAL] [MAIN ] BBVA: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:30:42.846187] [WARN ] [MAIN ] BBVA: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:30:42.846225] [INFO ] [LOCAL ] BBVA: Starting deletion of primary data. [2025-11-04T17:30:42.846279] [DEBUG] [LOCAL ] BBVA: No primary data file found to delete. [2025-11-04T17:30:42.846312] [INFO ] [LOCAL ] BBVA: Starting deletion of derivative data. [2025-11-04T17:30:42.846444] [DEBUG] [LOCAL ] BBVA: Deleted derivative data file. [2025-11-04T17:30:42.847108] [INFO ] [MAIN ] Update Progress: 228/2367 symbols completed. BBVA took 37.9s. Estimated remaining: 4h 47m 38s. Rate: 7.4 symbols/min [2025-11-04T17:30:43.017518] [INFO ] [MAIN ] BCAT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:30:43.054311] [DEBUG] [LOCAL ] BCAT: Loaded 33977 existing records from primary storage. [2025-11-04T17:30:43.054591] [INFO ] [MAIN ] BCAT: Loaded 33977 existing records. [2025-11-04T17:30:43.233396] [WARN ] [MAIN ] BCAT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:30:43.233636] [INFO ] [MAIN ] BCAT: Wiping existing primary data before full refresh. [2025-11-04T17:30:43.233716] [INFO ] [LOCAL ] BCAT: Starting deletion of primary data. [2025-11-04T17:30:43.233902] [DEBUG] [LOCAL ] BCAT: Deleted primary data file. [2025-11-04T17:30:43.233986] [INFO ] [MAIN ] BCAT: Starting full primary refresh. [2025-11-04T17:30:43.375244] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-06. [2025-11-04T17:30:43.606213] [INFO ] [REMOTE ] BCAT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:30:43.711862] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T17:30:44.167085] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-02. [2025-11-04T17:30:44.619975] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T17:30:44.967093] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-01. [2025-11-04T17:30:45.009078] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-11-04T17:30:45.767101] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-03. [2025-11-04T17:30:46.117488] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T17:30:46.567043] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-04. [2025-11-04T17:30:46.996442] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T17:30:47.367076] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-07. [2025-11-04T17:30:47.999006] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T17:30:48.167022] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-05. [2025-11-04T17:30:48.949171] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-11-04T17:30:48.967001] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-02. [2025-11-04T17:30:49.624277] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T17:30:49.767026] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-08. [2025-11-04T17:30:49.997512] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T17:30:50.567026] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-04. [2025-11-04T17:30:51.297033] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T17:30:51.367054] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-06. [2025-11-04T17:30:52.172455] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-09. [2025-11-04T17:30:52.178724] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-11-04T17:30:52.831670] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T17:30:52.967042] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-07. [2025-11-04T17:30:53.451620] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T17:30:53.767042] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-05. [2025-11-04T17:30:54.288077] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T17:30:54.567042] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-10. [2025-11-04T17:30:55.052157] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T17:30:55.367052] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-06. [2025-11-04T17:30:56.167083] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-03. [2025-11-04T17:30:56.460129] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T17:30:56.546195] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-11-04T17:30:56.967090] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-11. [2025-11-04T17:30:57.188462] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-11-04T17:30:57.332936] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-11 (0.37 seconds). [2025-11-04T17:30:57.335090] [INFO ] [REMOTE ] BBY: Full fetch complete (77418 records). [2025-11-04T17:30:57.335203] [DEBUG] [VALIDATOR ] BBY: Validating primary data. [2025-11-04T17:30:57.722433] [DEBUG] [VALIDATOR ] BBY: All expected trading days are present. [2025-11-04T17:30:57.722524] [DEBUG] [VALIDATOR ] BBY: Primary data validation passed. [2025-11-04T17:30:57.723315] [INFO ] [MAIN ] BBY: Fetched primary data is valid. [2025-11-04T17:30:57.723383] [DEBUG] [LOCAL ] BBY: Saving primary data. [2025-11-04T17:30:57.724853] [DEBUG] [LOCAL ] BBY: Trimmed 77418 records to 53416 (keeping 200 days of data) [2025-11-04T17:30:57.778755] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-08. [2025-11-04T17:30:58.066720] [DEBUG] [LOCAL ] BBY: Primary data saved (53416 records). [2025-11-04T17:30:58.066889] [INFO ] [MAIN ] BBY: Full history saved. [2025-11-04T17:30:58.066994] [INFO ] [MAIN ] Update Progress: 229/2367 symbols completed. BBY took 31.7s. Estimated remaining: 4h 48m 37s. Rate: 7.4 symbols/min [2025-11-04T17:30:58.238373] [INFO ] [MAIN ] BCC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:30:58.266470] [DEBUG] [LOCAL ] BCC: Loaded 28583 existing records from primary storage. [2025-11-04T17:30:58.266758] [INFO ] [MAIN ] BCC: Loaded 28583 existing records. [2025-11-04T17:30:58.443294] [WARN ] [MAIN ] BCC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:30:58.443497] [INFO ] [MAIN ] BCC: Wiping existing primary data before full refresh. [2025-11-04T17:30:58.443561] [INFO ] [LOCAL ] BCC: Starting deletion of primary data. [2025-11-04T17:30:58.443754] [DEBUG] [LOCAL ] BCC: Deleted primary data file. [2025-11-04T17:30:58.443811] [INFO ] [MAIN ] BCC: Starting full primary refresh. [2025-11-04T17:30:58.572202] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-04. [2025-11-04T17:30:58.807098] [INFO ] [REMOTE ] BCC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:30:59.216639] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-11-04T17:30:59.367029] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-09. [2025-11-04T17:30:59.689530] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T17:31:00.167067] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-05. [2025-11-04T17:31:00.915958] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-11-04T17:31:00.967075] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-10. [2025-11-04T17:31:01.539729] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T17:31:01.767086] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-01. [2025-11-04T17:31:02.505575] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-11-04T17:31:02.567095] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-07. [2025-11-04T17:31:02.845843] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T17:31:03.367048] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-02. [2025-11-04T17:31:03.729171] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-11-04T17:31:04.114592] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T17:31:04.167052] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-06. [2025-11-04T17:31:04.967036] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-11. [2025-11-04T17:31:05.143230] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T17:31:05.767052] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-03. [2025-11-04T17:31:06.413411] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-11 (1.45 seconds). [2025-11-04T17:31:06.414963] [INFO ] [REMOTE ] BBWI: Full fetch complete (77265 records). [2025-11-04T17:31:06.415083] [DEBUG] [VALIDATOR ] BBWI: Validating primary data. [2025-11-04T17:31:06.415890] [FATAL] [VALIDATOR ] BBWI: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:31:06.415976] [FATAL] [MAIN ] BBWI: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:31:06.416019] [WARN ] [MAIN ] BBWI: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:31:06.416244] [INFO ] [LOCAL ] BBWI: Starting deletion of primary data. [2025-11-04T17:31:06.416329] [DEBUG] [LOCAL ] BBWI: No primary data file found to delete. [2025-11-04T17:31:06.416379] [INFO ] [LOCAL ] BBWI: Starting deletion of derivative data. [2025-11-04T17:31:06.416523] [DEBUG] [LOCAL ] BBWI: Deleted derivative data file. [2025-11-04T17:31:06.417402] [INFO ] [MAIN ] Update Progress: 230/2367 symbols completed. BBWI took 43.1s. Estimated remaining: 4h 48m 31s. Rate: 7.4 symbols/min [2025-11-04T17:31:06.571892] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-08. [2025-11-04T17:31:06.605386] [INFO ] [MAIN ] BCE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:31:06.662301] [DEBUG] [LOCAL ] BCE: Loaded 52218 existing records from primary storage. [2025-11-04T17:31:06.662693] [INFO ] [MAIN ] BCE: Loaded 52218 existing records. [2025-11-04T17:31:06.850435] [WARN ] [MAIN ] BCE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:31:06.850753] [INFO ] [MAIN ] BCE: Wiping existing primary data before full refresh. [2025-11-04T17:31:06.850852] [INFO ] [LOCAL ] BCE: Starting deletion of primary data. [2025-11-04T17:31:06.860400] [DEBUG] [LOCAL ] BCE: Deleted primary data file. [2025-11-04T17:31:06.871894] [INFO ] [MAIN ] BCE: Starting full primary refresh. [2025-11-04T17:31:06.918207] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T17:31:07.238843] [INFO ] [REMOTE ] BCE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:31:07.367020] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-01. [2025-11-04T17:31:07.763992] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T17:31:08.167058] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-09. [2025-11-04T17:31:08.876244] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-11-04T17:31:08.967030] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-04. [2025-11-04T17:31:09.251812] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T17:31:09.767006] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-10. [2025-11-04T17:31:09.892263] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-11-04T17:31:10.567035] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-07. [2025-11-04T17:31:11.113639] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T17:31:11.367040] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-05. [2025-11-04T17:31:11.799847] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T17:31:12.172119] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-08. [2025-11-04T17:31:12.183202] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T17:31:12.967035] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-02. [2025-11-04T17:31:13.277483] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T17:31:13.767045] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-06. [2025-11-04T17:31:14.331988] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T17:31:14.567209] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-03. [2025-11-04T17:31:14.664612] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-11-04T17:31:15.367073] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-09. [2025-11-04T17:31:16.123276] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T17:31:16.167079] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-07. [2025-11-04T17:31:16.554373] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T17:31:16.967113] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-04. [2025-11-04T17:31:17.017265] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T17:31:17.767081] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-08. [2025-11-04T17:31:18.518230] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-11-04T17:31:18.567057] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-05. [2025-11-04T17:31:18.609179] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-11-04T17:31:19.367034] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-09. [2025-11-04T17:31:19.870627] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T17:31:20.167046] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-06. [2025-11-04T17:31:20.247335] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T17:31:20.967045] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-10. [2025-11-04T17:31:21.406343] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T17:31:21.767037] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-11. [2025-11-04T17:31:22.197907] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T17:31:22.567070] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-11. [2025-11-04T17:31:22.789016] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T17:31:22.791207] [INFO ] [REMOTE ] BC: Full fetch complete (63907 records). [2025-11-04T17:31:22.791370] [DEBUG] [VALIDATOR ] BC: Validating primary data. [2025-11-04T17:31:23.196833] [DEBUG] [VALIDATOR ] BC: All expected trading days are present. [2025-11-04T17:31:23.196968] [DEBUG] [VALIDATOR ] BC: Primary data validation passed. [2025-11-04T17:31:23.197682] [INFO ] [MAIN ] BC: Fetched primary data is valid. [2025-11-04T17:31:23.197745] [DEBUG] [LOCAL ] BC: Saving primary data. [2025-11-04T17:31:23.199027] [DEBUG] [LOCAL ] BC: Trimmed 63907 records to 43416 (keeping 200 days of data) [2025-11-04T17:31:23.370274] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-07. [2025-11-04T17:31:23.460253] [DEBUG] [LOCAL ] BC: Primary data saved (43416 records). [2025-11-04T17:31:23.460393] [INFO ] [MAIN ] BC: Full history saved. [2025-11-04T17:31:23.462066] [INFO ] [MAIN ] Update Progress: 231/2367 symbols completed. BC took 41.7s. Estimated remaining: 4h 49m 45s. Rate: 7.4 symbols/min [2025-11-04T17:31:23.633198] [INFO ] [MAIN ] BCH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:31:23.636588] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T17:31:23.638360] [INFO ] [REMOTE ] BCAT: Full fetch complete (49805 records). [2025-11-04T17:31:23.638445] [DEBUG] [VALIDATOR ] BCAT: Validating primary data. [2025-11-04T17:31:23.639228] [FATAL] [VALIDATOR ] BCAT: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:31:23.639314] [FATAL] [MAIN ] BCAT: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:31:23.639357] [WARN ] [MAIN ] BCAT: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:31:23.639394] [INFO ] [LOCAL ] BCAT: Starting deletion of primary data. [2025-11-04T17:31:23.639471] [DEBUG] [LOCAL ] BCAT: No primary data file found to delete. [2025-11-04T17:31:23.639508] [INFO ] [LOCAL ] BCAT: Starting deletion of derivative data. [2025-11-04T17:31:23.639673] [DEBUG] [LOCAL ] BCAT: Deleted derivative data file. [2025-11-04T17:31:23.640233] [INFO ] [MAIN ] Update Progress: 232/2367 symbols completed. BCAT took 40.8s. Estimated remaining: 4h 48m 24s. Rate: 7.4 symbols/min [2025-11-04T17:31:23.841948] [INFO ] [MAIN ] BCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:31:23.842775] [DEBUG] [LOCAL ] BCH: Loaded 29410 existing records from primary storage. [2025-11-04T17:31:23.843269] [INFO ] [MAIN ] BCH: Loaded 29410 existing records. [2025-11-04T17:31:24.047785] [DEBUG] [LOCAL ] BCO: Loaded 24087 existing records from primary storage. [2025-11-04T17:31:24.069759] [INFO ] [MAIN ] BCO: Loaded 24087 existing records. [2025-11-04T17:31:24.069571] [WARN ] [MAIN ] BCH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:31:24.114026] [INFO ] [MAIN ] BCH: Wiping existing primary data before full refresh. [2025-11-04T17:31:24.140839] [INFO ] [LOCAL ] BCH: Starting deletion of primary data. [2025-11-04T17:31:24.171054] [DEBUG] [LOCAL ] BCH: Deleted primary data file. [2025-11-04T17:31:24.171005] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-10. [2025-11-04T17:31:24.174152] [INFO ] [MAIN ] BCH: Starting full primary refresh. [2025-11-04T17:31:24.375608] [WARN ] [MAIN ] BCO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:31:24.378590] [INFO ] [MAIN ] BCO: Wiping existing primary data before full refresh. [2025-11-04T17:31:24.378926] [INFO ] [LOCAL ] BCO: Starting deletion of primary data. [2025-11-04T17:31:24.394781] [DEBUG] [LOCAL ] BCO: Deleted primary data file. [2025-11-04T17:31:24.423942] [INFO ] [MAIN ] BCO: Starting full primary refresh. [2025-11-04T17:31:24.884970] [INFO ] [REMOTE ] BCH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:31:24.975088] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-01. [2025-11-04T17:31:25.069720] [INFO ] [REMOTE ] BCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:31:25.116554] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-11-04T17:31:25.593705] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-11-04T17:31:25.767047] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-01. [2025-11-04T17:31:25.970699] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T17:31:26.567023] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-11. [2025-11-04T17:31:26.599328] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-11-04T17:31:27.367051] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-08. [2025-11-04T17:31:27.414365] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-11 (0.85 seconds). [2025-11-04T17:31:27.415770] [INFO ] [REMOTE ] BCC: Full fetch complete (41651 records). [2025-11-04T17:31:27.415840] [DEBUG] [VALIDATOR ] BCC: Validating primary data. [2025-11-04T17:31:27.416544] [FATAL] [VALIDATOR ] BCC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:31:27.416624] [FATAL] [MAIN ] BCC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:31:27.416700] [WARN ] [MAIN ] BCC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:31:27.416770] [INFO ] [LOCAL ] BCC: Starting deletion of primary data. [2025-11-04T17:31:27.416829] [DEBUG] [LOCAL ] BCC: No primary data file found to delete. [2025-11-04T17:31:27.416898] [INFO ] [LOCAL ] BCC: Starting deletion of derivative data. [2025-11-04T17:31:27.417016] [DEBUG] [LOCAL ] BCC: Deleted derivative data file. [2025-11-04T17:31:27.417080] [INFO ] [MAIN ] Update Progress: 233/2367 symbols completed. BCC took 29.3s. Estimated remaining: 4h 47m 36s. Rate: 7.4 symbols/min [2025-11-04T17:31:27.587485] [INFO ] [MAIN ] BCS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:31:27.587639] [DEBUG] [LOCAL ] BCS: No existing data file found in primary storage. [2025-11-04T17:31:27.587718] [WARN ] [MAIN ] BCS: No existing data found, performing full refresh instead. [2025-11-04T17:31:27.587827] [INFO ] [MAIN ] BCS: Wiping existing primary data before full refresh. [2025-11-04T17:31:27.587870] [INFO ] [LOCAL ] BCS: Starting deletion of primary data. [2025-11-04T17:31:27.587942] [DEBUG] [LOCAL ] BCS: No primary data file found to delete. [2025-11-04T17:31:27.587986] [INFO ] [MAIN ] BCS: Starting full primary refresh. [2025-11-04T17:31:27.948073] [INFO ] [REMOTE ] BCS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:31:28.167036] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-01. [2025-11-04T17:31:28.630378] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T17:31:28.967027] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-02. [2025-11-04T17:31:29.702429] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-11-04T17:31:29.728365] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T17:31:29.767020] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-03. [2025-11-04T17:31:30.567172] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-02. [2025-11-04T17:31:30.721689] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-11-04T17:31:31.367101] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-09. [2025-11-04T17:31:31.927973] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T17:31:32.167085] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-04. [2025-11-04T17:31:32.698208] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T17:31:32.967047] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-02. [2025-11-04T17:31:33.136256] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-11-04T17:31:33.767091] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-05. [2025-11-04T17:31:33.983056] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T17:31:34.567017] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-03. [2025-11-04T17:31:34.630221] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-11-04T17:31:35.367056] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-03. [2025-11-04T17:31:35.990451] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T17:31:36.167015] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-10. [2025-11-04T17:31:36.278123] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T17:31:36.967037] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-04. [2025-11-04T17:31:37.736579] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-11-04T17:31:37.767021] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-11. [2025-11-04T17:31:37.934214] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-11-04T17:31:38.567152] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-06. [2025-11-04T17:31:38.858286] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T17:31:38.860416] [INFO ] [REMOTE ] BCE: Full fetch complete (76080 records). [2025-11-04T17:31:38.860556] [DEBUG] [VALIDATOR ] BCE: Validating primary data. [2025-11-04T17:31:39.290934] [DEBUG] [VALIDATOR ] BCE: All expected trading days are present. [2025-11-04T17:31:39.291069] [DEBUG] [VALIDATOR ] BCE: Primary data validation passed. [2025-11-04T17:31:39.291848] [INFO ] [MAIN ] BCE: Fetched primary data is valid. [2025-11-04T17:31:39.291916] [DEBUG] [LOCAL ] BCE: Saving primary data. [2025-11-04T17:31:39.293395] [DEBUG] [LOCAL ] BCE: Trimmed 76080 records to 52217 (keeping 200 days of data) [2025-11-04T17:31:39.382251] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-05. [2025-11-04T17:31:39.534902] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-11-04T17:31:39.627975] [DEBUG] [LOCAL ] BCE: Primary data saved (52217 records). [2025-11-04T17:31:39.628096] [INFO ] [MAIN ] BCE: Full history saved. [2025-11-04T17:31:39.629023] [INFO ] [MAIN ] Update Progress: 234/2367 symbols completed. BCE took 33.2s. Estimated remaining: 4h 48m 6s. Rate: 7.4 symbols/min [2025-11-04T17:31:39.800147] [INFO ] [MAIN ] BCSF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:31:39.828702] [DEBUG] [LOCAL ] BCSF: Loaded 27789 existing records from primary storage. [2025-11-04T17:31:39.828958] [INFO ] [MAIN ] BCSF: Loaded 27789 existing records. [2025-11-04T17:31:40.010474] [WARN ] [MAIN ] BCSF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:31:40.010718] [INFO ] [MAIN ] BCSF: Wiping existing primary data before full refresh. [2025-11-04T17:31:40.010773] [INFO ] [LOCAL ] BCSF: Starting deletion of primary data. [2025-11-04T17:31:40.010964] [DEBUG] [LOCAL ] BCSF: Deleted primary data file. [2025-11-04T17:31:40.011018] [INFO ] [MAIN ] BCSF: Starting full primary refresh. [2025-11-04T17:31:40.169295] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-07. [2025-11-04T17:31:40.412408] [INFO ] [REMOTE ] BCSF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:31:40.407095] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T17:31:40.967092] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-06. [2025-11-04T17:31:41.185886] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T17:31:41.767043] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-01. [2025-11-04T17:31:41.801370] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T17:31:42.567060] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-07. [2025-11-04T17:31:42.622428] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T17:31:43.347138] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-11-04T17:31:43.367006] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-08. [2025-11-04T17:31:44.167085] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-08. [2025-11-04T17:31:44.267084] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-11-04T17:31:44.967022] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-04. [2025-11-04T17:31:45.016834] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-11-04T17:31:45.767108] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-02. [2025-11-04T17:31:46.569575] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-09. [2025-11-04T17:31:46.576548] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-11-04T17:31:46.768480] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-11-04T17:31:47.359591] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T17:31:47.366993] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-10. [2025-11-04T17:31:48.145677] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-11-04T17:31:48.166997] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-11. [2025-11-04T17:31:48.733056] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T17:31:48.734050] [INFO ] [REMOTE ] BCO: Full fetch complete (35678 records). [2025-11-04T17:31:48.734118] [DEBUG] [VALIDATOR ] BCO: Validating primary data. [2025-11-04T17:31:48.734601] [FATAL] [VALIDATOR ] BCO: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:31:48.734717] [FATAL] [MAIN ] BCO: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:31:48.734760] [WARN ] [MAIN ] BCO: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:31:48.734797] [INFO ] [LOCAL ] BCO: Starting deletion of primary data. [2025-11-04T17:31:48.734849] [DEBUG] [LOCAL ] BCO: No primary data file found to delete. [2025-11-04T17:31:48.734891] [INFO ] [LOCAL ] BCO: Starting deletion of derivative data. [2025-11-04T17:31:48.735003] [DEBUG] [LOCAL ] BCO: Deleted derivative data file. [2025-11-04T17:31:48.735069] [INFO ] [MAIN ] Update Progress: 235/2367 symbols completed. BCO took 25.1s. Estimated remaining: 4h 48m 7s. Rate: 7.4 symbols/min [2025-11-04T17:31:48.911497] [INFO ] [MAIN ] BCX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:31:48.911706] [DEBUG] [LOCAL ] BCX: No existing data file found in primary storage. [2025-11-04T17:31:48.911757] [WARN ] [MAIN ] BCX: No existing data found, performing full refresh instead. [2025-11-04T17:31:48.911869] [INFO ] [MAIN ] BCX: Wiping existing primary data before full refresh. [2025-11-04T17:31:48.911912] [INFO ] [LOCAL ] BCX: Starting deletion of primary data. [2025-11-04T17:31:48.911953] [DEBUG] [LOCAL ] BCX: No primary data file found to delete. [2025-11-04T17:31:48.911986] [INFO ] [MAIN ] BCX: Starting full primary refresh. [2025-11-04T17:31:48.974623] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-05. [2025-11-04T17:31:49.276989] [INFO ] [REMOTE ] BCX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:31:49.767072] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-01. [2025-11-04T17:31:50.567024] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-03. [2025-11-04T17:31:50.652256] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-11-04T17:31:50.680589] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-11-04T17:31:51.367079] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-06. [2025-11-04T17:31:51.487637] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-11-04T17:31:52.167103] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-04. [2025-11-04T17:31:52.694347] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T17:31:52.967087] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-07. [2025-11-04T17:31:53.412976] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T17:31:53.767076] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-09. [2025-11-04T17:31:54.373912] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T17:31:54.567046] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-02. [2025-11-04T17:31:54.745966] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T17:31:55.288146] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T17:31:55.367042] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-05. [2025-11-04T17:31:56.167017] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-10. [2025-11-04T17:31:56.305503] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T17:31:56.967042] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-06. [2025-11-04T17:31:57.212458] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-11-04T17:31:57.767027] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-08. [2025-11-04T17:31:57.859825] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T17:31:58.567083] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-03. [2025-11-04T17:31:59.244474] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T17:31:59.352676] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T17:31:59.367003] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-04. [2025-11-04T17:32:00.120702] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T17:32:00.167052] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-11. [2025-11-04T17:32:00.967062] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-09. [2025-11-04T17:32:01.203687] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T17:32:01.205310] [INFO ] [REMOTE ] BCH: Full fetch complete (41998 records). [2025-11-04T17:32:01.205384] [DEBUG] [VALIDATOR ] BCH: Validating primary data. [2025-11-04T17:32:01.631985] [DEBUG] [VALIDATOR ] BCH: All expected trading days are present. [2025-11-04T17:32:01.632075] [DEBUG] [VALIDATOR ] BCH: Primary data validation passed. [2025-11-04T17:32:01.632606] [INFO ] [MAIN ] BCH: Fetched primary data is valid. [2025-11-04T17:32:01.632682] [DEBUG] [LOCAL ] BCH: Saving primary data. [2025-11-04T17:32:01.633791] [DEBUG] [LOCAL ] BCH: Trimmed 41998 records to 29536 (keeping 200 days of data) [2025-11-04T17:32:01.771540] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-07. [2025-11-04T17:32:01.815891] [DEBUG] [LOCAL ] BCH: Primary data saved (29536 records). [2025-11-04T17:32:01.816064] [INFO ] [MAIN ] BCH: Full history saved. [2025-11-04T17:32:01.816187] [INFO ] [MAIN ] Update Progress: 236/2367 symbols completed. BCH took 38.4s. Estimated remaining: 4h 48m 44s. Rate: 7.4 symbols/min [2025-11-04T17:32:02.008517] [INFO ] [MAIN ] BDC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:32:02.033687] [DEBUG] [LOCAL ] BDC: Loaded 23484 existing records from primary storage. [2025-11-04T17:32:02.033978] [INFO ] [MAIN ] BDC: Loaded 23484 existing records. [2025-11-04T17:32:02.246768] [WARN ] [MAIN ] BDC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:32:02.247320] [INFO ] [MAIN ] BDC: Wiping existing primary data before full refresh. [2025-11-04T17:32:02.247380] [INFO ] [LOCAL ] BDC: Starting deletion of primary data. [2025-11-04T17:32:02.247631] [DEBUG] [LOCAL ] BDC: Deleted primary data file. [2025-11-04T17:32:02.247733] [INFO ] [MAIN ] BDC: Starting full primary refresh. [2025-11-04T17:32:02.587166] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-05. [2025-11-04T17:32:02.695390] [INFO ] [REMOTE ] BDC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:32:02.775916] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-11-04T17:32:02.773104] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T17:32:03.367067] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-08. [2025-11-04T17:32:03.506071] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T17:32:04.115429] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T17:32:04.166998] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-09. [2025-11-04T17:32:04.967037] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-01. [2025-11-04T17:32:05.152731] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T17:32:05.767046] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-10. [2025-11-04T17:32:05.963230] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T17:32:06.567047] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-10. [2025-11-04T17:32:06.840582] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T17:32:07.367084] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-11. [2025-11-04T17:32:08.063886] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-11-04T17:32:08.167006] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-11. [2025-11-04T17:32:08.334169] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T17:32:08.335560] [INFO ] [REMOTE ] BCSF: Full fetch complete (40474 records). [2025-11-04T17:32:08.335637] [DEBUG] [VALIDATOR ] BCSF: Validating primary data. [2025-11-04T17:32:08.720464] [DEBUG] [VALIDATOR ] BCSF: All expected trading days are present. [2025-11-04T17:32:08.720568] [DEBUG] [VALIDATOR ] BCSF: Primary data validation passed. [2025-11-04T17:32:08.721071] [INFO ] [MAIN ] BCSF: Fetched primary data is valid. [2025-11-04T17:32:08.721132] [DEBUG] [LOCAL ] BCSF: Saving primary data. [2025-11-04T17:32:08.722039] [DEBUG] [LOCAL ] BCSF: Trimmed 40474 records to 27840 (keeping 200 days of data) [2025-11-04T17:32:08.887322] [DEBUG] [LOCAL ] BCSF: Primary data saved (27840 records). [2025-11-04T17:32:08.887469] [INFO ] [MAIN ] BCSF: Full history saved. [2025-11-04T17:32:08.887547] [INFO ] [MAIN ] Update Progress: 237/2367 symbols completed. BCSF took 29.3s. Estimated remaining: 4h 48m 26s. Rate: 7.4 symbols/min [2025-11-04T17:32:08.967044] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-06. [2025-11-04T17:32:09.441595] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-11 (1.27 seconds). [2025-11-04T17:32:09.443116] [INFO ] [REMOTE ] BCS: Full fetch complete (77146 records). [2025-11-04T17:32:09.443206] [DEBUG] [VALIDATOR ] BCS: Validating primary data. [2025-11-04T17:32:09.443978] [FATAL] [VALIDATOR ] BCS: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:32:09.444085] [FATAL] [MAIN ] BCS: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:32:09.444129] [WARN ] [MAIN ] BCS: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:32:09.444165] [INFO ] [LOCAL ] BCS: Starting deletion of primary data. [2025-11-04T17:32:09.444218] [DEBUG] [LOCAL ] BCS: No primary data file found to delete. [2025-11-04T17:32:09.444252] [INFO ] [LOCAL ] BCS: Starting deletion of derivative data. [2025-11-04T17:32:09.444290] [DEBUG] [LOCAL ] BCS: No derivative data file found to delete. [2025-11-04T17:32:09.444359] [INFO ] [MAIN ] Update Progress: 238/2367 symbols completed. BCS took 42.0s. Estimated remaining: 4h 47m 10s. Rate: 7.4 symbols/min [2025-11-04T17:32:09.737572] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T17:32:09.767020] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-07. [2025-11-04T17:32:10.567050] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-02. [2025-11-04T17:32:10.588318] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T17:32:11.367082] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-08. [2025-11-04T17:32:11.474430] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T17:32:11.892202] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-11-04T17:32:12.167043] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-03. [2025-11-04T17:32:12.967076] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-09. [2025-11-04T17:32:13.305720] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T17:32:13.767061] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-04. [2025-11-04T17:32:13.789541] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-11-04T17:32:14.567124] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-10. [2025-11-04T17:32:14.787216] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T17:32:15.367049] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-05. [2025-11-04T17:32:15.530448] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-11-04T17:32:16.167084] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-11. [2025-11-04T17:32:16.357288] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T17:32:16.895996] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T17:32:16.897144] [INFO ] [REMOTE ] BCX: Full fetch complete (32747 records). [2025-11-04T17:32:16.897219] [DEBUG] [VALIDATOR ] BCX: Validating primary data. [2025-11-04T17:32:16.897909] [FATAL] [VALIDATOR ] BCX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:32:16.898007] [FATAL] [MAIN ] BCX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:32:16.898065] [WARN ] [MAIN ] BCX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:32:16.898156] [INFO ] [LOCAL ] BCX: Starting deletion of primary data. [2025-11-04T17:32:16.898232] [DEBUG] [LOCAL ] BCX: No primary data file found to delete. [2025-11-04T17:32:16.898283] [INFO ] [LOCAL ] BCX: Starting deletion of derivative data. [2025-11-04T17:32:16.898351] [DEBUG] [LOCAL ] BCX: No derivative data file found to delete. [2025-11-04T17:32:16.898419] [INFO ] [MAIN ] Update Progress: 239/2367 symbols completed. BCX took 28.2s. Estimated remaining: 4h 46m 56s. Rate: 7.4 symbols/min [2025-11-04T17:32:16.967039] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-06. [2025-11-04T17:32:17.714534] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-11-04T17:32:17.767065] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-07. [2025-11-04T17:32:18.726361] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T17:32:18.732738] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-08. [2025-11-04T17:32:19.592375] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-11-04T17:32:19.598436] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-09. [2025-11-04T17:32:20.341969] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-11-04T17:32:20.348225] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-10. [2025-11-04T17:32:21.125827] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-11-04T17:32:21.131844] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-11. [2025-11-04T17:32:21.799440] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T17:32:21.801094] [INFO ] [REMOTE ] BDC: Full fetch complete (35813 records). [2025-11-04T17:32:21.801184] [DEBUG] [VALIDATOR ] BDC: Validating primary data. [2025-11-04T17:32:22.237421] [DEBUG] [VALIDATOR ] BDC: All expected trading days are present. [2025-11-04T17:32:22.237503] [DEBUG] [VALIDATOR ] BDC: Primary data validation passed. [2025-11-04T17:32:22.237987] [INFO ] [MAIN ] BDC: Fetched primary data is valid. [2025-11-04T17:32:22.238048] [DEBUG] [LOCAL ] BDC: Saving primary data. [2025-11-04T17:32:22.238859] [DEBUG] [LOCAL ] BDC: Trimmed 35813 records to 23554 (keeping 200 days of data) [2025-11-04T17:32:22.382515] [DEBUG] [LOCAL ] BDC: Primary data saved (23554 records). [2025-11-04T17:32:22.382672] [INFO ] [MAIN ] BDC: Full history saved. [2025-11-04T17:32:22.382773] [INFO ] [MAIN ] Update Progress: 240/2367 symbols completed. BDC took 20.6s. Estimated remaining: 4h 46m 25s. Rate: 7.4 symbols/min [2025-11-04T17:32:22.888213] [INFO ] [MAIN ] BE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:32:23.075916] [INFO ] [MAIN ] BDJ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:32:23.023057] [INFO ] [MAIN ] BDX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:32:23.094376] [INFO ] [MAIN ] BDN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:32:23.094598] [DEBUG] [LOCAL ] BDN: No existing data file found in primary storage. [2025-11-04T17:32:23.094700] [WARN ] [MAIN ] BDN: No existing data found, performing full refresh instead. [2025-11-04T17:32:23.095105] [INFO ] [MAIN ] BDN: Wiping existing primary data before full refresh. [2025-11-04T17:32:23.095187] [INFO ] [LOCAL ] BDN: Starting deletion of primary data. [2025-11-04T17:32:23.095281] [DEBUG] [LOCAL ] BDN: No primary data file found to delete. [2025-11-04T17:32:23.095361] [INFO ] [MAIN ] BDN: Starting full primary refresh. [2025-11-04T17:32:23.322974] [DEBUG] [LOCAL ] BDJ: Loaded 28750 existing records from primary storage. [2025-11-04T17:32:23.378734] [DEBUG] [LOCAL ] BE: Loaded 53662 existing records from primary storage. [2025-11-04T17:32:23.388689] [INFO ] [MAIN ] BDJ: Loaded 28750 existing records. [2025-11-04T17:32:23.416517] [INFO ] [MAIN ] BE: Loaded 53662 existing records. [2025-11-04T17:32:23.561503] [DEBUG] [LOCAL ] BDX: Loaded 52021 existing records from primary storage. [2025-11-04T17:32:23.614550] [INFO ] [MAIN ] BDX: Loaded 52021 existing records. [2025-11-04T17:32:23.867620] [INFO ] [REMOTE ] BDN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:32:23.937136] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-01. [2025-11-04T17:32:23.999869] [WARN ] [MAIN ] BDJ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:32:24.078659] [INFO ] [MAIN ] BDJ: Wiping existing primary data before full refresh. [2025-11-04T17:32:24.078497] [WARN ] [MAIN ] BE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:32:24.107806] [INFO ] [MAIN ] BE: Wiping existing primary data before full refresh. [2025-11-04T17:32:24.094416] [INFO ] [LOCAL ] BDJ: Starting deletion of primary data. [2025-11-04T17:32:24.118900] [INFO ] [LOCAL ] BE: Starting deletion of primary data. [2025-11-04T17:32:24.129700] [DEBUG] [LOCAL ] BDJ: Deleted primary data file. [2025-11-04T17:32:24.151146] [WARN ] [MAIN ] BDX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:32:24.151356] [INFO ] [MAIN ] BDX: Wiping existing primary data before full refresh. [2025-11-04T17:32:24.146136] [DEBUG] [LOCAL ] BE: Deleted primary data file. [2025-11-04T17:32:24.151605] [INFO ] [MAIN ] BDJ: Starting full primary refresh. [2025-11-04T17:32:24.151535] [INFO ] [LOCAL ] BDX: Starting deletion of primary data. [2025-11-04T17:32:24.151682] [INFO ] [MAIN ] BE: Starting full primary refresh. [2025-11-04T17:32:24.157026] [DEBUG] [LOCAL ] BDX: Deleted primary data file. [2025-11-04T17:32:24.211043] [INFO ] [MAIN ] BDX: Starting full primary refresh. [2025-11-04T17:32:25.288835] [INFO ] [REMOTE ] BDJ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:32:25.346959] [INFO ] [REMOTE ] BE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:32:25.343814] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-01. [2025-11-04T17:32:25.384057] [INFO ] [REMOTE ] BDX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:32:25.385445] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-01. [2025-11-04T17:32:25.380953] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-01. [2025-11-04T17:32:25.440578] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T17:32:25.451062] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-02. [2025-11-04T17:32:26.263852] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-11-04T17:32:26.272329] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-02. [2025-11-04T17:32:26.607038] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T17:32:26.616888] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-03. [2025-11-04T17:32:26.693666] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T17:32:27.014599] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-11-04T17:32:27.143963] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T17:32:27.367053] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-03. [2025-11-04T17:32:27.909010] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-11-04T17:32:28.167062] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-02. [2025-11-04T17:32:28.343469] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T17:32:28.967112] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-04. [2025-11-04T17:32:29.543680] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-11-04T17:32:29.767030] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-02. [2025-11-04T17:32:29.788115] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T17:32:30.567074] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-04. [2025-11-04T17:32:31.223857] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-11-04T17:32:31.367065] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-05. [2025-11-04T17:32:32.052150] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-11-04T17:32:32.079605] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-11-04T17:32:32.167002] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-06. [2025-11-04T17:32:32.972141] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-03. [2025-11-04T17:32:32.983838] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-11-04T17:32:33.767089] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-07. [2025-11-04T17:32:34.397856] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-11-04T17:32:34.567019] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-04. [2025-11-04T17:32:34.709427] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-11-04T17:32:35.367087] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-05. [2025-11-04T17:32:35.913921] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T17:32:36.167085] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-05. [2025-11-04T17:32:36.698515] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T17:32:36.967084] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-06. [2025-11-04T17:32:37.595391] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T17:32:37.767050] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-03. [2025-11-04T17:32:37.944197] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T17:32:38.567078] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-08. [2025-11-04T17:32:39.367015] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-06. [2025-11-04T17:32:39.401241] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-11-04T17:32:39.555214] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-11-04T17:32:40.167039] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-09. [2025-11-04T17:32:40.734144] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-11-04T17:32:40.944072] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-11-04T17:32:40.967030] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-07. [2025-11-04T17:32:41.767120] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-10. [2025-11-04T17:32:42.567079] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-07. [2025-11-04T17:32:42.748496] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-11-04T17:32:42.793007] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-11-04T17:32:43.367087] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-08. [2025-11-04T17:32:44.009842] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:32:44.167093] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-04. [2025-11-04T17:32:44.711686] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T17:32:44.967109] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-08. [2025-11-04T17:32:45.767158] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-09. [2025-11-04T17:32:45.860849] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-11-04T17:32:46.264441] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T17:32:46.567073] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-05. [2025-11-04T17:32:47.190116] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T17:32:47.367098] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-09. [2025-11-04T17:32:47.991734] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-11-04T17:32:48.167117] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-11. [2025-11-04T17:32:48.573729] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T17:32:48.967076] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-10. [2025-11-04T17:32:49.150500] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-11 (0.98 seconds). [2025-11-04T17:32:49.151906] [INFO ] [REMOTE ] BDJ: Full fetch complete (42281 records). [2025-11-04T17:32:49.152002] [DEBUG] [VALIDATOR ] BDJ: Validating primary data. [2025-11-04T17:32:49.518564] [DEBUG] [VALIDATOR ] BDJ: All expected trading days are present. [2025-11-04T17:32:49.518661] [DEBUG] [VALIDATOR ] BDJ: Primary data validation passed. [2025-11-04T17:32:49.519156] [INFO ] [MAIN ] BDJ: Fetched primary data is valid. [2025-11-04T17:32:49.519196] [DEBUG] [LOCAL ] BDJ: Saving primary data. [2025-11-04T17:32:49.520119] [DEBUG] [LOCAL ] BDJ: Trimmed 42281 records to 28615 (keeping 200 days of data) [2025-11-04T17:32:49.681572] [DEBUG] [LOCAL ] BDJ: Primary data saved (28615 records). [2025-11-04T17:32:49.681712] [INFO ] [MAIN ] BDJ: Full history saved. [2025-11-04T17:32:49.681799] [INFO ] [MAIN ] Update Progress: 241/2367 symbols completed. BDJ took 27.3s. Estimated remaining: 4h 49m 7s. Rate: 7.4 symbols/min [2025-11-04T17:32:49.772295] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-06. [2025-11-04T17:32:49.853504] [INFO ] [MAIN ] BEDU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:32:49.853726] [DEBUG] [LOCAL ] BEDU: No existing data file found in primary storage. [2025-11-04T17:32:49.853820] [WARN ] [MAIN ] BEDU: No existing data found, performing full refresh instead. [2025-11-04T17:32:49.853903] [INFO ] [MAIN ] BEDU: Wiping existing primary data before full refresh. [2025-11-04T17:32:49.853940] [INFO ] [LOCAL ] BEDU: Starting deletion of primary data. [2025-11-04T17:32:49.853980] [DEBUG] [LOCAL ] BEDU: No primary data file found to delete. [2025-11-04T17:32:49.854012] [INFO ] [MAIN ] BEDU: Starting full primary refresh. [2025-11-04T17:32:50.204026] [INFO ] [REMOTE ] BEDU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:32:50.366248] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T17:32:50.567027] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-11. [2025-11-04T17:32:51.367045] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-01. [2025-11-04T17:32:51.505240] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-11-04T17:32:51.521157] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-11 (0.95 seconds). [2025-11-04T17:32:51.523407] [INFO ] [REMOTE ] BDN: Full fetch complete (69107 records). [2025-11-04T17:32:51.523516] [DEBUG] [VALIDATOR ] BDN: Validating primary data. [2025-11-04T17:32:51.524366] [FATAL] [VALIDATOR ] BDN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:32:51.524453] [FATAL] [MAIN ] BDN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:32:51.524495] [WARN ] [MAIN ] BDN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:32:51.524561] [INFO ] [LOCAL ] BDN: Starting deletion of primary data. [2025-11-04T17:32:51.524625] [DEBUG] [LOCAL ] BDN: No primary data file found to delete. [2025-11-04T17:32:51.524698] [INFO ] [LOCAL ] BDN: Starting deletion of derivative data. [2025-11-04T17:32:51.524754] [DEBUG] [LOCAL ] BDN: No derivative data file found to delete. [2025-11-04T17:32:51.524816] [INFO ] [MAIN ] Update Progress: 242/2367 symbols completed. BDN took 29.1s. Estimated remaining: 4h 48m 3s. Rate: 7.4 symbols/min [2025-11-04T17:32:51.690984] [INFO ] [MAIN ] BEKE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:32:51.741964] [DEBUG] [LOCAL ] BEKE: Loaded 53130 existing records from primary storage. [2025-11-04T17:32:51.742298] [INFO ] [MAIN ] BEKE: Loaded 53130 existing records. [2025-11-04T17:32:51.835164] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-01 (0.47 seconds). [2025-11-04T17:32:51.921529] [WARN ] [MAIN ] BEKE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:32:51.921779] [INFO ] [MAIN ] BEKE: Wiping existing primary data before full refresh. [2025-11-04T17:32:51.921829] [INFO ] [LOCAL ] BEKE: Starting deletion of primary data. [2025-11-04T17:32:51.922025] [DEBUG] [LOCAL ] BEKE: Deleted primary data file. [2025-11-04T17:32:51.922079] [INFO ] [MAIN ] BEKE: Starting full primary refresh. [2025-11-04T17:32:52.172359] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-10. [2025-11-04T17:32:52.273487] [INFO ] [REMOTE ] BEKE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:32:52.967091] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-01. [2025-11-04T17:32:53.767047] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-07. [2025-11-04T17:32:54.419821] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-10 (2.25 seconds). [2025-11-04T17:32:54.468677] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T17:32:54.567111] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-02. [2025-11-04T17:32:55.314462] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-11-04T17:32:55.367090] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-08. [2025-11-04T17:32:56.117593] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-11-04T17:32:56.167038] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-03. [2025-11-04T17:32:56.956878] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-11-04T17:32:56.968068] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-02. [2025-11-04T17:32:57.179080] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-02 (0.21 seconds). [2025-11-04T17:32:57.645834] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-11-04T17:32:57.767058] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-11. [2025-11-04T17:32:58.002494] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-11 (0.24 seconds). [2025-11-04T17:32:58.004286] [INFO ] [REMOTE ] BDX: Full fetch complete (74974 records). [2025-11-04T17:32:58.004431] [DEBUG] [VALIDATOR ] BDX: Validating primary data. [2025-11-04T17:32:58.006525] [FATAL] [VALIDATOR ] BDX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:32:58.006808] [FATAL] [MAIN ] BDX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:32:58.006916] [WARN ] [MAIN ] BDX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:32:58.006998] [INFO ] [LOCAL ] BDX: Starting deletion of primary data. [2025-11-04T17:32:58.007081] [DEBUG] [LOCAL ] BDX: No primary data file found to delete. [2025-11-04T17:32:58.007124] [INFO ] [LOCAL ] BDX: Starting deletion of derivative data. [2025-11-04T17:32:58.007334] [DEBUG] [LOCAL ] BDX: Deleted derivative data file. [2025-11-04T17:32:58.008555] [INFO ] [MAIN ] Update Progress: 243/2367 symbols completed. BDX took 35.6s. Estimated remaining: 4h 47m 40s. Rate: 7.4 symbols/min [2025-11-04T17:32:58.183337] [INFO ] [MAIN ] BEN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:32:58.183618] [DEBUG] [LOCAL ] BEN: No existing data file found in primary storage. [2025-11-04T17:32:58.183691] [WARN ] [MAIN ] BEN: No existing data found, performing full refresh instead. [2025-11-04T17:32:58.183824] [INFO ] [MAIN ] BEN: Wiping existing primary data before full refresh. [2025-11-04T17:32:58.183867] [INFO ] [LOCAL ] BEN: Starting deletion of primary data. [2025-11-04T17:32:58.183908] [DEBUG] [LOCAL ] BEN: No primary data file found to delete. [2025-11-04T17:32:58.183944] [INFO ] [MAIN ] BEN: Starting full primary refresh. [2025-11-04T17:32:58.555188] [INFO ] [REMOTE ] BEN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:32:58.567047] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-01. [2025-11-04T17:32:59.367080] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-09. [2025-11-04T17:32:59.861094] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T17:33:00.167070] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-03. [2025-11-04T17:33:00.624697] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-11-04T17:33:00.967036] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-04. [2025-11-04T17:33:01.501635] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-09 (2.13 seconds). [2025-11-04T17:33:01.767045] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-10. [2025-11-04T17:33:02.467356] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-11-04T17:33:02.567032] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-05. [2025-11-04T17:33:03.367050] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-02. [2025-11-04T17:33:03.863796] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T17:33:04.006218] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-10 (2.24 seconds). [2025-11-04T17:33:04.167016] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-11. [2025-11-04T17:33:04.567184] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T17:33:04.967035] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-03. [2025-11-04T17:33:05.767054] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-04. [2025-11-04T17:33:05.946965] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-11 (1.78 seconds). [2025-11-04T17:33:05.949781] [INFO ] [REMOTE ] BE: Full fetch complete (77818 records). [2025-11-04T17:33:05.949925] [DEBUG] [VALIDATOR ] BE: Validating primary data. [2025-11-04T17:33:06.105927] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-11-04T17:33:06.379393] [DEBUG] [VALIDATOR ] BE: All expected trading days are present. [2025-11-04T17:33:06.379556] [DEBUG] [VALIDATOR ] BE: Primary data validation passed. [2025-11-04T17:33:06.380372] [INFO ] [MAIN ] BE: Fetched primary data is valid. [2025-11-04T17:33:06.380479] [DEBUG] [LOCAL ] BE: Saving primary data. [2025-11-04T17:33:06.382284] [DEBUG] [LOCAL ] BE: Trimmed 77818 records to 53662 (keeping 200 days of data) [2025-11-04T17:33:06.504406] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-11-04T17:33:06.602300] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-05. [2025-11-04T17:33:06.754724] [DEBUG] [LOCAL ] BE: Primary data saved (53662 records). [2025-11-04T17:33:06.754855] [INFO ] [MAIN ] BE: Full history saved. [2025-11-04T17:33:06.754944] [INFO ] [MAIN ] Update Progress: 244/2367 symbols completed. BE took 44.4s. Estimated remaining: 4h 47m 38s. Rate: 7.4 symbols/min [2025-11-04T17:33:06.935476] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-11-04T17:33:06.935258] [INFO ] [MAIN ] BEP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:33:06.971734] [DEBUG] [LOCAL ] BEP: Loaded 38923 existing records from primary storage. [2025-11-04T17:33:06.972040] [INFO ] [MAIN ] BEP: Loaded 38923 existing records. [2025-11-04T17:33:07.160091] [WARN ] [MAIN ] BEP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:33:07.160317] [INFO ] [MAIN ] BEP: Wiping existing primary data before full refresh. [2025-11-04T17:33:07.160365] [INFO ] [LOCAL ] BEP: Starting deletion of primary data. [2025-11-04T17:33:07.160577] [DEBUG] [LOCAL ] BEP: Deleted primary data file. [2025-11-04T17:33:07.160633] [INFO ] [MAIN ] BEP: Starting full primary refresh. [2025-11-04T17:33:07.376674] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-04. [2025-11-04T17:33:07.539501] [INFO ] [REMOTE ] BEP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:33:08.167041] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-06. [2025-11-04T17:33:08.414351] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-06 (0.25 seconds). [2025-11-04T17:33:08.865348] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-11-04T17:33:08.967046] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-06. [2025-11-04T17:33:09.767032] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-05. [2025-11-04T17:33:10.391368] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-11-04T17:33:10.567057] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-07. [2025-11-04T17:33:11.045692] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T17:33:11.367042] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-06. [2025-11-04T17:33:11.999868] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-11-04T17:33:12.167027] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-08. [2025-11-04T17:33:12.645541] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T17:33:12.967056] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-07. [2025-11-04T17:33:13.767057] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-07. [2025-11-04T17:33:13.880448] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-11-04T17:33:13.957686] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-07 (0.19 seconds). [2025-11-04T17:33:14.263282] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:33:14.567044] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-08. [2025-11-04T17:33:14.748633] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-08 (0.18 seconds). [2025-11-04T17:33:15.367041] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-01. [2025-11-04T17:33:16.167032] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-09. [2025-11-04T17:33:16.691020] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T17:33:16.738124] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-11-04T17:33:16.967043] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-02. [2025-11-04T17:33:17.767051] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-09. [2025-11-04T17:33:18.078888] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T17:33:18.567039] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-03. [2025-11-04T17:33:19.265396] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T17:33:19.367105] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-10. [2025-11-04T17:33:19.637208] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T17:33:20.167108] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-08. [2025-11-04T17:33:20.892305] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-11-04T17:33:20.967024] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-11. [2025-11-04T17:33:21.685748] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-11-04T17:33:21.767093] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-10. [2025-11-04T17:33:22.042780] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-10 (0.28 seconds). [2025-11-04T17:33:22.345831] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-11 (1.38 seconds). [2025-11-04T17:33:22.349192] [INFO ] [REMOTE ] BEKE: Full fetch complete (77297 records). [2025-11-04T17:33:22.349396] [DEBUG] [VALIDATOR ] BEKE: Validating primary data. [2025-11-04T17:33:22.567354] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-04. [2025-11-04T17:33:22.819288] [DEBUG] [VALIDATOR ] BEKE: All expected trading days are present. [2025-11-04T17:33:22.819459] [DEBUG] [VALIDATOR ] BEKE: Primary data validation passed. [2025-11-04T17:33:22.820721] [INFO ] [MAIN ] BEKE: Fetched primary data is valid. [2025-11-04T17:33:22.820804] [DEBUG] [LOCAL ] BEKE: Saving primary data. [2025-11-04T17:33:22.823232] [DEBUG] [LOCAL ] BEKE: Trimmed 77297 records to 53142 (keeping 200 days of data) [2025-11-04T17:33:23.155361] [DEBUG] [LOCAL ] BEKE: Primary data saved (53142 records). [2025-11-04T17:33:23.155577] [INFO ] [MAIN ] BEKE: Full history saved. [2025-11-04T17:33:23.155703] [INFO ] [MAIN ] Update Progress: 245/2367 symbols completed. BEKE took 31.6s. Estimated remaining: 4h 48m 41s. Rate: 7.4 symbols/min [2025-11-04T17:33:23.335136] [INFO ] [MAIN ] BEPC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:33:23.367105] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-11. [2025-11-04T17:33:23.383231] [DEBUG] [LOCAL ] BEPC: Loaded 46314 existing records from primary storage. [2025-11-04T17:33:23.383779] [INFO ] [MAIN ] BEPC: Loaded 46314 existing records. [2025-11-04T17:33:23.572578] [WARN ] [MAIN ] BEPC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:33:23.572866] [INFO ] [MAIN ] BEPC: Wiping existing primary data before full refresh. [2025-11-04T17:33:23.572964] [INFO ] [LOCAL ] BEPC: Starting deletion of primary data. [2025-11-04T17:33:23.573147] [DEBUG] [LOCAL ] BEPC: Deleted primary data file. [2025-11-04T17:33:23.573186] [INFO ] [MAIN ] BEPC: Starting full primary refresh. [2025-11-04T17:33:23.778577] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T17:33:23.802281] [INFO ] [REMOTE ] BEDU: Full fetch complete (1316 records). [2025-11-04T17:33:23.807673] [DEBUG] [VALIDATOR ] BEDU: Validating primary data. [2025-11-04T17:33:24.061256] [INFO ] [REMOTE ] BEPC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:33:24.123366] [WARN ] [VALIDATOR ] BEDU: Validation failed - insufficient historical data. Have 169 unique days, need 200 days [2025-11-04T17:33:24.123599] [FATAL] [MAIN ] BEDU: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:33:24.123772] [WARN ] [MAIN ] BEDU: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:33:24.123850] [INFO ] [LOCAL ] BEDU: Starting deletion of primary data. [2025-11-04T17:33:24.123942] [DEBUG] [LOCAL ] BEDU: No primary data file found to delete. [2025-11-04T17:33:24.124079] [INFO ] [LOCAL ] BEDU: Starting deletion of derivative data. [2025-11-04T17:33:24.124239] [DEBUG] [LOCAL ] BEDU: No derivative data file found to delete. [2025-11-04T17:33:24.124326] [INFO ] [MAIN ] Update Progress: 246/2367 symbols completed. BEDU took 34.4s. Estimated remaining: 4h 47m 31s. Rate: 7.4 symbols/min [2025-11-04T17:33:24.172262] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-09. [2025-11-04T17:33:24.220031] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-11-04T17:33:24.340676] [INFO ] [MAIN ] BEPH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:33:24.347765] [DEBUG] [LOCAL ] BEPH: Loaded 4803 existing records from primary storage. [2025-11-04T17:33:24.347983] [INFO ] [MAIN ] BEPH: Loaded 4803 existing records. [2025-11-04T17:33:24.524332] [WARN ] [MAIN ] BEPH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:33:24.524543] [INFO ] [MAIN ] BEPH: Wiping existing primary data before full refresh. [2025-11-04T17:33:24.524591] [INFO ] [LOCAL ] BEPH: Starting deletion of primary data. [2025-11-04T17:33:24.524744] [DEBUG] [LOCAL ] BEPH: Deleted primary data file. [2025-11-04T17:33:24.524806] [INFO ] [MAIN ] BEPH: Starting full primary refresh. [2025-11-04T17:33:24.868024] [INFO ] [REMOTE ] BEPH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:33:24.966997] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-01. [2025-11-04T17:33:25.451915] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-01 (0.48 seconds). [2025-11-04T17:33:25.603321] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T17:33:25.767000] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-10. [2025-11-04T17:33:26.567080] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-02. [2025-11-04T17:33:26.924236] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-11-04T17:33:27.217375] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T17:33:27.367028] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-05. [2025-11-04T17:33:28.167055] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-01. [2025-11-04T17:33:28.359745] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T17:33:28.967034] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-03. [2025-11-04T17:33:29.441788] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-11-04T17:33:29.501118] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T17:33:29.767066] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-06. [2025-11-04T17:33:30.567052] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-04. [2025-11-04T17:33:30.850615] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T17:33:31.037681] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-11-04T17:33:31.367027] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-11. [2025-11-04T17:33:32.167046] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-07. [2025-11-04T17:33:32.783156] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-11 (1.42 seconds). [2025-11-04T17:33:32.784868] [INFO ] [REMOTE ] BEN: Full fetch complete (76654 records). [2025-11-04T17:33:32.784993] [DEBUG] [VALIDATOR ] BEN: Validating primary data. [2025-11-04T17:33:32.785810] [FATAL] [VALIDATOR ] BEN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:33:32.785901] [FATAL] [MAIN ] BEN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:33:32.785946] [WARN ] [MAIN ] BEN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:33:32.785987] [INFO ] [LOCAL ] BEN: Starting deletion of primary data. [2025-11-04T17:33:32.786043] [DEBUG] [LOCAL ] BEN: No primary data file found to delete. [2025-11-04T17:33:32.786079] [INFO ] [LOCAL ] BEN: Starting deletion of derivative data. [2025-11-04T17:33:32.786120] [DEBUG] [LOCAL ] BEN: No derivative data file found to delete. [2025-11-04T17:33:32.786195] [INFO ] [MAIN ] Update Progress: 247/2367 symbols completed. BEN took 34.8s. Estimated remaining: 4h 47m 27s. Rate: 7.4 symbols/min [2025-11-04T17:33:32.976927] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-05. [2025-11-04T17:33:32.997778] [INFO ] [MAIN ] BEPI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:33:33.005531] [DEBUG] [LOCAL ] BEPI: Loaded 3446 existing records from primary storage. [2025-11-04T17:33:33.005760] [INFO ] [MAIN ] BEPI: Loaded 3446 existing records. [2025-11-04T17:33:33.186078] [WARN ] [MAIN ] BEPI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:33:33.186499] [INFO ] [MAIN ] BEPI: Wiping existing primary data before full refresh. [2025-11-04T17:33:33.186677] [INFO ] [LOCAL ] BEPI: Starting deletion of primary data. [2025-11-04T17:33:33.186920] [DEBUG] [LOCAL ] BEPI: Deleted primary data file. [2025-11-04T17:33:33.187121] [INFO ] [MAIN ] BEPI: Starting full primary refresh. [2025-11-04T17:33:33.460997] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-11-04T17:33:33.505191] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T17:33:33.603464] [INFO ] [REMOTE ] BEPI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:33:33.767017] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-01. [2025-11-04T17:33:34.311018] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-11-04T17:33:34.567059] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-06. [2025-11-04T17:33:34.999316] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-11-04T17:33:35.367052] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-02. [2025-11-04T17:33:35.733338] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-11-04T17:33:36.167031] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-02. [2025-11-04T17:33:36.967075] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-03. [2025-11-04T17:33:37.265780] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T17:33:37.294854] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-11-04T17:33:37.767061] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-04. [2025-11-04T17:33:38.149028] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T17:33:38.567067] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-08. [2025-11-04T17:33:39.367075] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-05. [2025-11-04T17:33:39.756201] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-11-04T17:33:39.859554] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T17:33:40.167057] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-09. [2025-11-04T17:33:40.967094] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-06. [2025-11-04T17:33:41.205438] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-11-04T17:33:41.381590] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-11-04T17:33:41.767057] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-07. [2025-11-04T17:33:42.137139] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-11-04T17:33:42.567109] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-10. [2025-11-04T17:33:43.367051] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-08. [2025-11-04T17:33:43.689524] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-11-04T17:33:43.825786] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-11-04T17:33:44.167039] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-07. [2025-11-04T17:33:44.727104] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-11-04T17:33:44.967047] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-09. [2025-11-04T17:33:45.565585] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-11-04T17:33:45.767077] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-11. [2025-11-04T17:33:46.567082] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-03. [2025-11-04T17:33:47.046607] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-11 (1.28 seconds). [2025-11-04T17:33:47.048024] [INFO ] [REMOTE ] BEP: Full fetch complete (58759 records). [2025-11-04T17:33:47.048135] [DEBUG] [VALIDATOR ] BEP: Validating primary data. [2025-11-04T17:33:47.048908] [FATAL] [VALIDATOR ] BEP: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:33:47.049000] [FATAL] [MAIN ] BEP: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:33:47.049042] [WARN ] [MAIN ] BEP: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:33:47.049079] [INFO ] [LOCAL ] BEP: Starting deletion of primary data. [2025-11-04T17:33:47.049135] [DEBUG] [LOCAL ] BEP: No primary data file found to delete. [2025-11-04T17:33:47.049169] [INFO ] [LOCAL ] BEP: Starting deletion of derivative data. [2025-11-04T17:33:47.049293] [DEBUG] [LOCAL ] BEP: Deleted derivative data file. [2025-11-04T17:33:47.049363] [INFO ] [MAIN ] Update Progress: 248/2367 symbols completed. BEP took 40.3s. Estimated remaining: 4h 48m 12s. Rate: 7.4 symbols/min [2025-11-04T17:33:47.217061] [INFO ] [MAIN ] BEPJ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:33:47.217262] [DEBUG] [LOCAL ] BEPJ: No existing data file found in primary storage. [2025-11-04T17:33:47.217298] [WARN ] [MAIN ] BEPJ: No existing data found, performing full refresh instead. [2025-11-04T17:33:47.217407] [INFO ] [MAIN ] BEPJ: Wiping existing primary data before full refresh. [2025-11-04T17:33:47.217447] [INFO ] [LOCAL ] BEPJ: Starting deletion of primary data. [2025-11-04T17:33:47.217486] [DEBUG] [LOCAL ] BEPJ: No primary data file found to delete. [2025-11-04T17:33:47.217516] [INFO ] [MAIN ] BEPJ: Starting full primary refresh. [2025-11-04T17:33:47.374975] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-08. [2025-11-04T17:33:47.570856] [INFO ] [REMOTE ] BEPJ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:33:47.973856] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-11-04T17:33:48.004623] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T17:33:48.167055] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-10. [2025-11-04T17:33:48.544537] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-11-04T17:33:48.967038] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-04. [2025-11-04T17:33:49.767039] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-09. [2025-11-04T17:33:50.149182] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-11-04T17:33:50.177177] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T17:33:50.567095] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-05. [2025-11-04T17:33:51.367096] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-10. [2025-11-04T17:33:51.740184] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T17:33:51.782550] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-11-04T17:33:52.167037] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-06. [2025-11-04T17:33:52.967039] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-11. [2025-11-04T17:33:53.520692] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T17:33:53.521931] [INFO ] [REMOTE ] BEPH: Full fetch complete (7520 records). [2025-11-04T17:33:53.522009] [DEBUG] [VALIDATOR ] BEPH: Validating primary data. [2025-11-04T17:33:53.773893] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-01. [2025-11-04T17:33:53.898230] [DEBUG] [VALIDATOR ] BEPH: All expected trading days are present. [2025-11-04T17:33:53.898437] [DEBUG] [VALIDATOR ] BEPH: Primary data validation passed. [2025-11-04T17:33:53.898718] [INFO ] [MAIN ] BEPH: Fetched primary data is valid. [2025-11-04T17:33:53.898902] [DEBUG] [LOCAL ] BEPH: Saving primary data. [2025-11-04T17:33:53.899680] [DEBUG] [LOCAL ] BEPH: Trimmed 7520 records to 4829 (keeping 200 days of data) [2025-11-04T17:33:53.959962] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-11-04T17:33:53.972955] [DEBUG] [LOCAL ] BEPH: Primary data saved (4829 records). [2025-11-04T17:33:53.973753] [INFO ] [MAIN ] BEPH: Full history saved. [2025-11-04T17:33:53.973861] [INFO ] [MAIN ] Update Progress: 249/2367 symbols completed. BEPH took 29.8s. Estimated remaining: 4h 47m 53s. Rate: 7.4 symbols/min [2025-11-04T17:33:54.144920] [INFO ] [MAIN ] BETA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:33:54.145183] [DEBUG] [LOCAL ] BETA: No existing data file found in primary storage. [2025-11-04T17:33:54.145239] [WARN ] [MAIN ] BETA: No existing data found, performing full refresh instead. [2025-11-04T17:33:54.145352] [INFO ] [MAIN ] BETA: Wiping existing primary data before full refresh. [2025-11-04T17:33:54.145397] [INFO ] [LOCAL ] BETA: Starting deletion of primary data. [2025-11-04T17:33:54.145437] [DEBUG] [LOCAL ] BETA: No primary data file found to delete. [2025-11-04T17:33:54.145468] [INFO ] [MAIN ] BETA: Starting full primary refresh. [2025-11-04T17:33:54.475989] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-11-04T17:33:54.570305] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-02. [2025-11-04T17:33:54.574912] [INFO ] [REMOTE ] BETA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:33:54.893232] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T17:33:55.367078] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-03. [2025-11-04T17:33:55.790607] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T17:33:56.167015] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-07. [2025-11-04T17:33:56.967048] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-11. [2025-11-04T17:33:57.561710] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-11 (0.59 seconds). [2025-11-04T17:33:57.563993] [INFO ] [REMOTE ] BEPI: Full fetch complete (4762 records). [2025-11-04T17:33:57.563910] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T17:33:57.564116] [DEBUG] [VALIDATOR ] BEPI: Validating primary data. [2025-11-04T17:33:57.777396] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-08. [2025-11-04T17:33:57.931006] [DEBUG] [VALIDATOR ] BEPI: All expected trading days are present. [2025-11-04T17:33:57.931085] [DEBUG] [VALIDATOR ] BEPI: Primary data validation passed. [2025-11-04T17:33:57.931226] [INFO ] [MAIN ] BEPI: Fetched primary data is valid. [2025-11-04T17:33:57.931265] [DEBUG] [LOCAL ] BEPI: Saving primary data. [2025-11-04T17:33:57.931718] [DEBUG] [LOCAL ] BEPI: Trimmed 4762 records to 3443 (keeping 200 days of data) [2025-11-04T17:33:57.952368] [DEBUG] [LOCAL ] BEPI: Primary data saved (3443 records). [2025-11-04T17:33:57.952479] [INFO ] [MAIN ] BEPI: Full history saved. [2025-11-04T17:33:57.952566] [INFO ] [MAIN ] Update Progress: 250/2367 symbols completed. BEPI took 25.2s. Estimated remaining: 4h 47m 9s. Rate: 7.4 symbols/min [2025-11-04T17:33:58.117993] [INFO ] [MAIN ] BFAM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:33:58.156800] [DEBUG] [LOCAL ] BFAM: Loaded 34252 existing records from primary storage. [2025-11-04T17:33:58.157091] [INFO ] [MAIN ] BFAM: Loaded 34252 existing records. [2025-11-04T17:33:58.330062] [WARN ] [MAIN ] BFAM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:33:58.330325] [INFO ] [MAIN ] BFAM: Wiping existing primary data before full refresh. [2025-11-04T17:33:58.330376] [INFO ] [LOCAL ] BFAM: Starting deletion of primary data. [2025-11-04T17:33:58.330553] [DEBUG] [LOCAL ] BFAM: Deleted primary data file. [2025-11-04T17:33:58.330608] [INFO ] [MAIN ] BFAM: Starting full primary refresh. [2025-11-04T17:33:58.575552] [INFO ] [REMOTE ] BETA: Fetching primary data for 2025-01. [2025-11-04T17:33:58.681821] [INFO ] [REMOTE ] BFAM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:33:58.809859] [DEBUG] [REMOTE ] BETA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:33:58.810125] [WARN ] [REMOTE ] BETA: No data returned for month 2025-01 [2025-11-04T17:33:58.810208] [FATAL] [REMOTE ] BETA: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:33:58.810258] [FATAL] [MAIN ] BETA: Failed to fetch primary data. [2025-11-04T17:33:58.810315] [WARN ] [MAIN ] BETA: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:33:58.810385] [INFO ] [LOCAL ] BETA: Starting deletion of primary data. [2025-11-04T17:33:58.810445] [DEBUG] [LOCAL ] BETA: No primary data file found to delete. [2025-11-04T17:33:58.810479] [INFO ] [LOCAL ] BETA: Starting deletion of derivative data. [2025-11-04T17:33:58.810549] [DEBUG] [LOCAL ] BETA: No derivative data file found to delete. [2025-11-04T17:33:58.810615] [INFO ] [MAIN ] Update Progress: 251/2367 symbols completed. BETA took 4.8s. Estimated remaining: 4h 45m 60s. Rate: 7.4 symbols/min [2025-11-04T17:33:58.981981] [INFO ] [MAIN ] BFH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:33:59.017538] [DEBUG] [LOCAL ] BFH: Loaded 40812 existing records from primary storage. [2025-11-04T17:33:59.017859] [INFO ] [MAIN ] BFH: Loaded 40812 existing records. [2025-11-04T17:33:59.202405] [WARN ] [MAIN ] BFH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:33:59.202695] [INFO ] [MAIN ] BFH: Wiping existing primary data before full refresh. [2025-11-04T17:33:59.202859] [INFO ] [LOCAL ] BFH: Starting deletion of primary data. [2025-11-04T17:33:59.203307] [DEBUG] [LOCAL ] BFH: Deleted primary data file. [2025-11-04T17:33:59.203361] [INFO ] [MAIN ] BFH: Starting full primary refresh. [2025-11-04T17:33:59.332094] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-11-04T17:33:59.367471] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-04. [2025-11-04T17:33:59.606217] [INFO ] [REMOTE ] BFH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:33:59.759459] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-04 (0.39 seconds). [2025-11-04T17:34:00.167038] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-05. [2025-11-04T17:34:00.760531] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T17:34:00.967031] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-01. [2025-11-04T17:34:01.767077] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-01. [2025-11-04T17:34:02.384128] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-11-04T17:34:02.567039] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-02. [2025-11-04T17:34:03.091430] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T17:34:03.367066] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-02. [2025-11-04T17:34:03.478712] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T17:34:04.167009] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-06. [2025-11-04T17:34:04.253779] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T17:34:04.686419] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-11-04T17:34:04.967036] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-09. [2025-11-04T17:34:05.767034] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-07. [2025-11-04T17:34:06.164933] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-11-04T17:34:06.310463] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T17:34:06.567021] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-03. [2025-11-04T17:34:07.367013] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-10. [2025-11-04T17:34:07.507671] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-11-04T17:34:08.167034] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-03. [2025-11-04T17:34:08.744604] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T17:34:08.967039] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-11. [2025-11-04T17:34:09.231286] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-11-04T17:34:09.767074] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-04. [2025-11-04T17:34:10.018914] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T17:34:10.020356] [INFO ] [REMOTE ] BEPC: Full fetch complete (68482 records). [2025-11-04T17:34:10.020439] [DEBUG] [VALIDATOR ] BEPC: Validating primary data. [2025-11-04T17:34:10.021074] [FATAL] [VALIDATOR ] BEPC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:34:10.021155] [FATAL] [MAIN ] BEPC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:34:10.021197] [WARN ] [MAIN ] BEPC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:34:10.021258] [INFO ] [LOCAL ] BEPC: Starting deletion of primary data. [2025-11-04T17:34:10.021319] [DEBUG] [LOCAL ] BEPC: No primary data file found to delete. [2025-11-04T17:34:10.021352] [INFO ] [LOCAL ] BEPC: Starting deletion of derivative data. [2025-11-04T17:34:10.021488] [DEBUG] [LOCAL ] BEPC: Deleted derivative data file. [2025-11-04T17:34:10.021573] [INFO ] [MAIN ] Update Progress: 252/2367 symbols completed. BEPC took 46.9s. Estimated remaining: 4h 46m 18s. Rate: 7.4 symbols/min [2025-11-04T17:34:10.190774] [INFO ] [MAIN ] BFK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:34:10.204904] [DEBUG] [LOCAL ] BFK: Loaded 13771 existing records from primary storage. [2025-11-04T17:34:10.205097] [INFO ] [MAIN ] BFK: Loaded 13771 existing records. [2025-11-04T17:34:10.379516] [WARN ] [MAIN ] BFK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:34:10.379741] [INFO ] [MAIN ] BFK: Wiping existing primary data before full refresh. [2025-11-04T17:34:10.379794] [INFO ] [LOCAL ] BFK: Starting deletion of primary data. [2025-11-04T17:34:10.379975] [DEBUG] [LOCAL ] BFK: Deleted primary data file. [2025-11-04T17:34:10.380028] [INFO ] [MAIN ] BFK: Starting full primary refresh. [2025-11-04T17:34:10.572279] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-08. [2025-11-04T17:34:10.730060] [INFO ] [REMOTE ] BFK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:34:10.773171] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-11-04T17:34:10.944172] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-11-04T17:34:11.367055] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-05. [2025-11-04T17:34:12.167486] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-04. [2025-11-04T17:34:12.376497] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-11-04T17:34:12.967067] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-09. [2025-11-04T17:34:13.355452] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-11-04T17:34:13.570219] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T17:34:13.767047] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-05. [2025-11-04T17:34:14.567049] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-01. [2025-11-04T17:34:14.894613] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T17:34:15.367088] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-06. [2025-11-04T17:34:15.685396] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T17:34:16.167003] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-06. [2025-11-04T17:34:16.255614] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T17:34:16.967070] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-10. [2025-11-04T17:34:17.261710] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T17:34:17.388634] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-11-04T17:34:17.767077] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-02. [2025-11-04T17:34:18.252447] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-11-04T17:34:18.567035] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-07. [2025-11-04T17:34:19.367063] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-11. [2025-11-04T17:34:19.790064] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T17:34:19.790737] [INFO ] [REMOTE ] BEPJ: Full fetch complete (4146 records). [2025-11-04T17:34:19.790804] [DEBUG] [VALIDATOR ] BEPJ: Validating primary data. [2025-11-04T17:34:19.978625] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T17:34:20.176133] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-03. [2025-11-04T17:34:20.202437] [DEBUG] [VALIDATOR ] BEPJ: All expected trading days are present. [2025-11-04T17:34:20.203190] [DEBUG] [VALIDATOR ] BEPJ: Primary data validation passed. [2025-11-04T17:34:20.203363] [INFO ] [MAIN ] BEPJ: Fetched primary data is valid. [2025-11-04T17:34:20.203429] [DEBUG] [LOCAL ] BEPJ: Saving primary data. [2025-11-04T17:34:20.203860] [DEBUG] [LOCAL ] BEPJ: Trimmed 4146 records to 2980 (keeping 200 days of data) [2025-11-04T17:34:20.223969] [DEBUG] [LOCAL ] BEPJ: Primary data saved (2980 records). [2025-11-04T17:34:20.224057] [INFO ] [MAIN ] BEPJ: Full history saved. [2025-11-04T17:34:20.224122] [INFO ] [MAIN ] Update Progress: 253/2367 symbols completed. BEPJ took 33.2s. Estimated remaining: 4h 46m 27s. Rate: 7.4 symbols/min [2025-11-04T17:34:20.393438] [INFO ] [MAIN ] BFLY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:34:20.439602] [DEBUG] [LOCAL ] BFLY: Loaded 51854 existing records from primary storage. [2025-11-04T17:34:20.439947] [INFO ] [MAIN ] BFLY: Loaded 51854 existing records. [2025-11-04T17:34:20.612099] [WARN ] [MAIN ] BFLY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:34:20.612284] [INFO ] [MAIN ] BFLY: Wiping existing primary data before full refresh. [2025-11-04T17:34:20.612331] [INFO ] [LOCAL ] BFLY: Starting deletion of primary data. [2025-11-04T17:34:20.612493] [DEBUG] [LOCAL ] BFLY: Deleted primary data file. [2025-11-04T17:34:20.612531] [INFO ] [MAIN ] BFLY: Starting full primary refresh. [2025-11-04T17:34:20.914168] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T17:34:20.970784] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-08. [2025-11-04T17:34:20.992471] [INFO ] [REMOTE ] BFLY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:34:21.767107] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-01. [2025-11-04T17:34:22.024834] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-11-04T17:34:22.567086] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-09. [2025-11-04T17:34:23.367038] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-07. [2025-11-04T17:34:23.468901] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-11-04T17:34:23.691850] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T17:34:24.167027] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-04. [2025-11-04T17:34:24.267825] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T17:34:24.818304] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-11-04T17:34:24.967056] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-10. [2025-11-04T17:34:25.767067] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-05. [2025-11-04T17:34:26.067949] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T17:34:26.567474] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-08. [2025-11-04T17:34:26.569317] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-11-04T17:34:27.367085] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-02. [2025-11-04T17:34:27.760803] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T17:34:28.167029] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-06. [2025-11-04T17:34:28.690494] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-11-04T17:34:28.786533] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-11-04T17:34:28.967044] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-07. [2025-11-04T17:34:29.576368] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-11-04T17:34:29.767025] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-08. [2025-11-04T17:34:30.394366] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-11-04T17:34:30.567024] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-11. [2025-11-04T17:34:31.367030] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-03. [2025-11-04T17:34:31.682435] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T17:34:31.684109] [INFO ] [REMOTE ] BFH: Full fetch complete (60389 records). [2025-11-04T17:34:31.684192] [DEBUG] [VALIDATOR ] BFH: Validating primary data. [2025-11-04T17:34:32.065376] [DEBUG] [VALIDATOR ] BFH: All expected trading days are present. [2025-11-04T17:34:32.065460] [DEBUG] [VALIDATOR ] BFH: Primary data validation passed. [2025-11-04T17:34:32.066102] [INFO ] [MAIN ] BFH: Fetched primary data is valid. [2025-11-04T17:34:32.066185] [DEBUG] [LOCAL ] BFH: Saving primary data. [2025-11-04T17:34:32.067411] [DEBUG] [LOCAL ] BFH: Trimmed 60389 records to 40819 (keeping 200 days of data) [2025-11-04T17:34:32.167070] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-09. [2025-11-04T17:34:32.312011] [DEBUG] [LOCAL ] BFH: Primary data saved (40819 records). [2025-11-04T17:34:32.312168] [INFO ] [MAIN ] BFH: Full history saved. [2025-11-04T17:34:32.313127] [INFO ] [MAIN ] Update Progress: 254/2367 symbols completed. BFH took 33.5s. Estimated remaining: 4h 46m 52s. Rate: 7.4 symbols/min [2025-11-04T17:34:32.482447] [INFO ] [MAIN ] BFS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:34:32.494913] [DEBUG] [LOCAL ] BFS: Loaded 9555 existing records from primary storage. [2025-11-04T17:34:32.495125] [INFO ] [MAIN ] BFS: Loaded 9555 existing records. [2025-11-04T17:34:32.673507] [WARN ] [MAIN ] BFS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:34:32.673765] [INFO ] [MAIN ] BFS: Wiping existing primary data before full refresh. [2025-11-04T17:34:32.673833] [INFO ] [LOCAL ] BFS: Starting deletion of primary data. [2025-11-04T17:34:32.674203] [DEBUG] [LOCAL ] BFS: Deleted primary data file. [2025-11-04T17:34:32.674265] [INFO ] [MAIN ] BFS: Starting full primary refresh. [2025-11-04T17:34:32.942910] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-11-04T17:34:32.976184] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-09. [2025-11-04T17:34:33.105540] [INFO ] [REMOTE ] BFS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:34:33.188284] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T17:34:33.741893] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-11-04T17:34:33.766994] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-10. [2025-11-04T17:34:34.478749] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-11-04T17:34:34.567008] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-04. [2025-11-04T17:34:35.367053] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-01. [2025-11-04T17:34:36.060028] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-11-04T17:34:36.167009] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-10. [2025-11-04T17:34:36.224559] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T17:34:36.967039] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-02. [2025-11-04T17:34:37.243292] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-11-04T17:34:37.443283] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T17:34:37.767079] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-03. [2025-11-04T17:34:38.252783] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-11-04T17:34:38.567038] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-11. [2025-11-04T17:34:39.367021] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-04. [2025-11-04T17:34:39.397232] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T17:34:39.398466] [INFO ] [REMOTE ] BFK: Full fetch complete (19990 records). [2025-11-04T17:34:39.398534] [DEBUG] [VALIDATOR ] BFK: Validating primary data. [2025-11-04T17:34:39.767874] [DEBUG] [VALIDATOR ] BFK: All expected trading days are present. [2025-11-04T17:34:39.768033] [DEBUG] [VALIDATOR ] BFK: Primary data validation passed. [2025-11-04T17:34:39.768363] [INFO ] [MAIN ] BFK: Fetched primary data is valid. [2025-11-04T17:34:39.768402] [DEBUG] [LOCAL ] BFK: Saving primary data. [2025-11-04T17:34:39.769214] [DEBUG] [LOCAL ] BFK: Trimmed 19990 records to 13797 (keeping 200 days of data) [2025-11-04T17:34:39.850142] [DEBUG] [LOCAL ] BFK: Primary data saved (13797 records). [2025-11-04T17:34:39.850446] [INFO ] [MAIN ] BFK: Full history saved. [2025-11-04T17:34:39.850572] [INFO ] [MAIN ] Update Progress: 255/2367 symbols completed. BFK took 29.8s. Estimated remaining: 4h 46m 39s. Rate: 7.4 symbols/min [2025-11-04T17:34:39.947697] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-11-04T17:34:40.026687] [INFO ] [MAIN ] BFZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:34:40.026867] [DEBUG] [LOCAL ] BFZ: No existing data file found in primary storage. [2025-11-04T17:34:40.026927] [WARN ] [MAIN ] BFZ: No existing data found, performing full refresh instead. [2025-11-04T17:34:40.027060] [INFO ] [MAIN ] BFZ: Wiping existing primary data before full refresh. [2025-11-04T17:34:40.027101] [INFO ] [LOCAL ] BFZ: Starting deletion of primary data. [2025-11-04T17:34:40.027141] [DEBUG] [LOCAL ] BFZ: No primary data file found to delete. [2025-11-04T17:34:40.027171] [INFO ] [MAIN ] BFZ: Starting full primary refresh. [2025-11-04T17:34:40.172739] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-11. [2025-11-04T17:34:40.381050] [INFO ] [REMOTE ] BFZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:34:40.967073] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-01. [2025-11-04T17:34:41.427356] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T17:34:41.429113] [INFO ] [REMOTE ] BFAM: Full fetch complete (48338 records). [2025-11-04T17:34:41.429205] [DEBUG] [VALIDATOR ] BFAM: Validating primary data. [2025-11-04T17:34:41.767020] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-05. [2025-11-04T17:34:41.822942] [DEBUG] [VALIDATOR ] BFAM: All expected trading days are present. [2025-11-04T17:34:41.823174] [DEBUG] [VALIDATOR ] BFAM: Primary data validation passed. [2025-11-04T17:34:41.824280] [INFO ] [MAIN ] BFAM: Fetched primary data is valid. [2025-11-04T17:34:41.828218] [DEBUG] [LOCAL ] BFAM: Saving primary data. [2025-11-04T17:34:41.832707] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T17:34:41.836310] [DEBUG] [LOCAL ] BFAM: Trimmed 48338 records to 34341 (keeping 200 days of data) [2025-11-04T17:34:42.033479] [DEBUG] [LOCAL ] BFAM: Primary data saved (34341 records). [2025-11-04T17:34:42.033610] [INFO ] [MAIN ] BFAM: Full history saved. [2025-11-04T17:34:42.034686] [INFO ] [MAIN ] Update Progress: 256/2367 symbols completed. BFAM took 44.1s. Estimated remaining: 4h 45m 41s. Rate: 7.4 symbols/min [2025-11-04T17:34:42.198002] [INFO ] [MAIN ] BG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:34:42.244923] [DEBUG] [LOCAL ] BG: Loaded 50370 existing records from primary storage. [2025-11-04T17:34:42.245217] [INFO ] [MAIN ] BG: Loaded 50370 existing records. [2025-11-04T17:34:42.351049] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-11-04T17:34:42.428303] [WARN ] [MAIN ] BG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:34:42.428533] [INFO ] [MAIN ] BG: Wiping existing primary data before full refresh. [2025-11-04T17:34:42.428580] [INFO ] [LOCAL ] BG: Starting deletion of primary data. [2025-11-04T17:34:42.428811] [DEBUG] [LOCAL ] BG: Deleted primary data file. [2025-11-04T17:34:42.428867] [INFO ] [MAIN ] BG: Starting full primary refresh. [2025-11-04T17:34:42.568129] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-06. [2025-11-04T17:34:42.776387] [INFO ] [REMOTE ] BG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:34:43.237389] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-11-04T17:34:43.367019] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-01. [2025-11-04T17:34:44.167048] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-02. [2025-11-04T17:34:44.760084] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-11-04T17:34:44.967054] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-07. [2025-11-04T17:34:45.174373] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-11-04T17:34:45.506611] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-11-04T17:34:45.767061] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-02. [2025-11-04T17:34:46.567050] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-03. [2025-11-04T17:34:47.081685] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T17:34:47.100002] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-11-04T17:34:47.367082] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-03. [2025-11-04T17:34:48.167066] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-08. [2025-11-04T17:34:48.811503] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T17:34:48.829046] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-11-04T17:34:48.967057] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-04. [2025-11-04T17:34:49.767112] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-09. [2025-11-04T17:34:50.328827] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T17:34:50.359719] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T17:34:50.567047] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-04. [2025-11-04T17:34:51.169716] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-11-04T17:34:51.367044] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-05. [2025-11-04T17:34:52.028133] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T17:34:52.167029] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-06. [2025-11-04T17:34:52.778014] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T17:34:52.967050] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-07. [2025-11-04T17:34:53.544192] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-11-04T17:34:53.767044] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-05. [2025-11-04T17:34:54.567071] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-05. [2025-11-04T17:34:55.326266] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-11-04T17:34:55.367000] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-06. [2025-11-04T17:34:55.953733] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-11-04T17:34:56.167077] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-08. [2025-11-04T17:34:56.712437] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-11-04T17:34:56.773920] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-11-04T17:34:56.967089] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-10. [2025-11-04T17:34:57.528146] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-11-04T17:34:57.767048] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-09. [2025-11-04T17:34:58.441841] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-11-04T17:34:58.567043] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-07. [2025-11-04T17:34:59.367060] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-10. [2025-11-04T17:34:59.961162] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-11-04T17:35:00.025633] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T17:35:00.167069] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-11. [2025-11-04T17:35:00.967069] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-11. [2025-11-04T17:35:01.499958] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-11 (1.33 seconds). [2025-11-04T17:35:01.501070] [INFO ] [REMOTE ] BFZ: Full fetch complete (17869 records). [2025-11-04T17:35:01.501142] [DEBUG] [VALIDATOR ] BFZ: Validating primary data. [2025-11-04T17:35:01.773586] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-06. [2025-11-04T17:35:01.915281] [DEBUG] [VALIDATOR ] BFZ: All expected trading days are present. [2025-11-04T17:35:01.915360] [DEBUG] [VALIDATOR ] BFZ: Primary data validation passed. [2025-11-04T17:35:01.915688] [INFO ] [MAIN ] BFZ: Fetched primary data is valid. [2025-11-04T17:35:01.915748] [DEBUG] [LOCAL ] BFZ: Saving primary data. [2025-11-04T17:35:01.916345] [DEBUG] [LOCAL ] BFZ: Trimmed 17869 records to 12126 (keeping 200 days of data) [2025-11-04T17:35:01.990956] [DEBUG] [LOCAL ] BFZ: Primary data saved (12126 records). [2025-11-04T17:35:01.991102] [INFO ] [MAIN ] BFZ: Full history saved. [2025-11-04T17:35:01.991174] [INFO ] [MAIN ] Update Progress: 257/2367 symbols completed. BFZ took 22.1s. Estimated remaining: 4h 47m 11s. Rate: 7.3 symbols/min [2025-11-04T17:35:02.159254] [INFO ] [MAIN ] BGB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:35:02.180512] [DEBUG] [LOCAL ] BGB: Loaded 16950 existing records from primary storage. [2025-11-04T17:35:02.180806] [INFO ] [MAIN ] BGB: Loaded 16950 existing records. [2025-11-04T17:35:02.357056] [WARN ] [MAIN ] BGB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:35:02.357266] [INFO ] [MAIN ] BGB: Wiping existing primary data before full refresh. [2025-11-04T17:35:02.357321] [INFO ] [LOCAL ] BGB: Starting deletion of primary data. [2025-11-04T17:35:02.357551] [DEBUG] [LOCAL ] BGB: Deleted primary data file. [2025-11-04T17:35:02.357608] [INFO ] [MAIN ] BGB: Starting full primary refresh. [2025-11-04T17:35:02.440622] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-11 (1.47 seconds). [2025-11-04T17:35:02.463161] [INFO ] [REMOTE ] BFS: Full fetch complete (13222 records). [2025-11-04T17:35:02.474023] [DEBUG] [VALIDATOR ] BFS: Validating primary data. [2025-11-04T17:35:02.574531] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-08. [2025-11-04T17:35:02.947829] [INFO ] [REMOTE ] BGB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:35:03.113475] [DEBUG] [VALIDATOR ] BFS: All expected trading days are present. [2025-11-04T17:35:03.113581] [DEBUG] [VALIDATOR ] BFS: Primary data validation passed. [2025-11-04T17:35:03.113884] [INFO ] [MAIN ] BFS: Fetched primary data is valid. [2025-11-04T17:35:03.113981] [DEBUG] [LOCAL ] BFS: Saving primary data. [2025-11-04T17:35:03.115033] [DEBUG] [LOCAL ] BFS: Trimmed 13222 records to 9569 (keeping 200 days of data) [2025-11-04T17:35:03.174275] [DEBUG] [LOCAL ] BFS: Primary data saved (9569 records). [2025-11-04T17:35:03.174537] [INFO ] [MAIN ] BFS: Full history saved. [2025-11-04T17:35:03.174721] [INFO ] [MAIN ] Update Progress: 258/2367 symbols completed. BFS took 30.9s. Estimated remaining: 4h 46m 5s. Rate: 7.4 symbols/min [2025-11-04T17:35:03.333982] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-11-04T17:35:03.385002] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-07. [2025-11-04T17:35:03.431589] [INFO ] [MAIN ] BGH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:35:03.449421] [DEBUG] [LOCAL ] BGH: Loaded 16442 existing records from primary storage. [2025-11-04T17:35:03.449689] [INFO ] [MAIN ] BGH: Loaded 16442 existing records. [2025-11-04T17:35:03.639396] [WARN ] [MAIN ] BGH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:35:03.639607] [INFO ] [MAIN ] BGH: Wiping existing primary data before full refresh. [2025-11-04T17:35:03.639684] [INFO ] [LOCAL ] BGH: Starting deletion of primary data. [2025-11-04T17:35:03.639863] [DEBUG] [LOCAL ] BGH: Deleted primary data file. [2025-11-04T17:35:03.639903] [INFO ] [MAIN ] BGH: Starting full primary refresh. [2025-11-04T17:35:04.003379] [INFO ] [REMOTE ] BGH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:35:04.167026] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-01. [2025-11-04T17:35:04.215011] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-11-04T17:35:04.831948] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T17:35:04.967039] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-08. [2025-11-04T17:35:05.225032] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T17:35:05.767074] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-09. [2025-11-04T17:35:06.473593] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T17:35:06.567027] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-09. [2025-11-04T17:35:07.259692] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-11-04T17:35:07.367035] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-01. [2025-11-04T17:35:07.823672] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T17:35:08.167055] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-02. [2025-11-04T17:35:08.231682] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T17:35:08.891859] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T17:35:08.967061] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-03. [2025-11-04T17:35:09.594570] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-11-04T17:35:09.767047] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-10. [2025-11-04T17:35:10.567046] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-02. [2025-11-04T17:35:11.089088] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T17:35:11.112978] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T17:35:11.367040] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-03. [2025-11-04T17:35:12.061913] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-11-04T17:35:12.167044] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-04. [2025-11-04T17:35:12.967051] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-11. [2025-11-04T17:35:13.109975] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-11-04T17:35:13.230618] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-11 (0.26 seconds). [2025-11-04T17:35:13.233420] [INFO ] [REMOTE ] BG: Full fetch complete (73004 records). [2025-11-04T17:35:13.233551] [DEBUG] [VALIDATOR ] BG: Validating primary data. [2025-11-04T17:35:13.645802] [DEBUG] [VALIDATOR ] BG: All expected trading days are present. [2025-11-04T17:35:13.645888] [DEBUG] [VALIDATOR ] BG: Primary data validation passed. [2025-11-04T17:35:13.646603] [INFO ] [MAIN ] BG: Fetched primary data is valid. [2025-11-04T17:35:13.646667] [DEBUG] [LOCAL ] BG: Saving primary data. [2025-11-04T17:35:13.648152] [DEBUG] [LOCAL ] BG: Trimmed 73004 records to 50399 (keeping 200 days of data) [2025-11-04T17:35:13.767150] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-04. [2025-11-04T17:35:13.964222] [DEBUG] [LOCAL ] BG: Primary data saved (50399 records). [2025-11-04T17:35:13.964390] [INFO ] [MAIN ] BG: Full history saved. [2025-11-04T17:35:13.965685] [INFO ] [MAIN ] Update Progress: 259/2367 symbols completed. BG took 31.9s. Estimated remaining: 4h 46m 19s. Rate: 7.4 symbols/min [2025-11-04T17:35:14.154565] [INFO ] [MAIN ] BGR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:35:14.170212] [DEBUG] [LOCAL ] BGR: Loaded 12841 existing records from primary storage. [2025-11-04T17:35:14.170453] [INFO ] [MAIN ] BGR: Loaded 12841 existing records. [2025-11-04T17:35:14.347729] [WARN ] [MAIN ] BGR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:35:14.347945] [INFO ] [MAIN ] BGR: Wiping existing primary data before full refresh. [2025-11-04T17:35:14.348017] [INFO ] [LOCAL ] BGR: Starting deletion of primary data. [2025-11-04T17:35:14.348263] [DEBUG] [LOCAL ] BGR: Deleted primary data file. [2025-11-04T17:35:14.348323] [INFO ] [MAIN ] BGR: Starting full primary refresh. [2025-11-04T17:35:14.572918] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-10. [2025-11-04T17:35:14.757226] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-11-04T17:35:14.761587] [INFO ] [REMOTE ] BGR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:35:15.367049] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-01. [2025-11-04T17:35:16.167034] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-05. [2025-11-04T17:35:16.275003] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-11-04T17:35:16.528926] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T17:35:16.967012] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-11. [2025-11-04T17:35:16.989301] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T17:35:17.767018] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-06. [2025-11-04T17:35:18.367093] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-11-04T17:35:18.447306] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-11 (1.48 seconds). [2025-11-04T17:35:18.448900] [INFO ] [REMOTE ] BFLY: Full fetch complete (75135 records). [2025-11-04T17:35:18.449005] [DEBUG] [VALIDATOR ] BFLY: Validating primary data. [2025-11-04T17:35:18.449761] [FATAL] [VALIDATOR ] BFLY: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:35:18.449845] [FATAL] [MAIN ] BFLY: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:35:18.449890] [WARN ] [MAIN ] BFLY: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:35:18.449930] [INFO ] [LOCAL ] BFLY: Starting deletion of primary data. [2025-11-04T17:35:18.450097] [DEBUG] [LOCAL ] BFLY: No primary data file found to delete. [2025-11-04T17:35:18.450153] [INFO ] [LOCAL ] BFLY: Starting deletion of derivative data. [2025-11-04T17:35:18.450308] [DEBUG] [LOCAL ] BFLY: Deleted derivative data file. [2025-11-04T17:35:18.450397] [INFO ] [MAIN ] Update Progress: 260/2367 symbols completed. BFLY took 58.2s. Estimated remaining: 4h 45m 41s. Rate: 7.4 symbols/min [2025-11-04T17:35:18.572147] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-02. [2025-11-04T17:35:18.625866] [INFO ] [MAIN ] BGS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:35:18.626010] [DEBUG] [LOCAL ] BGS: No existing data file found in primary storage. [2025-11-04T17:35:18.626050] [WARN ] [MAIN ] BGS: No existing data found, performing full refresh instead. [2025-11-04T17:35:18.626192] [INFO ] [MAIN ] BGS: Wiping existing primary data before full refresh. [2025-11-04T17:35:18.626237] [INFO ] [LOCAL ] BGS: Starting deletion of primary data. [2025-11-04T17:35:18.626277] [DEBUG] [LOCAL ] BGS: No primary data file found to delete. [2025-11-04T17:35:18.626309] [INFO ] [MAIN ] BGS: Starting full primary refresh. [2025-11-04T17:35:18.989167] [INFO ] [REMOTE ] BGS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:35:19.354065] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-11-04T17:35:19.366992] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-03. [2025-11-04T17:35:19.972344] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-11-04T17:35:20.167023] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-07. [2025-11-04T17:35:20.893103] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-11-04T17:35:20.967007] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-08. [2025-11-04T17:35:21.686726] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-11-04T17:35:21.767017] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-09. [2025-11-04T17:35:22.420442] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-11-04T17:35:22.567024] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-01. [2025-11-04T17:35:23.367033] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-10. [2025-11-04T17:35:23.713202] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T17:35:24.062339] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-11-04T17:35:24.167041] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-02. [2025-11-04T17:35:24.967086] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-05. [2025-11-04T17:35:25.499146] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T17:35:25.681785] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-11-04T17:35:25.767020] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-06. [2025-11-04T17:35:26.427753] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-11-04T17:35:26.567013] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-07. [2025-11-04T17:35:27.224159] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-11-04T17:35:27.367065] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-04. [2025-11-04T17:35:28.167037] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-08. [2025-11-04T17:35:28.238752] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-11-04T17:35:28.836365] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T17:35:28.967057] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-05. [2025-11-04T17:35:29.608219] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T17:35:29.767053] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-06. [2025-11-04T17:35:30.500050] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-11-04T17:35:30.567101] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-09. [2025-11-04T17:35:31.259520] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-11-04T17:35:31.367077] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-10. [2025-11-04T17:35:32.076212] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-11-04T17:35:32.167060] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-03. [2025-11-04T17:35:32.967097] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-11. [2025-11-04T17:35:33.741899] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-11-04T17:35:33.766998] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-04. [2025-11-04T17:35:33.801731] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T17:35:33.803055] [INFO ] [REMOTE ] BGB: Full fetch complete (26138 records). [2025-11-04T17:35:33.803153] [DEBUG] [VALIDATOR ] BGB: Validating primary data. [2025-11-04T17:35:34.170041] [DEBUG] [VALIDATOR ] BGB: All expected trading days are present. [2025-11-04T17:35:34.170132] [DEBUG] [VALIDATOR ] BGB: Primary data validation passed. [2025-11-04T17:35:34.170505] [INFO ] [MAIN ] BGB: Fetched primary data is valid. [2025-11-04T17:35:34.170545] [DEBUG] [LOCAL ] BGB: Saving primary data. [2025-11-04T17:35:34.171161] [DEBUG] [LOCAL ] BGB: Trimmed 26138 records to 16888 (keeping 200 days of data) [2025-11-04T17:35:34.271353] [DEBUG] [LOCAL ] BGB: Primary data saved (16888 records). [2025-11-04T17:35:34.271492] [INFO ] [MAIN ] BGB: Full history saved. [2025-11-04T17:35:34.271569] [INFO ] [MAIN ] Update Progress: 261/2367 symbols completed. BGB took 32.3s. Estimated remaining: 4h 46m 35s. Rate: 7.3 symbols/min [2025-11-04T17:35:34.460226] [INFO ] [MAIN ] BGSF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:35:34.470022] [DEBUG] [LOCAL ] BGSF: Loaded 8446 existing records from primary storage. [2025-11-04T17:35:34.470206] [INFO ] [MAIN ] BGSF: Loaded 8446 existing records. [2025-11-04T17:35:34.569353] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-07. [2025-11-04T17:35:34.646138] [WARN ] [MAIN ] BGSF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:35:34.646334] [INFO ] [MAIN ] BGSF: Wiping existing primary data before full refresh. [2025-11-04T17:35:34.646380] [INFO ] [LOCAL ] BGSF: Starting deletion of primary data. [2025-11-04T17:35:34.646573] [DEBUG] [LOCAL ] BGSF: Deleted primary data file. [2025-11-04T17:35:34.646628] [INFO ] [MAIN ] BGSF: Starting full primary refresh. [2025-11-04T17:35:35.025673] [INFO ] [REMOTE ] BGSF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:35:35.062067] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T17:35:35.182699] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-11-04T17:35:35.367058] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-11. [2025-11-04T17:35:36.055558] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-11 (0.69 seconds). [2025-11-04T17:35:36.056484] [INFO ] [REMOTE ] BGH: Full fetch complete (23064 records). [2025-11-04T17:35:36.056556] [DEBUG] [VALIDATOR ] BGH: Validating primary data. [2025-11-04T17:35:36.172224] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-08. [2025-11-04T17:35:36.427867] [DEBUG] [VALIDATOR ] BGH: All expected trading days are present. [2025-11-04T17:35:36.427944] [DEBUG] [VALIDATOR ] BGH: Primary data validation passed. [2025-11-04T17:35:36.428311] [INFO ] [MAIN ] BGH: Fetched primary data is valid. [2025-11-04T17:35:36.428351] [DEBUG] [LOCAL ] BGH: Saving primary data. [2025-11-04T17:35:36.429023] [DEBUG] [LOCAL ] BGH: Trimmed 23064 records to 16365 (keeping 200 days of data) [2025-11-04T17:35:36.524670] [DEBUG] [LOCAL ] BGH: Primary data saved (16365 records). [2025-11-04T17:35:36.524805] [INFO ] [MAIN ] BGH: Full history saved. [2025-11-04T17:35:36.524881] [INFO ] [MAIN ] Update Progress: 262/2367 symbols completed. BGH took 33.3s. Estimated remaining: 4h 45m 39s. Rate: 7.4 symbols/min [2025-11-04T17:35:36.692027] [INFO ] [MAIN ] BGSI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:35:36.692170] [DEBUG] [LOCAL ] BGSI: No existing data file found in primary storage. [2025-11-04T17:35:36.692224] [WARN ] [MAIN ] BGSI: No existing data found, performing full refresh instead. [2025-11-04T17:35:36.692331] [INFO ] [MAIN ] BGSI: Wiping existing primary data before full refresh. [2025-11-04T17:35:36.692372] [INFO ] [LOCAL ] BGSI: Starting deletion of primary data. [2025-11-04T17:35:36.692413] [DEBUG] [LOCAL ] BGSI: No primary data file found to delete. [2025-11-04T17:35:36.692445] [INFO ] [MAIN ] BGSI: Starting full primary refresh. [2025-11-04T17:35:36.967305] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-11-04T17:35:36.972636] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-01. [2025-11-04T17:35:37.088672] [INFO ] [REMOTE ] BGSI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:35:37.694250] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-11-04T17:35:37.767117] [INFO ] [REMOTE ] BGSI: Fetching primary data for 2025-01. [2025-11-04T17:35:37.917429] [DEBUG] [REMOTE ] BGSI: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:35:37.917863] [WARN ] [REMOTE ] BGSI: No data returned for month 2025-01 [2025-11-04T17:35:37.917976] [FATAL] [REMOTE ] BGSI: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:35:37.918046] [FATAL] [MAIN ] BGSI: Failed to fetch primary data. [2025-11-04T17:35:37.918140] [WARN ] [MAIN ] BGSI: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:35:37.918227] [INFO ] [LOCAL ] BGSI: Starting deletion of primary data. [2025-11-04T17:35:37.918306] [DEBUG] [LOCAL ] BGSI: No primary data file found to delete. [2025-11-04T17:35:37.918356] [INFO ] [LOCAL ] BGSI: Starting deletion of derivative data. [2025-11-04T17:35:37.918411] [DEBUG] [LOCAL ] BGSI: No derivative data file found to delete. [2025-11-04T17:35:37.918472] [INFO ] [MAIN ] Update Progress: 263/2367 symbols completed. BGSI took 1.4s. Estimated remaining: 4h 44m 37s. Rate: 7.4 symbols/min [2025-11-04T17:35:38.168029] [INFO ] [MAIN ] BGT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:35:38.168229] [DEBUG] [LOCAL ] BGT: No existing data file found in primary storage. [2025-11-04T17:35:38.168268] [WARN ] [MAIN ] BGT: No existing data found, performing full refresh instead. [2025-11-04T17:35:38.168381] [INFO ] [MAIN ] BGT: Wiping existing primary data before full refresh. [2025-11-04T17:35:38.168435] [INFO ] [LOCAL ] BGT: Starting deletion of primary data. [2025-11-04T17:35:38.168519] [DEBUG] [LOCAL ] BGT: No primary data file found to delete. [2025-11-04T17:35:38.168587] [INFO ] [MAIN ] BGT: Starting full primary refresh. [2025-11-04T17:35:38.575950] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-09. [2025-11-04T17:35:38.614085] [INFO ] [REMOTE ] BGT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:35:39.367034] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-05. [2025-11-04T17:35:39.413260] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-11-04T17:35:40.167058] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-10. [2025-11-04T17:35:40.780318] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T17:35:40.802296] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-11-04T17:35:40.967081] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-01. [2025-11-04T17:35:41.767063] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-11. [2025-11-04T17:35:42.159036] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T17:35:42.275860] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-11 (0.51 seconds). [2025-11-04T17:35:42.277003] [INFO ] [REMOTE ] BGR: Full fetch complete (19834 records). [2025-11-04T17:35:42.277111] [DEBUG] [VALIDATOR ] BGR: Validating primary data. [2025-11-04T17:35:42.573178] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-02. [2025-11-04T17:35:42.705091] [DEBUG] [VALIDATOR ] BGR: All expected trading days are present. [2025-11-04T17:35:42.705165] [DEBUG] [VALIDATOR ] BGR: Primary data validation passed. [2025-11-04T17:35:42.705476] [INFO ] [MAIN ] BGR: Fetched primary data is valid. [2025-11-04T17:35:42.705514] [DEBUG] [LOCAL ] BGR: Saving primary data. [2025-11-04T17:35:42.706281] [DEBUG] [LOCAL ] BGR: Trimmed 19834 records to 12833 (keeping 200 days of data) [2025-11-04T17:35:42.781826] [DEBUG] [LOCAL ] BGR: Primary data saved (12833 records). [2025-11-04T17:35:42.781962] [INFO ] [MAIN ] BGR: Full history saved. [2025-11-04T17:35:42.782041] [INFO ] [MAIN ] Update Progress: 264/2367 symbols completed. BGR took 28.8s. Estimated remaining: 4h 44m 3s. Rate: 7.4 symbols/min [2025-11-04T17:35:42.954613] [INFO ] [MAIN ] BGX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:35:42.966806] [DEBUG] [LOCAL ] BGX: Loaded 11123 existing records from primary storage. [2025-11-04T17:35:42.967024] [INFO ] [MAIN ] BGX: Loaded 11123 existing records. [2025-11-04T17:35:43.144269] [WARN ] [MAIN ] BGX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:35:43.144606] [INFO ] [MAIN ] BGX: Wiping existing primary data before full refresh. [2025-11-04T17:35:43.144747] [INFO ] [LOCAL ] BGX: Starting deletion of primary data. [2025-11-04T17:35:43.144942] [DEBUG] [LOCAL ] BGX: Deleted primary data file. [2025-11-04T17:35:43.144998] [INFO ] [MAIN ] BGX: Starting full primary refresh. [2025-11-04T17:35:43.284502] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T17:35:43.372142] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-02. [2025-11-04T17:35:43.516886] [INFO ] [REMOTE ] BGX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:35:43.724434] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-11-04T17:35:44.167080] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-01. [2025-11-04T17:35:44.967012] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-03. [2025-11-04T17:35:45.059611] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-11-04T17:35:45.356220] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-11-04T17:35:45.767101] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-03. [2025-11-04T17:35:46.390598] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-11-04T17:35:46.567092] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-06. [2025-11-04T17:35:47.367055] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-02. [2025-11-04T17:35:47.951832] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-11-04T17:35:47.951196] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-11-04T17:35:48.167077] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-03. [2025-11-04T17:35:48.810955] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T17:35:48.967068] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-07. [2025-11-04T17:35:49.767008] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-04. [2025-11-04T17:35:50.149771] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T17:35:50.153129] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-11-04T17:35:50.567049] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-04. [2025-11-04T17:35:51.095619] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T17:35:51.367042] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-05. [2025-11-04T17:35:51.959480] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T17:35:52.167053] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-06. [2025-11-04T17:35:52.672328] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T17:35:52.967036] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-05. [2025-11-04T17:35:53.283226] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-11-04T17:35:53.767095] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-06. [2025-11-04T17:35:54.355058] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-11-04T17:35:54.567098] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-08. [2025-11-04T17:35:55.367084] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-07. [2025-11-04T17:35:55.817357] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T17:35:55.916717] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-11-04T17:35:56.167052] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-09. [2025-11-04T17:35:56.967047] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-07. [2025-11-04T17:35:57.505356] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T17:35:57.630310] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-11-04T17:35:57.767049] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-04. [2025-11-04T17:35:58.567034] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-10. [2025-11-04T17:35:58.679715] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-11-04T17:35:59.367040] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-08. [2025-11-04T17:35:59.736608] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-11-04T17:35:59.890699] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T17:36:00.167077] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-11. [2025-11-04T17:36:00.967067] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-08. [2025-11-04T17:36:01.421790] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-11-04T17:36:01.439251] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-11 (1.27 seconds). [2025-11-04T17:36:01.441746] [INFO ] [REMOTE ] BGS: Full fetch complete (72754 records). [2025-11-04T17:36:01.441881] [DEBUG] [VALIDATOR ] BGS: Validating primary data. [2025-11-04T17:36:01.772228] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-05. [2025-11-04T17:36:01.924612] [DEBUG] [VALIDATOR ] BGS: All expected trading days are present. [2025-11-04T17:36:01.924734] [DEBUG] [VALIDATOR ] BGS: Primary data validation passed. [2025-11-04T17:36:01.925501] [INFO ] [MAIN ] BGS: Fetched primary data is valid. [2025-11-04T17:36:01.925546] [DEBUG] [LOCAL ] BGS: Saving primary data. [2025-11-04T17:36:01.927075] [DEBUG] [LOCAL ] BGS: Trimmed 72754 records to 50527 (keeping 200 days of data) [2025-11-04T17:36:02.219665] [DEBUG] [LOCAL ] BGS: Primary data saved (50527 records). [2025-11-04T17:36:02.219799] [INFO ] [MAIN ] BGS: Full history saved. [2025-11-04T17:36:02.219876] [INFO ] [MAIN ] Update Progress: 265/2367 symbols completed. BGS took 43.8s. Estimated remaining: 4h 45m 25s. Rate: 7.4 symbols/min [2025-11-04T17:36:02.394398] [INFO ] [MAIN ] BGY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:36:02.415469] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T17:36:02.426920] [DEBUG] [LOCAL ] BGY: Loaded 21337 existing records from primary storage. [2025-11-04T17:36:02.427202] [INFO ] [MAIN ] BGY: Loaded 21337 existing records. [2025-11-04T17:36:02.572122] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-09. [2025-11-04T17:36:02.609332] [WARN ] [MAIN ] BGY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:36:02.609586] [INFO ] [MAIN ] BGY: Wiping existing primary data before full refresh. [2025-11-04T17:36:02.609683] [INFO ] [LOCAL ] BGY: Starting deletion of primary data. [2025-11-04T17:36:02.609922] [DEBUG] [LOCAL ] BGY: Deleted primary data file. [2025-11-04T17:36:02.609991] [INFO ] [MAIN ] BGY: Starting full primary refresh. [2025-11-04T17:36:02.980431] [INFO ] [REMOTE ] BGY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:36:03.232938] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-11-04T17:36:03.367047] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-01. [2025-11-04T17:36:04.167047] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-06. [2025-11-04T17:36:04.465499] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-11-04T17:36:04.799430] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-11-04T17:36:04.967026] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-07. [2025-11-04T17:36:05.637984] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T17:36:05.767083] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-08. [2025-11-04T17:36:06.480369] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T17:36:06.567053] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-10. [2025-11-04T17:36:07.307868] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-11-04T17:36:07.366994] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-11. [2025-11-04T17:36:08.034724] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T17:36:08.035676] [INFO ] [REMOTE ] BGX: Full fetch complete (16569 records). [2025-11-04T17:36:08.035749] [DEBUG] [VALIDATOR ] BGX: Validating primary data. [2025-11-04T17:36:08.174010] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-09. [2025-11-04T17:36:08.394705] [DEBUG] [VALIDATOR ] BGX: All expected trading days are present. [2025-11-04T17:36:08.394782] [DEBUG] [VALIDATOR ] BGX: Primary data validation passed. [2025-11-04T17:36:08.395067] [INFO ] [MAIN ] BGX: Fetched primary data is valid. [2025-11-04T17:36:08.395105] [DEBUG] [LOCAL ] BGX: Saving primary data. [2025-11-04T17:36:08.395581] [DEBUG] [LOCAL ] BGX: Trimmed 16569 records to 11182 (keeping 200 days of data) [2025-11-04T17:36:08.460581] [DEBUG] [LOCAL ] BGX: Primary data saved (11182 records). [2025-11-04T17:36:08.460725] [INFO ] [MAIN ] BGX: Full history saved. [2025-11-04T17:36:08.460809] [INFO ] [MAIN ] Update Progress: 266/2367 symbols completed. BGX took 25.7s. Estimated remaining: 4h 45m 2s. Rate: 7.4 symbols/min [2025-11-04T17:36:08.626453] [INFO ] [MAIN ] BH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:36:08.643425] [DEBUG] [LOCAL ] BH: Loaded 16756 existing records from primary storage. [2025-11-04T17:36:08.643696] [INFO ] [MAIN ] BH: Loaded 16756 existing records. [2025-11-04T17:36:08.816462] [WARN ] [MAIN ] BH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:36:08.816694] [INFO ] [MAIN ] BH: Wiping existing primary data before full refresh. [2025-11-04T17:36:08.816752] [INFO ] [LOCAL ] BH: Starting deletion of primary data. [2025-11-04T17:36:08.816957] [DEBUG] [LOCAL ] BH: Deleted primary data file. [2025-11-04T17:36:08.817014] [INFO ] [MAIN ] BH: Starting full primary refresh. [2025-11-04T17:36:08.971204] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-02. [2025-11-04T17:36:09.173146] [INFO ] [REMOTE ] BH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:36:09.190136] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T17:36:09.691898] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T17:36:09.767070] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-01. [2025-11-04T17:36:10.567062] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-09. [2025-11-04T17:36:10.856799] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-11-04T17:36:11.355670] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T17:36:11.366990] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-10. [2025-11-04T17:36:12.045174] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-11-04T17:36:12.167037] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-11. [2025-11-04T17:36:12.778841] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T17:36:12.779925] [INFO ] [REMOTE ] BGT: Full fetch complete (23080 records). [2025-11-04T17:36:12.779995] [DEBUG] [VALIDATOR ] BGT: Validating primary data. [2025-11-04T17:36:12.972212] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-03. [2025-11-04T17:36:13.140255] [DEBUG] [VALIDATOR ] BGT: All expected trading days are present. [2025-11-04T17:36:13.140326] [DEBUG] [VALIDATOR ] BGT: Primary data validation passed. [2025-11-04T17:36:13.140679] [INFO ] [MAIN ] BGT: Fetched primary data is valid. [2025-11-04T17:36:13.140738] [DEBUG] [LOCAL ] BGT: Saving primary data. [2025-11-04T17:36:13.141289] [DEBUG] [LOCAL ] BGT: Trimmed 23080 records to 15834 (keeping 200 days of data) [2025-11-04T17:36:13.235563] [DEBUG] [LOCAL ] BGT: Primary data saved (15834 records). [2025-11-04T17:36:13.235719] [INFO ] [MAIN ] BGT: Full history saved. [2025-11-04T17:36:13.235803] [INFO ] [MAIN ] Update Progress: 267/2367 symbols completed. BGT took 35.3s. Estimated remaining: 4h 44m 27s. Rate: 7.4 symbols/min [2025-11-04T17:36:13.401351] [INFO ] [MAIN ] BHC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:36:13.443015] [DEBUG] [LOCAL ] BHC: Loaded 46417 existing records from primary storage. [2025-11-04T17:36:13.443289] [INFO ] [MAIN ] BHC: Loaded 46417 existing records. [2025-11-04T17:36:13.616617] [WARN ] [MAIN ] BHC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:36:13.616891] [INFO ] [MAIN ] BHC: Wiping existing primary data before full refresh. [2025-11-04T17:36:13.616947] [INFO ] [LOCAL ] BHC: Starting deletion of primary data. [2025-11-04T17:36:13.617120] [DEBUG] [LOCAL ] BHC: Deleted primary data file. [2025-11-04T17:36:13.617156] [INFO ] [MAIN ] BHC: Starting full primary refresh. [2025-11-04T17:36:13.767024] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-02. [2025-11-04T17:36:14.033161] [INFO ] [REMOTE ] BHC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:36:14.034734] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-11-04T17:36:14.200787] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-11-04T17:36:14.567143] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-03. [2025-11-04T17:36:15.092022] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-11-04T17:36:15.367036] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-01. [2025-11-04T17:36:16.167065] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-04. [2025-11-04T17:36:16.760537] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-11-04T17:36:16.967010] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-10. [2025-11-04T17:36:17.029323] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T17:36:17.612346] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-11-04T17:36:17.767053] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-02. [2025-11-04T17:36:18.567090] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-05. [2025-11-04T17:36:18.912680] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T17:36:19.303220] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-11-04T17:36:19.367031] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-06. [2025-11-04T17:36:20.115629] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-11-04T17:36:20.167050] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-04. [2025-11-04T17:36:20.692421] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T17:36:20.967032] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-05. [2025-11-04T17:36:21.716808] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-11-04T17:36:21.767078] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-03. [2025-11-04T17:36:22.567051] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-11. [2025-11-04T17:36:22.904248] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T17:36:23.292309] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T17:36:23.293357] [INFO ] [REMOTE ] BGSF: Full fetch complete (9897 records). [2025-11-04T17:36:23.293439] [DEBUG] [VALIDATOR ] BGSF: Validating primary data. [2025-11-04T17:36:23.372220] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-04. [2025-11-04T17:36:23.723367] [DEBUG] [VALIDATOR ] BGSF: All expected trading days are present. [2025-11-04T17:36:23.723539] [DEBUG] [VALIDATOR ] BGSF: Primary data validation passed. [2025-11-04T17:36:23.723930] [INFO ] [MAIN ] BGSF: Fetched primary data is valid. [2025-11-04T17:36:23.724004] [DEBUG] [LOCAL ] BGSF: Saving primary data. [2025-11-04T17:36:23.724600] [DEBUG] [LOCAL ] BGSF: Trimmed 9897 records to 8490 (keeping 200 days of data) [2025-11-04T17:36:23.781395] [DEBUG] [LOCAL ] BGSF: Primary data saved (8490 records). [2025-11-04T17:36:23.781568] [INFO ] [MAIN ] BGSF: Full history saved. [2025-11-04T17:36:23.781697] [INFO ] [MAIN ] Update Progress: 268/2367 symbols completed. BGSF took 49.5s. Estimated remaining: 4h 44m 38s. Rate: 7.4 symbols/min [2025-11-04T17:36:23.983027] [INFO ] [MAIN ] BHE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:36:24.005958] [DEBUG] [LOCAL ] BHE: Loaded 23165 existing records from primary storage. [2025-11-04T17:36:24.006420] [INFO ] [MAIN ] BHE: Loaded 23165 existing records. [2025-11-04T17:36:24.175223] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-06. [2025-11-04T17:36:24.180336] [WARN ] [MAIN ] BHE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:36:24.181742] [INFO ] [MAIN ] BHE: Wiping existing primary data before full refresh. [2025-11-04T17:36:24.181837] [INFO ] [LOCAL ] BHE: Starting deletion of primary data. [2025-11-04T17:36:24.182075] [DEBUG] [LOCAL ] BHE: Deleted primary data file. [2025-11-04T17:36:24.182119] [INFO ] [MAIN ] BHE: Starting full primary refresh. [2025-11-04T17:36:24.551149] [INFO ] [REMOTE ] BHE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:36:24.665050] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T17:36:24.967012] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-05. [2025-11-04T17:36:25.010084] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T17:36:25.767083] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-07. [2025-11-04T17:36:26.218735] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-11-04T17:36:26.534869] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T17:36:26.567022] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-08. [2025-11-04T17:36:27.367020] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-07. [2025-11-04T17:36:27.472972] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-11-04T17:36:28.083822] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-11-04T17:36:28.167051] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-09. [2025-11-04T17:36:28.817839] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-11-04T17:36:28.967047] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-10. [2025-11-04T17:36:29.699876] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-11-04T17:36:29.767109] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-08. [2025-11-04T17:36:30.543400] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-11-04T17:36:30.567017] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-09. [2025-11-04T17:36:31.255628] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-11-04T17:36:31.367041] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-10. [2025-11-04T17:36:32.094293] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-11-04T17:36:32.167034] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-01. [2025-11-04T17:36:32.967011] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-11. [2025-11-04T17:36:33.063160] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T17:36:33.592151] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T17:36:33.593552] [INFO ] [REMOTE ] BGY: Full fetch complete (33045 records). [2025-11-04T17:36:33.593626] [DEBUG] [VALIDATOR ] BGY: Validating primary data. [2025-11-04T17:36:33.770858] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-06. [2025-11-04T17:36:33.974160] [DEBUG] [VALIDATOR ] BGY: All expected trading days are present. [2025-11-04T17:36:33.974238] [DEBUG] [VALIDATOR ] BGY: Primary data validation passed. [2025-11-04T17:36:33.974668] [INFO ] [MAIN ] BGY: Fetched primary data is valid. [2025-11-04T17:36:33.974730] [DEBUG] [LOCAL ] BGY: Saving primary data. [2025-11-04T17:36:33.975473] [DEBUG] [LOCAL ] BGY: Trimmed 33045 records to 21413 (keeping 200 days of data) [2025-11-04T17:36:34.094785] [DEBUG] [LOCAL ] BGY: Primary data saved (21413 records). [2025-11-04T17:36:34.094912] [INFO ] [MAIN ] BGY: Full history saved. [2025-11-04T17:36:34.094986] [INFO ] [MAIN ] Update Progress: 269/2367 symbols completed. BGY took 31.9s. Estimated remaining: 4h 44m 47s. Rate: 7.4 symbols/min [2025-11-04T17:36:34.267266] [INFO ] [MAIN ] BHK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:36:34.291458] [DEBUG] [LOCAL ] BHK: Loaded 24447 existing records from primary storage. [2025-11-04T17:36:34.291792] [INFO ] [MAIN ] BHK: Loaded 24447 existing records. [2025-11-04T17:36:34.464047] [WARN ] [MAIN ] BHK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:36:34.464215] [INFO ] [MAIN ] BHK: Wiping existing primary data before full refresh. [2025-11-04T17:36:34.464261] [INFO ] [LOCAL ] BHK: Starting deletion of primary data. [2025-11-04T17:36:34.464426] [DEBUG] [LOCAL ] BHK: Deleted primary data file. [2025-11-04T17:36:34.464492] [INFO ] [MAIN ] BHK: Starting full primary refresh. [2025-11-04T17:36:34.567765] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-11. [2025-11-04T17:36:34.826470] [INFO ] [REMOTE ] BHK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:36:35.134913] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T17:36:35.136163] [INFO ] [REMOTE ] BH: Full fetch complete (19612 records). [2025-11-04T17:36:35.136261] [DEBUG] [VALIDATOR ] BH: Validating primary data. [2025-11-04T17:36:35.367833] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-02. [2025-11-04T17:36:35.500517] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-11-04T17:36:35.609011] [DEBUG] [VALIDATOR ] BH: All expected trading days are present. [2025-11-04T17:36:35.609134] [DEBUG] [VALIDATOR ] BH: Primary data validation passed. [2025-11-04T17:36:35.609832] [INFO ] [MAIN ] BH: Fetched primary data is valid. [2025-11-04T17:36:35.609904] [DEBUG] [LOCAL ] BH: Saving primary data. [2025-11-04T17:36:35.610878] [DEBUG] [LOCAL ] BH: Trimmed 19612 records to 16718 (keeping 200 days of data) [2025-11-04T17:36:35.715707] [DEBUG] [LOCAL ] BH: Primary data saved (16718 records). [2025-11-04T17:36:35.715902] [INFO ] [MAIN ] BH: Full history saved. [2025-11-04T17:36:35.716007] [INFO ] [MAIN ] Update Progress: 270/2367 symbols completed. BH took 27.3s. Estimated remaining: 4h 43m 48s. Rate: 7.4 symbols/min [2025-11-04T17:36:35.889461] [INFO ] [MAIN ] BHP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:36:35.951934] [DEBUG] [LOCAL ] BHP: Loaded 53360 existing records from primary storage. [2025-11-04T17:36:35.952456] [INFO ] [MAIN ] BHP: Loaded 53360 existing records. [2025-11-04T17:36:36.140456] [WARN ] [MAIN ] BHP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:36:36.140772] [INFO ] [MAIN ] BHP: Wiping existing primary data before full refresh. [2025-11-04T17:36:36.140832] [INFO ] [LOCAL ] BHP: Starting deletion of primary data. [2025-11-04T17:36:36.141077] [DEBUG] [LOCAL ] BHP: Deleted primary data file. [2025-11-04T17:36:36.141121] [INFO ] [MAIN ] BHP: Starting full primary refresh. [2025-11-04T17:36:36.173139] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-01. [2025-11-04T17:36:36.492954] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T17:36:36.576832] [INFO ] [REMOTE ] BHP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:36:36.967116] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-01. [2025-11-04T17:36:37.234519] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T17:36:37.767061] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-07. [2025-11-04T17:36:38.567015] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-03. [2025-11-04T17:36:38.712758] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-11-04T17:36:39.000191] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T17:36:39.367022] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-02. [2025-11-04T17:36:39.455897] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-11-04T17:36:40.167047] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-04. [2025-11-04T17:36:40.753078] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-11-04T17:36:40.967085] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-03. [2025-11-04T17:36:41.131496] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-11-04T17:36:41.767068] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-02. [2025-11-04T17:36:42.362175] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-11-04T17:36:42.407416] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T17:36:42.567086] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-03. [2025-11-04T17:36:43.184693] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-11-04T17:36:43.367040] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-05. [2025-11-04T17:36:44.167024] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-04. [2025-11-04T17:36:44.187266] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T17:36:44.967039] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-06. [2025-11-04T17:36:45.738266] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T17:36:45.772133] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-07. [2025-11-04T17:36:45.790697] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-11-04T17:36:46.567023] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-08. [2025-11-04T17:36:46.600365] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T17:36:47.367042] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-04. [2025-11-04T17:36:48.175820] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-05. [2025-11-04T17:36:48.204606] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-11-04T17:36:48.287557] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T17:36:48.967143] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-09. [2025-11-04T17:36:49.767020] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-05. [2025-11-04T17:36:50.164121] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T17:36:50.225040] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-05 (2.05 seconds). [2025-11-04T17:36:50.560393] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-11-04T17:36:50.566987] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-06. [2025-11-04T17:36:51.337153] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T17:36:51.367004] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-07. [2025-11-04T17:36:52.174717] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-10. [2025-11-04T17:36:52.179261] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-11-04T17:36:52.967049] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-08. [2025-11-04T17:36:53.655219] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-11-04T17:36:53.767086] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-06. [2025-11-04T17:36:53.788074] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-11-04T17:36:54.567055] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-11. [2025-11-04T17:36:55.240585] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T17:36:55.366999] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-07. [2025-11-04T17:36:55.649433] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-11 (1.08 seconds). [2025-11-04T17:36:55.651305] [INFO ] [REMOTE ] BHC: Full fetch complete (67794 records). [2025-11-04T17:36:55.651386] [DEBUG] [VALIDATOR ] BHC: Validating primary data. [2025-11-04T17:36:56.050564] [DEBUG] [VALIDATOR ] BHC: All expected trading days are present. [2025-11-04T17:36:56.050679] [DEBUG] [VALIDATOR ] BHC: Primary data validation passed. [2025-11-04T17:36:56.051415] [INFO ] [MAIN ] BHC: Fetched primary data is valid. [2025-11-04T17:36:56.051458] [DEBUG] [LOCAL ] BHC: Saving primary data. [2025-11-04T17:36:56.052901] [DEBUG] [LOCAL ] BHC: Trimmed 67794 records to 46395 (keeping 200 days of data) [2025-11-04T17:36:56.167128] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-09. [2025-11-04T17:36:56.319760] [DEBUG] [LOCAL ] BHC: Primary data saved (46395 records). [2025-11-04T17:36:56.319868] [INFO ] [MAIN ] BHC: Full history saved. [2025-11-04T17:36:56.320738] [INFO ] [MAIN ] Update Progress: 271/2367 symbols completed. BHC took 43.1s. Estimated remaining: 4h 45m 16s. Rate: 7.3 symbols/min [2025-11-04T17:36:56.490033] [INFO ] [MAIN ] BHR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:36:56.511477] [DEBUG] [LOCAL ] BHR: Loaded 21411 existing records from primary storage. [2025-11-04T17:36:56.511743] [INFO ] [MAIN ] BHR: Loaded 21411 existing records. [2025-11-04T17:36:56.690111] [WARN ] [MAIN ] BHR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:36:56.690361] [INFO ] [MAIN ] BHR: Wiping existing primary data before full refresh. [2025-11-04T17:36:56.690465] [INFO ] [LOCAL ] BHR: Starting deletion of primary data. [2025-11-04T17:36:56.690746] [DEBUG] [LOCAL ] BHR: Deleted primary data file. [2025-11-04T17:36:56.690802] [INFO ] [MAIN ] BHR: Starting full primary refresh. [2025-11-04T17:36:56.973408] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-08. [2025-11-04T17:36:57.081526] [INFO ] [REMOTE ] BHR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:36:57.115135] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-11-04T17:36:57.143930] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T17:36:57.767045] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-08. [2025-11-04T17:36:57.838570] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-11-04T17:36:58.567036] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-01. [2025-11-04T17:36:59.219699] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T17:36:59.367014] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-10. [2025-11-04T17:36:59.459845] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-11-04T17:37:00.167064] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-09. [2025-11-04T17:37:00.464721] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T17:37:00.967041] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-09. [2025-11-04T17:37:01.782773] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-02. [2025-11-04T17:37:01.820432] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-11-04T17:37:02.136475] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T17:37:02.539330] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T17:37:02.567036] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-10. [2025-11-04T17:37:03.334854] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-11-04T17:37:03.367033] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-10. [2025-11-04T17:37:04.167069] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-03. [2025-11-04T17:37:04.977366] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-11. [2025-11-04T17:37:04.984710] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-11-04T17:37:05.017458] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-11-04T17:37:05.749038] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T17:37:05.750281] [INFO ] [REMOTE ] BHK: Full fetch complete (31171 records). [2025-11-04T17:37:05.750365] [DEBUG] [VALIDATOR ] BHK: Validating primary data. [2025-11-04T17:37:05.774760] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-11. [2025-11-04T17:37:06.128083] [DEBUG] [VALIDATOR ] BHK: All expected trading days are present. [2025-11-04T17:37:06.128161] [DEBUG] [VALIDATOR ] BHK: Primary data validation passed. [2025-11-04T17:37:06.128591] [INFO ] [MAIN ] BHK: Fetched primary data is valid. [2025-11-04T17:37:06.128631] [DEBUG] [LOCAL ] BHK: Saving primary data. [2025-11-04T17:37:06.129414] [DEBUG] [LOCAL ] BHK: Trimmed 31171 records to 24518 (keeping 200 days of data) [2025-11-04T17:37:06.268947] [DEBUG] [LOCAL ] BHK: Primary data saved (24518 records). [2025-11-04T17:37:06.269069] [INFO ] [MAIN ] BHK: Full history saved. [2025-11-04T17:37:06.269143] [INFO ] [MAIN ] Update Progress: 272/2367 symbols completed. BHK took 32.2s. Estimated remaining: 4h 45m 22s. Rate: 7.3 symbols/min [2025-11-04T17:37:06.442335] [INFO ] [MAIN ] BHV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:37:06.446040] [DEBUG] [LOCAL ] BHV: No existing data file found in primary storage. [2025-11-04T17:37:06.446189] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T17:37:06.446276] [WARN ] [MAIN ] BHV: No existing data found, performing full refresh instead. [2025-11-04T17:37:06.447820] [INFO ] [MAIN ] BHV: Wiping existing primary data before full refresh. [2025-11-04T17:37:06.447944] [INFO ] [REMOTE ] BHE: Full fetch complete (36433 records). [2025-11-04T17:37:06.447878] [INFO ] [LOCAL ] BHV: Starting deletion of primary data. [2025-11-04T17:37:06.448147] [DEBUG] [VALIDATOR ] BHE: Validating primary data. [2025-11-04T17:37:06.448266] [DEBUG] [LOCAL ] BHV: No primary data file found to delete. [2025-11-04T17:37:06.448500] [INFO ] [MAIN ] BHV: Starting full primary refresh. [2025-11-04T17:37:06.572168] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-04. [2025-11-04T17:37:07.198219] [DEBUG] [VALIDATOR ] BHE: All expected trading days are present. [2025-11-04T17:37:07.214118] [DEBUG] [VALIDATOR ] BHE: Primary data validation passed. [2025-11-04T17:37:07.219001] [INFO ] [MAIN ] BHE: Fetched primary data is valid. [2025-11-04T17:37:07.219110] [DEBUG] [LOCAL ] BHE: Saving primary data. [2025-11-04T17:37:07.218428] [INFO ] [REMOTE ] BHV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:37:07.220189] [DEBUG] [LOCAL ] BHE: Trimmed 36433 records to 23080 (keeping 200 days of data) [2025-11-04T17:37:07.356094] [DEBUG] [LOCAL ] BHE: Primary data saved (23080 records). [2025-11-04T17:37:07.356218] [INFO ] [MAIN ] BHE: Full history saved. [2025-11-04T17:37:07.356296] [INFO ] [MAIN ] Update Progress: 273/2367 symbols completed. BHE took 43.6s. Estimated remaining: 4h 44m 19s. Rate: 7.4 symbols/min [2025-11-04T17:37:07.376880] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-01. [2025-11-04T17:37:07.529951] [INFO ] [MAIN ] BHVN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:37:07.576055] [DEBUG] [LOCAL ] BHVN: Loaded 50717 existing records from primary storage. [2025-11-04T17:37:07.576455] [INFO ] [MAIN ] BHVN: Loaded 50717 existing records. [2025-11-04T17:37:07.692169] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T17:37:07.780415] [WARN ] [MAIN ] BHVN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:37:07.780682] [INFO ] [MAIN ] BHVN: Wiping existing primary data before full refresh. [2025-11-04T17:37:07.780740] [INFO ] [LOCAL ] BHVN: Starting deletion of primary data. [2025-11-04T17:37:07.780966] [DEBUG] [LOCAL ] BHVN: Deleted primary data file. [2025-11-04T17:37:07.781022] [INFO ] [MAIN ] BHVN: Starting full primary refresh. [2025-11-04T17:37:08.066218] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T17:37:08.146140] [INFO ] [REMOTE ] BHVN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:37:08.167020] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-01. [2025-11-04T17:37:08.967036] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-02. [2025-11-04T17:37:09.215802] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-02 (0.25 seconds). [2025-11-04T17:37:09.513475] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T17:37:09.767092] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-11. [2025-11-04T17:37:10.567083] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-02. [2025-11-04T17:37:11.233064] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-11 (1.47 seconds). [2025-11-04T17:37:11.235181] [INFO ] [REMOTE ] BHP: Full fetch complete (77453 records). [2025-11-04T17:37:11.235298] [DEBUG] [VALIDATOR ] BHP: Validating primary data. [2025-11-04T17:37:11.373124] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-03. [2025-11-04T17:37:11.643499] [DEBUG] [VALIDATOR ] BHP: All expected trading days are present. [2025-11-04T17:37:11.643673] [DEBUG] [VALIDATOR ] BHP: Primary data validation passed. [2025-11-04T17:37:11.644904] [INFO ] [MAIN ] BHP: Fetched primary data is valid. [2025-11-04T17:37:11.645009] [DEBUG] [LOCAL ] BHP: Saving primary data. [2025-11-04T17:37:11.647109] [DEBUG] [LOCAL ] BHP: Trimmed 77453 records to 53360 (keeping 200 days of data) [2025-11-04T17:37:11.738304] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-11-04T17:37:11.742232] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T17:37:12.009535] [DEBUG] [LOCAL ] BHP: Primary data saved (53360 records). [2025-11-04T17:37:12.009711] [INFO ] [MAIN ] BHP: Full history saved. [2025-11-04T17:37:12.010816] [INFO ] [MAIN ] Update Progress: 274/2367 symbols completed. BHP took 36.3s. Estimated remaining: 4h 43m 44s. Rate: 7.4 symbols/min [2025-11-04T17:37:12.179529] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-05. [2025-11-04T17:37:12.182109] [INFO ] [MAIN ] BILL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:37:12.234208] [DEBUG] [LOCAL ] BILL: Loaded 52547 existing records from primary storage. [2025-11-04T17:37:12.234531] [INFO ] [MAIN ] BILL: Loaded 52547 existing records. [2025-11-04T17:37:12.419069] [WARN ] [MAIN ] BILL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:37:12.419288] [INFO ] [MAIN ] BILL: Wiping existing primary data before full refresh. [2025-11-04T17:37:12.419371] [INFO ] [LOCAL ] BILL: Starting deletion of primary data. [2025-11-04T17:37:12.419674] [DEBUG] [LOCAL ] BILL: Deleted primary data file. [2025-11-04T17:37:12.419744] [INFO ] [MAIN ] BILL: Starting full primary refresh. [2025-11-04T17:37:12.802238] [INFO ] [REMOTE ] BILL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:37:12.967020] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-01. [2025-11-04T17:37:13.074867] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T17:37:13.767070] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-04. [2025-11-04T17:37:14.143063] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T17:37:14.567083] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-05. [2025-11-04T17:37:14.837329] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-05 (0.27 seconds). [2025-11-04T17:37:14.840514] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-11-04T17:37:15.367056] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-06. [2025-11-04T17:37:15.641380] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-11-04T17:37:16.167115] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-03. [2025-11-04T17:37:16.967132] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-02. [2025-11-04T17:37:17.428179] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-11-04T17:37:17.767083] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-06. [2025-11-04T17:37:18.462169] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-11-04T17:37:18.567029] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-04. [2025-11-04T17:37:18.595506] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T17:37:19.367060] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-07. [2025-11-04T17:37:20.072287] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-11-04T17:37:20.118112] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-11-04T17:37:20.167077] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-03. [2025-11-04T17:37:20.967106] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-08. [2025-11-04T17:37:21.541155] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T17:37:21.675629] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T17:37:21.767059] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-07. [2025-11-04T17:37:22.185724] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-11-04T17:37:22.567057] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-05. [2025-11-04T17:37:23.367096] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-09. [2025-11-04T17:37:23.920354] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-11-04T17:37:24.167035] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-06. [2025-11-04T17:37:24.264130] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-11-04T17:37:24.967106] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-04. [2025-11-04T17:37:25.297195] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T17:37:25.767037] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-08. [2025-11-04T17:37:26.078546] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-08 (0.31 seconds). [2025-11-04T17:37:26.238789] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T17:37:26.567052] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-10. [2025-11-04T17:37:27.367066] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-07. [2025-11-04T17:37:27.456355] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-11-04T17:37:28.167073] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-09. [2025-11-04T17:37:28.483440] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-11-04T17:37:28.708823] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T17:37:28.967093] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-08. [2025-11-04T17:37:29.767055] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-10. [2025-11-04T17:37:30.036820] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-10 (0.27 seconds). [2025-11-04T17:37:30.476412] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T17:37:30.567082] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-09. [2025-11-04T17:37:31.367110] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-11. [2025-11-04T17:37:31.884003] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-11-04T17:37:32.145305] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T17:37:32.146541] [INFO ] [REMOTE ] BHR: Full fetch complete (30911 records). [2025-11-04T17:37:32.146611] [DEBUG] [VALIDATOR ] BHR: Validating primary data. [2025-11-04T17:37:32.176380] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-10. [2025-11-04T17:37:32.518509] [DEBUG] [VALIDATOR ] BHR: All expected trading days are present. [2025-11-04T17:37:32.518598] [DEBUG] [VALIDATOR ] BHR: Primary data validation passed. [2025-11-04T17:37:32.519084] [INFO ] [MAIN ] BHR: Fetched primary data is valid. [2025-11-04T17:37:32.519143] [DEBUG] [LOCAL ] BHR: Saving primary data. [2025-11-04T17:37:32.520198] [DEBUG] [LOCAL ] BHR: Trimmed 30911 records to 21470 (keeping 200 days of data) [2025-11-04T17:37:32.644216] [DEBUG] [LOCAL ] BHR: Primary data saved (21470 records). [2025-11-04T17:37:32.644388] [INFO ] [MAIN ] BHR: Full history saved. [2025-11-04T17:37:32.644499] [INFO ] [MAIN ] Update Progress: 275/2367 symbols completed. BHR took 36.3s. Estimated remaining: 4h 45m 11s. Rate: 7.3 symbols/min [2025-11-04T17:37:32.811041] [INFO ] [MAIN ] BIO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:37:32.811209] [DEBUG] [LOCAL ] BIO: No existing data file found in primary storage. [2025-11-04T17:37:32.811246] [WARN ] [MAIN ] BIO: No existing data found, performing full refresh instead. [2025-11-04T17:37:32.811382] [INFO ] [MAIN ] BIO: Wiping existing primary data before full refresh. [2025-11-04T17:37:32.811451] [INFO ] [LOCAL ] BIO: Starting deletion of primary data. [2025-11-04T17:37:32.811505] [DEBUG] [LOCAL ] BIO: No primary data file found to delete. [2025-11-04T17:37:32.811538] [INFO ] [MAIN ] BIO: Starting full primary refresh. [2025-11-04T17:37:32.969243] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-11. [2025-11-04T17:37:33.172211] [INFO ] [REMOTE ] BIO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:37:33.690747] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T17:37:33.691353] [INFO ] [REMOTE ] BHV: Full fetch complete (1404 records). [2025-11-04T17:37:33.691419] [DEBUG] [VALIDATOR ] BHV: Validating primary data. [2025-11-04T17:37:33.770794] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-05. [2025-11-04T17:37:33.882992] [WARN ] [VALIDATOR ] BHV: Validation failed - insufficient historical data. Have 194 unique days, need 200 days [2025-11-04T17:37:33.883337] [FATAL] [MAIN ] BHV: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:37:33.883420] [WARN ] [MAIN ] BHV: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:37:33.883514] [INFO ] [LOCAL ] BHV: Starting deletion of primary data. [2025-11-04T17:37:33.883594] [DEBUG] [LOCAL ] BHV: No primary data file found to delete. [2025-11-04T17:37:33.883632] [INFO ] [LOCAL ] BHV: Starting deletion of derivative data. [2025-11-04T17:37:33.883740] [DEBUG] [LOCAL ] BHV: No derivative data file found to delete. [2025-11-04T17:37:33.883819] [INFO ] [MAIN ] Update Progress: 276/2367 symbols completed. BHV took 27.6s. Estimated remaining: 4h 44m 11s. Rate: 7.4 symbols/min [2025-11-04T17:37:34.098940] [INFO ] [MAIN ] BIP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:37:34.096343] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-11-04T17:37:34.144396] [DEBUG] [LOCAL ] BIP: Loaded 38452 existing records from primary storage. [2025-11-04T17:37:34.144720] [INFO ] [MAIN ] BIP: Loaded 38452 existing records. [2025-11-04T17:37:34.318112] [WARN ] [MAIN ] BIP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:37:34.318333] [INFO ] [MAIN ] BIP: Wiping existing primary data before full refresh. [2025-11-04T17:37:34.318399] [INFO ] [LOCAL ] BIP: Starting deletion of primary data. [2025-11-04T17:37:34.318592] [DEBUG] [LOCAL ] BIP: Deleted primary data file. [2025-11-04T17:37:34.318679] [INFO ] [MAIN ] BIP: Starting full primary refresh. [2025-11-04T17:37:34.572123] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-11. [2025-11-04T17:37:34.673077] [INFO ] [REMOTE ] BIP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:37:35.225390] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T17:37:35.367051] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-06. [2025-11-04T17:37:35.931297] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-11 (1.36 seconds). [2025-11-04T17:37:35.933277] [INFO ] [REMOTE ] BHVN: Full fetch complete (71041 records). [2025-11-04T17:37:35.933371] [DEBUG] [VALIDATOR ] BHVN: Validating primary data. [2025-11-04T17:37:36.175742] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-01. [2025-11-04T17:37:36.338125] [DEBUG] [VALIDATOR ] BHVN: All expected trading days are present. [2025-11-04T17:37:36.338199] [DEBUG] [VALIDATOR ] BHVN: Primary data validation passed. [2025-11-04T17:37:36.338916] [INFO ] [MAIN ] BHVN: Fetched primary data is valid. [2025-11-04T17:37:36.338980] [DEBUG] [LOCAL ] BHVN: Saving primary data. [2025-11-04T17:37:36.340238] [DEBUG] [LOCAL ] BHVN: Trimmed 71041 records to 50722 (keeping 200 days of data) [2025-11-04T17:37:36.625362] [DEBUG] [LOCAL ] BHVN: Primary data saved (50722 records). [2025-11-04T17:37:36.625489] [INFO ] [MAIN ] BHVN: Full history saved. [2025-11-04T17:37:36.625579] [INFO ] [MAIN ] Update Progress: 277/2367 symbols completed. BHVN took 29.3s. Estimated remaining: 4h 43m 22s. Rate: 7.4 symbols/min [2025-11-04T17:37:36.942211] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-11-04T17:37:36.967023] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-01. [2025-11-04T17:37:37.449854] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T17:37:37.663606] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-11-04T17:37:37.767180] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-02. [2025-11-04T17:37:38.567103] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-07. [2025-11-04T17:37:38.770028] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T17:37:39.367126] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-02. [2025-11-04T17:37:39.904096] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T17:37:40.167052] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-08. [2025-11-04T17:37:40.236136] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T17:37:40.967052] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-03. [2025-11-04T17:37:41.683613] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-11-04T17:37:41.767026] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-09. [2025-11-04T17:37:42.102455] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T17:37:42.567057] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-03. [2025-11-04T17:37:43.017455] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T17:37:43.367066] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-10. [2025-11-04T17:37:43.593980] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T17:37:44.167074] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-04. [2025-11-04T17:37:44.885565] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-11-04T17:37:44.967067] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-04. [2025-11-04T17:37:45.106086] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-11-04T17:37:45.767109] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-11. [2025-11-04T17:37:46.008803] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-11-04T17:37:46.567040] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-05. [2025-11-04T17:37:47.028806] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-11 (1.26 seconds). [2025-11-04T17:37:47.030778] [INFO ] [REMOTE ] BILL: Full fetch complete (76161 records). [2025-11-04T17:37:47.030880] [DEBUG] [VALIDATOR ] BILL: Validating primary data. [2025-11-04T17:37:47.369000] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-05. [2025-11-04T17:37:47.413893] [DEBUG] [VALIDATOR ] BILL: All expected trading days are present. [2025-11-04T17:37:47.413992] [DEBUG] [VALIDATOR ] BILL: Primary data validation passed. [2025-11-04T17:37:47.414830] [INFO ] [MAIN ] BILL: Fetched primary data is valid. [2025-11-04T17:37:47.414913] [DEBUG] [LOCAL ] BILL: Saving primary data. [2025-11-04T17:37:47.416688] [DEBUG] [LOCAL ] BILL: Trimmed 76161 records to 52576 (keeping 200 days of data) [2025-11-04T17:37:47.651237] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-11-04T17:37:47.746783] [DEBUG] [LOCAL ] BILL: Primary data saved (52576 records). [2025-11-04T17:37:47.746947] [INFO ] [MAIN ] BILL: Full history saved. [2025-11-04T17:37:47.747591] [INFO ] [MAIN ] Update Progress: 278/2367 symbols completed. BILL took 35.7s. Estimated remaining: 4h 43m 36s. Rate: 7.4 symbols/min [2025-11-04T17:37:48.140183] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-11-04T17:37:48.166995] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-06. [2025-11-04T17:37:48.967029] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-06. [2025-11-04T17:37:49.066215] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-11-04T17:37:49.767092] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-07. [2025-11-04T17:37:50.076501] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T17:37:50.567003] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-07. [2025-11-04T17:37:50.615545] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T17:37:51.367081] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-08. [2025-11-04T17:37:51.606503] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T17:37:52.166610] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-11-04T17:37:52.171248] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-08. [2025-11-04T17:37:52.967080] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-09. [2025-11-04T17:37:53.322686] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-11-04T17:37:53.767020] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-09. [2025-11-04T17:37:53.829520] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-11-04T17:37:54.567075] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-10. [2025-11-04T17:37:54.811985] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-11-04T17:37:55.198515] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-11-04T17:37:55.367022] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-10. [2025-11-04T17:37:56.167079] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-11. [2025-11-04T17:37:56.491511] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-11-04T17:37:56.895406] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T17:37:56.896536] [INFO ] [REMOTE ] BIO: Full fetch complete (34916 records). [2025-11-04T17:37:56.896622] [DEBUG] [VALIDATOR ] BIO: Validating primary data. [2025-11-04T17:37:56.897165] [FATAL] [VALIDATOR ] BIO: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:37:56.897263] [FATAL] [MAIN ] BIO: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:37:56.897339] [WARN ] [MAIN ] BIO: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:37:56.897425] [INFO ] [LOCAL ] BIO: Starting deletion of primary data. [2025-11-04T17:37:56.897505] [DEBUG] [LOCAL ] BIO: No primary data file found to delete. [2025-11-04T17:37:56.897542] [INFO ] [LOCAL ] BIO: Starting deletion of derivative data. [2025-11-04T17:37:56.897580] [DEBUG] [LOCAL ] BIO: No derivative data file found to delete. [2025-11-04T17:37:56.897631] [INFO ] [MAIN ] Update Progress: 279/2367 symbols completed. BIO took 24.3s. Estimated remaining: 4h 43m 35s. Rate: 7.4 symbols/min [2025-11-04T17:37:56.967053] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-11. [2025-11-04T17:37:58.100081] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T17:37:58.101744] [INFO ] [REMOTE ] BIP: Full fetch complete (57904 records). [2025-11-04T17:37:58.101854] [DEBUG] [VALIDATOR ] BIP: Validating primary data. [2025-11-04T17:37:58.485458] [DEBUG] [VALIDATOR ] BIP: All expected trading days are present. [2025-11-04T17:37:58.485549] [DEBUG] [VALIDATOR ] BIP: Primary data validation passed. [2025-11-04T17:37:58.486191] [INFO ] [MAIN ] BIP: Fetched primary data is valid. [2025-11-04T17:37:58.486255] [DEBUG] [LOCAL ] BIP: Saving primary data. [2025-11-04T17:37:58.487493] [DEBUG] [LOCAL ] BIP: Trimmed 57904 records to 38399 (keeping 200 days of data) [2025-11-04T17:37:58.711598] [DEBUG] [LOCAL ] BIP: Primary data saved (38399 records). [2025-11-04T17:37:58.711773] [INFO ] [MAIN ] BIP: Full history saved. [2025-11-04T17:37:58.711857] [INFO ] [MAIN ] Update Progress: 280/2367 symbols completed. BIP took 24.8s. Estimated remaining: 4h 42m 40s. Rate: 7.4 symbols/min [2025-11-04T17:37:59.359039] [INFO ] [MAIN ] BIPH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:37:59.460295] [INFO ] [MAIN ] BIPC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:37:59.464047] [DEBUG] [LOCAL ] BIPH: Loaded 2058 existing records from primary storage. [2025-11-04T17:37:59.438943] [INFO ] [MAIN ] BIPJ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:37:59.464587] [INFO ] [MAIN ] BIPH: Loaded 2058 existing records. [2025-11-04T17:37:59.461828] [INFO ] [MAIN ] BIPI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:37:59.574107] [DEBUG] [LOCAL ] BIPJ: Loaded 2896 existing records from primary storage. [2025-11-04T17:37:59.575062] [INFO ] [MAIN ] BIPJ: Loaded 2896 existing records. [2025-11-04T17:37:59.603809] [DEBUG] [LOCAL ] BIPI: Loaded 2663 existing records from primary storage. [2025-11-04T17:37:59.609229] [INFO ] [MAIN ] BIPI: Loaded 2663 existing records. [2025-11-04T17:37:59.772530] [WARN ] [MAIN ] BIPH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:37:59.809424] [INFO ] [MAIN ] BIPH: Wiping existing primary data before full refresh. [2025-11-04T17:37:59.893292] [INFO ] [LOCAL ] BIPH: Starting deletion of primary data. [2025-11-04T17:37:59.919272] [DEBUG] [LOCAL ] BIPH: Deleted primary data file. [2025-11-04T17:37:59.931025] [DEBUG] [LOCAL ] BIPC: Loaded 40678 existing records from primary storage. [2025-11-04T17:37:59.963767] [INFO ] [MAIN ] BIPC: Loaded 40678 existing records. [2025-11-04T17:37:59.919417] [INFO ] [MAIN ] BIPH: Starting full primary refresh. [2025-11-04T17:38:00.036810] [WARN ] [MAIN ] BIPJ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:38:00.139383] [INFO ] [MAIN ] BIPJ: Wiping existing primary data before full refresh. [2025-11-04T17:38:00.139230] [WARN ] [MAIN ] BIPI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:38:00.144634] [INFO ] [LOCAL ] BIPJ: Starting deletion of primary data. [2025-11-04T17:38:00.264725] [DEBUG] [LOCAL ] BIPJ: Deleted primary data file. [2025-11-04T17:38:00.253048] [INFO ] [MAIN ] BIPI: Wiping existing primary data before full refresh. [2025-11-04T17:38:00.272176] [INFO ] [MAIN ] BIPJ: Starting full primary refresh. [2025-11-04T17:38:00.272291] [INFO ] [LOCAL ] BIPI: Starting deletion of primary data. [2025-11-04T17:38:00.319982] [DEBUG] [LOCAL ] BIPI: Deleted primary data file. [2025-11-04T17:38:00.320125] [INFO ] [MAIN ] BIPI: Starting full primary refresh. [2025-11-04T17:38:00.521101] [WARN ] [MAIN ] BIPC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:38:00.626745] [INFO ] [MAIN ] BIPC: Wiping existing primary data before full refresh. [2025-11-04T17:38:00.681273] [INFO ] [LOCAL ] BIPC: Starting deletion of primary data. [2025-11-04T17:38:00.737599] [DEBUG] [LOCAL ] BIPC: Deleted primary data file. [2025-11-04T17:38:00.778408] [INFO ] [MAIN ] BIPC: Starting full primary refresh. [2025-11-04T17:38:01.320271] [INFO ] [REMOTE ] BIPH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:38:01.386876] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-01. [2025-11-04T17:38:01.670907] [INFO ] [REMOTE ] BIPJ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:38:01.701905] [INFO ] [REMOTE ] BIPI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:38:01.702225] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-01. [2025-11-04T17:38:01.703417] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-01. [2025-11-04T17:38:01.813808] [INFO ] [REMOTE ] BIPC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:38:01.814121] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-01. [2025-11-04T17:38:02.237162] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T17:38:02.238493] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-02. [2025-11-04T17:38:02.279817] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-11-04T17:38:02.281041] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-02. [2025-11-04T17:38:02.479523] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-11-04T17:38:02.488324] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-02 (0.25 seconds). [2025-11-04T17:38:02.566989] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-02. [2025-11-04T17:38:02.591693] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-11-04T17:38:02.913519] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-11-04T17:38:03.134047] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T17:38:03.367041] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-02. [2025-11-04T17:38:04.167102] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-03. [2025-11-04T17:38:04.418104] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T17:38:04.437065] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-03 (0.27 seconds). [2025-11-04T17:38:04.967046] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-04. [2025-11-04T17:38:05.326143] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T17:38:05.767041] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-05. [2025-11-04T17:38:06.063948] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-11-04T17:38:06.567043] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-03. [2025-11-04T17:38:06.879201] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-11-04T17:38:07.367036] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-04. [2025-11-04T17:38:07.727793] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T17:38:08.167041] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-06. [2025-11-04T17:38:08.465137] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-06 (0.30 seconds). [2025-11-04T17:38:08.967062] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-03. [2025-11-04T17:38:09.396528] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-11-04T17:38:09.767063] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-07. [2025-11-04T17:38:10.082034] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-07 (0.31 seconds). [2025-11-04T17:38:10.567084] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-08. [2025-11-04T17:38:10.894399] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-11-04T17:38:11.367025] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-03. [2025-11-04T17:38:12.167086] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-09. [2025-11-04T17:38:12.562463] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-11-04T17:38:12.566626] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T17:38:12.967069] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-05. [2025-11-04T17:38:13.364536] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-11-04T17:38:13.767086] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-04. [2025-11-04T17:38:14.290205] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-11-04T17:38:14.567077] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-05. [2025-11-04T17:38:14.848848] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-11-04T17:38:15.367063] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-06. [2025-11-04T17:38:15.660384] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-06 (0.29 seconds). [2025-11-04T17:38:16.167102] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-06. [2025-11-04T17:38:16.534167] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-11-04T17:38:16.967067] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-07. [2025-11-04T17:38:17.354568] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-11-04T17:38:17.767037] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-10. [2025-11-04T17:38:18.065228] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-10 (0.30 seconds). [2025-11-04T17:38:18.567075] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-07. [2025-11-04T17:38:18.920811] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-11-04T17:38:19.367057] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-11. [2025-11-04T17:38:19.723295] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T17:38:19.723990] [INFO ] [REMOTE ] BIPH: Full fetch complete (3517 records). [2025-11-04T17:38:19.724057] [DEBUG] [VALIDATOR ] BIPH: Validating primary data. [2025-11-04T17:38:20.072189] [DEBUG] [VALIDATOR ] BIPH: All expected trading days are present. [2025-11-04T17:38:20.072262] [DEBUG] [VALIDATOR ] BIPH: Primary data validation passed. [2025-11-04T17:38:20.072395] [INFO ] [MAIN ] BIPH: Fetched primary data is valid. [2025-11-04T17:38:20.072433] [DEBUG] [LOCAL ] BIPH: Saving primary data. [2025-11-04T17:38:20.072796] [DEBUG] [LOCAL ] BIPH: Trimmed 3517 records to 2067 (keeping 200 days of data) [2025-11-04T17:38:20.085228] [DEBUG] [LOCAL ] BIPH: Primary data saved (2067 records). [2025-11-04T17:38:20.085291] [INFO ] [MAIN ] BIPH: Full history saved. [2025-11-04T17:38:20.085355] [INFO ] [MAIN ] Update Progress: 281/2367 symbols completed. BIPH took 21.4s. Estimated remaining: 4h 44m 10s. Rate: 7.3 symbols/min [2025-11-04T17:38:20.172197] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-04. [2025-11-04T17:38:20.256772] [INFO ] [MAIN ] BIRK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:38:20.303722] [DEBUG] [LOCAL ] BIRK: Loaded 51105 existing records from primary storage. [2025-11-04T17:38:20.304023] [INFO ] [MAIN ] BIRK: Loaded 51105 existing records. [2025-11-04T17:38:20.477227] [WARN ] [MAIN ] BIRK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:38:20.477424] [INFO ] [MAIN ] BIRK: Wiping existing primary data before full refresh. [2025-11-04T17:38:20.477469] [INFO ] [LOCAL ] BIRK: Starting deletion of primary data. [2025-11-04T17:38:20.477679] [DEBUG] [LOCAL ] BIRK: Deleted primary data file. [2025-11-04T17:38:20.477735] [INFO ] [MAIN ] BIRK: Starting full primary refresh. [2025-11-04T17:38:20.832828] [INFO ] [REMOTE ] BIRK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:38:20.967058] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-01. [2025-11-04T17:38:21.289028] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T17:38:21.767114] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-08. [2025-11-04T17:38:22.172206] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-11-04T17:38:22.172049] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T17:38:22.567080] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-09. [2025-11-04T17:38:22.927035] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-09 (0.36 seconds). [2025-11-04T17:38:23.367043] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-10. [2025-11-04T17:38:23.681286] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-10 (0.31 seconds). [2025-11-04T17:38:24.167101] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-11. [2025-11-04T17:38:24.673913] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-11 (0.51 seconds). [2025-11-04T17:38:24.674917] [INFO ] [REMOTE ] BIPI: Full fetch complete (4486 records). [2025-11-04T17:38:24.674994] [DEBUG] [VALIDATOR ] BIPI: Validating primary data. [2025-11-04T17:38:24.972346] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-02. [2025-11-04T17:38:25.100073] [DEBUG] [VALIDATOR ] BIPI: All expected trading days are present. [2025-11-04T17:38:25.100159] [DEBUG] [VALIDATOR ] BIPI: Primary data validation passed. [2025-11-04T17:38:25.100325] [INFO ] [MAIN ] BIPI: Fetched primary data is valid. [2025-11-04T17:38:25.100367] [DEBUG] [LOCAL ] BIPI: Saving primary data. [2025-11-04T17:38:25.100863] [DEBUG] [LOCAL ] BIPI: Trimmed 4486 records to 2663 (keeping 200 days of data) [2025-11-04T17:38:25.119873] [DEBUG] [LOCAL ] BIPI: Primary data saved (2663 records). [2025-11-04T17:38:25.120021] [INFO ] [MAIN ] BIPI: Full history saved. [2025-11-04T17:38:25.120106] [INFO ] [MAIN ] Update Progress: 282/2367 symbols completed. BIPI took 26.4s. Estimated remaining: 4h 43m 39s. Rate: 7.4 symbols/min [2025-11-04T17:38:25.362264] [INFO ] [MAIN ] BIT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:38:25.393829] [DEBUG] [LOCAL ] BIT: Loaded 23767 existing records from primary storage. [2025-11-04T17:38:25.394150] [INFO ] [MAIN ] BIT: Loaded 23767 existing records. [2025-11-04T17:38:25.652743] [WARN ] [MAIN ] BIT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:38:25.652965] [INFO ] [MAIN ] BIT: Wiping existing primary data before full refresh. [2025-11-04T17:38:25.653014] [INFO ] [LOCAL ] BIT: Starting deletion of primary data. [2025-11-04T17:38:25.653184] [DEBUG] [LOCAL ] BIT: Deleted primary data file. [2025-11-04T17:38:25.653223] [INFO ] [MAIN ] BIT: Starting full primary refresh. [2025-11-04T17:38:25.775547] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-08. [2025-11-04T17:38:26.103680] [INFO ] [REMOTE ] BIT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:38:26.202064] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-11-04T17:38:26.206581] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T17:38:26.567070] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-09. [2025-11-04T17:38:26.935224] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-11-04T17:38:27.367112] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-05. [2025-11-04T17:38:28.167108] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-10. [2025-11-04T17:38:28.477518] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-10 (0.31 seconds). [2025-11-04T17:38:28.593479] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T17:38:28.967093] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-06. [2025-11-04T17:38:29.767128] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-01. [2025-11-04T17:38:30.053367] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T17:38:30.567079] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-07. [2025-11-04T17:38:30.689900] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-11-04T17:38:31.367130] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-11. [2025-11-04T17:38:31.706529] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T17:38:31.719089] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-11 (0.35 seconds). [2025-11-04T17:38:31.720510] [INFO ] [REMOTE ] BIPJ: Full fetch complete (4072 records). [2025-11-04T17:38:31.720582] [DEBUG] [VALIDATOR ] BIPJ: Validating primary data. [2025-11-04T17:38:32.086666] [DEBUG] [VALIDATOR ] BIPJ: All expected trading days are present. [2025-11-04T17:38:32.086765] [DEBUG] [VALIDATOR ] BIPJ: Primary data validation passed. [2025-11-04T17:38:32.086930] [INFO ] [MAIN ] BIPJ: Fetched primary data is valid. [2025-11-04T17:38:32.086993] [DEBUG] [LOCAL ] BIPJ: Saving primary data. [2025-11-04T17:38:32.087537] [DEBUG] [LOCAL ] BIPJ: Trimmed 4072 records to 2886 (keeping 200 days of data) [2025-11-04T17:38:32.106781] [DEBUG] [LOCAL ] BIPJ: Primary data saved (2886 records). [2025-11-04T17:38:32.106955] [INFO ] [MAIN ] BIPJ: Full history saved. [2025-11-04T17:38:32.107046] [INFO ] [MAIN ] Update Progress: 283/2367 symbols completed. BIPJ took 33.4s. Estimated remaining: 4h 43m 22s. Rate: 7.4 symbols/min [2025-11-04T17:38:32.175523] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-03. [2025-11-04T17:38:32.287810] [INFO ] [MAIN ] BJ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:38:32.341026] [DEBUG] [LOCAL ] BJ: Loaded 51371 existing records from primary storage. [2025-11-04T17:38:32.341319] [INFO ] [MAIN ] BJ: Loaded 51371 existing records. [2025-11-04T17:38:32.516508] [WARN ] [MAIN ] BJ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:38:32.516824] [INFO ] [MAIN ] BJ: Wiping existing primary data before full refresh. [2025-11-04T17:38:32.516902] [INFO ] [LOCAL ] BJ: Starting deletion of primary data. [2025-11-04T17:38:32.517113] [DEBUG] [LOCAL ] BJ: Deleted primary data file. [2025-11-04T17:38:32.517169] [INFO ] [MAIN ] BJ: Starting full primary refresh. [2025-11-04T17:38:32.861377] [INFO ] [REMOTE ] BJ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:38:32.967072] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-01. [2025-11-04T17:38:33.610541] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T17:38:33.767096] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-02. [2025-11-04T17:38:34.229063] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T17:38:34.426049] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-11-04T17:38:34.567052] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-08. [2025-11-04T17:38:35.367055] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-04. [2025-11-04T17:38:35.597007] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-11-04T17:38:36.167046] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-02. [2025-11-04T17:38:36.554291] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T17:38:36.967104] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-09. [2025-11-04T17:38:37.466208] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T17:38:37.767056] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-03. [2025-11-04T17:38:38.169524] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T17:38:38.567053] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-10. [2025-11-04T17:38:39.021267] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-11-04T17:38:39.367039] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-04. [2025-11-04T17:38:39.639453] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T17:38:40.167042] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-05. [2025-11-04T17:38:40.788451] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T17:38:40.967089] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-05. [2025-11-04T17:38:41.524580] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T17:38:41.767098] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-06. [2025-11-04T17:38:42.305769] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T17:38:42.567067] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-06. [2025-11-04T17:38:42.888337] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T17:38:43.367035] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-07. [2025-11-04T17:38:43.664142] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T17:38:44.167039] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-11. [2025-11-04T17:38:44.847218] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T17:38:44.967031] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-08. [2025-11-04T17:38:45.083835] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T17:38:45.085563] [INFO ] [REMOTE ] BIPC: Full fetch complete (60363 records). [2025-11-04T17:38:45.085677] [DEBUG] [VALIDATOR ] BIPC: Validating primary data. [2025-11-04T17:38:45.509756] [DEBUG] [VALIDATOR ] BIPC: All expected trading days are present. [2025-11-04T17:38:45.509868] [DEBUG] [VALIDATOR ] BIPC: Primary data validation passed. [2025-11-04T17:38:45.510624] [INFO ] [MAIN ] BIPC: Fetched primary data is valid. [2025-11-04T17:38:45.510702] [DEBUG] [LOCAL ] BIPC: Saving primary data. [2025-11-04T17:38:45.512000] [DEBUG] [LOCAL ] BIPC: Trimmed 60363 records to 40668 (keeping 200 days of data) [2025-11-04T17:38:45.774734] [DEBUG] [LOCAL ] BIPC: Primary data saved (40668 records). [2025-11-04T17:38:45.774838] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-07. [2025-11-04T17:38:45.775204] [INFO ] [MAIN ] BIPC: Full history saved. [2025-11-04T17:38:45.775592] [INFO ] [MAIN ] Update Progress: 284/2367 symbols completed. BIPC took 47.1s. Estimated remaining: 4h 43m 54s. Rate: 7.3 symbols/min [2025-11-04T17:38:45.985242] [INFO ] [MAIN ] BK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:38:46.038805] [DEBUG] [LOCAL ] BK: Loaded 53524 existing records from primary storage. [2025-11-04T17:38:46.039147] [INFO ] [MAIN ] BK: Loaded 53524 existing records. [2025-11-04T17:38:46.281846] [WARN ] [MAIN ] BK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:38:46.282045] [INFO ] [MAIN ] BK: Wiping existing primary data before full refresh. [2025-11-04T17:38:46.282094] [INFO ] [LOCAL ] BK: Starting deletion of primary data. [2025-11-04T17:38:46.282380] [DEBUG] [LOCAL ] BK: Deleted primary data file. [2025-11-04T17:38:46.282438] [INFO ] [MAIN ] BK: Starting full primary refresh. [2025-11-04T17:38:46.568466] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-03. [2025-11-04T17:38:46.731838] [INFO ] [REMOTE ] BK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:38:46.774663] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-11-04T17:38:47.102448] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T17:38:47.366999] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-08. [2025-11-04T17:38:47.538698] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-11-04T17:38:48.167038] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-01. [2025-11-04T17:38:48.967012] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-04. [2025-11-04T17:38:49.008534] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-11-04T17:38:49.577913] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-11-04T17:38:49.745334] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-11-04T17:38:49.767025] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-02. [2025-11-04T17:38:50.567048] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-05. [2025-11-04T17:38:50.941956] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T17:38:51.183850] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-11-04T17:38:51.367041] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-09. [2025-11-04T17:38:52.167056] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-06. [2025-11-04T17:38:52.558950] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T17:38:52.838541] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-11-04T17:38:52.967042] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-03. [2025-11-04T17:38:53.767039] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-10. [2025-11-04T17:38:54.389375] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T17:38:54.567063] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-09. [2025-11-04T17:38:55.178969] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T17:38:55.367057] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-11. [2025-11-04T17:38:55.868079] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T17:38:56.167037] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-10. [2025-11-04T17:38:56.523609] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T17:38:56.525892] [INFO ] [REMOTE ] BIRK: Full fetch complete (73651 records). [2025-11-04T17:38:56.526029] [DEBUG] [VALIDATOR ] BIRK: Validating primary data. [2025-11-04T17:38:56.918936] [DEBUG] [VALIDATOR ] BIRK: All expected trading days are present. [2025-11-04T17:38:56.919017] [DEBUG] [VALIDATOR ] BIRK: Primary data validation passed. [2025-11-04T17:38:56.919771] [INFO ] [MAIN ] BIRK: Fetched primary data is valid. [2025-11-04T17:38:56.919834] [DEBUG] [LOCAL ] BIRK: Saving primary data. [2025-11-04T17:38:56.921255] [DEBUG] [LOCAL ] BIRK: Trimmed 73651 records to 51125 (keeping 200 days of data) [2025-11-04T17:38:56.969422] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-04. [2025-11-04T17:38:57.216179] [DEBUG] [LOCAL ] BIRK: Primary data saved (51125 records). [2025-11-04T17:38:57.216296] [INFO ] [MAIN ] BIRK: Full history saved. [2025-11-04T17:38:57.217249] [INFO ] [MAIN ] Update Progress: 285/2367 symbols completed. BIRK took 37.1s. Estimated remaining: 4h 44m 10s. Rate: 7.3 symbols/min [2025-11-04T17:38:57.391304] [INFO ] [MAIN ] BKD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:38:57.438441] [DEBUG] [LOCAL ] BKD: Loaded 50268 existing records from primary storage. [2025-11-04T17:38:57.438764] [INFO ] [MAIN ] BKD: Loaded 50268 existing records. [2025-11-04T17:38:57.620357] [WARN ] [MAIN ] BKD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:38:57.620627] [INFO ] [MAIN ] BKD: Wiping existing primary data before full refresh. [2025-11-04T17:38:57.620728] [INFO ] [LOCAL ] BKD: Starting deletion of primary data. [2025-11-04T17:38:57.620959] [DEBUG] [LOCAL ] BKD: Deleted primary data file. [2025-11-04T17:38:57.621011] [INFO ] [MAIN ] BKD: Starting full primary refresh. [2025-11-04T17:38:57.769736] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-07. [2025-11-04T17:38:57.972857] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-11-04T17:38:58.023704] [INFO ] [REMOTE ] BKD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:38:58.504933] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-11-04T17:38:58.567016] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-01. [2025-11-04T17:38:58.843381] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T17:38:59.367074] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-05. [2025-11-04T17:38:59.758271] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T17:39:00.167046] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-08. [2025-11-04T17:39:00.967139] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-02. [2025-11-04T17:39:01.137534] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-11-04T17:39:01.372065] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T17:39:01.767112] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-06. [2025-11-04T17:39:02.067133] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T17:39:02.567150] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-09. [2025-11-04T17:39:02.980175] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T17:39:03.367044] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-03. [2025-11-04T17:39:03.946074] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T17:39:04.167061] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-07. [2025-11-04T17:39:04.695845] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T17:39:04.967130] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-11. [2025-11-04T17:39:05.701550] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T17:39:05.767001] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-08. [2025-11-04T17:39:06.367198] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-11 (1.40 seconds). [2025-11-04T17:39:06.369148] [INFO ] [REMOTE ] BJ: Full fetch complete (74393 records). [2025-11-04T17:39:06.369240] [DEBUG] [VALIDATOR ] BJ: Validating primary data. [2025-11-04T17:39:06.572296] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-10. [2025-11-04T17:39:06.755606] [DEBUG] [VALIDATOR ] BJ: All expected trading days are present. [2025-11-04T17:39:06.755709] [DEBUG] [VALIDATOR ] BJ: Primary data validation passed. [2025-11-04T17:39:06.756446] [INFO ] [MAIN ] BJ: Fetched primary data is valid. [2025-11-04T17:39:06.756490] [DEBUG] [LOCAL ] BJ: Saving primary data. [2025-11-04T17:39:06.757884] [DEBUG] [LOCAL ] BJ: Trimmed 74393 records to 51340 (keeping 200 days of data) [2025-11-04T17:39:07.056025] [DEBUG] [LOCAL ] BJ: Primary data saved (51340 records). [2025-11-04T17:39:07.056291] [INFO ] [MAIN ] BJ: Full history saved. [2025-11-04T17:39:07.057277] [INFO ] [MAIN ] Update Progress: 286/2367 symbols completed. BJ took 35.0s. Estimated remaining: 4h 44m 14s. Rate: 7.3 symbols/min [2025-11-04T17:39:07.237520] [INFO ] [MAIN ] BKE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:39:07.274526] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T17:39:07.311837] [DEBUG] [LOCAL ] BKE: Loaded 32806 existing records from primary storage. [2025-11-04T17:39:07.312099] [INFO ] [MAIN ] BKE: Loaded 32806 existing records. [2025-11-04T17:39:07.375512] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-04. [2025-11-04T17:39:07.495207] [WARN ] [MAIN ] BKE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:39:07.495518] [INFO ] [MAIN ] BKE: Wiping existing primary data before full refresh. [2025-11-04T17:39:07.495606] [INFO ] [LOCAL ] BKE: Starting deletion of primary data. [2025-11-04T17:39:07.495969] [DEBUG] [LOCAL ] BKE: Deleted primary data file. [2025-11-04T17:39:07.496028] [INFO ] [MAIN ] BKE: Starting full primary refresh. [2025-11-04T17:39:07.726970] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-11-04T17:39:07.877691] [INFO ] [REMOTE ] BKE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:39:08.167029] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-11. [2025-11-04T17:39:08.614852] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T17:39:08.952286] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-11 (0.79 seconds). [2025-11-04T17:39:08.953713] [INFO ] [REMOTE ] BIT: Full fetch complete (32547 records). [2025-11-04T17:39:08.953794] [DEBUG] [VALIDATOR ] BIT: Validating primary data. [2025-11-04T17:39:08.972130] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-01. [2025-11-04T17:39:09.349456] [DEBUG] [VALIDATOR ] BIT: All expected trading days are present. [2025-11-04T17:39:09.349530] [DEBUG] [VALIDATOR ] BIT: Primary data validation passed. [2025-11-04T17:39:09.349955] [INFO ] [MAIN ] BIT: Fetched primary data is valid. [2025-11-04T17:39:09.350017] [DEBUG] [LOCAL ] BIT: Saving primary data. [2025-11-04T17:39:09.350772] [DEBUG] [LOCAL ] BIT: Trimmed 32547 records to 23805 (keeping 200 days of data) [2025-11-04T17:39:09.487725] [DEBUG] [LOCAL ] BIT: Primary data saved (23805 records). [2025-11-04T17:39:09.487885] [INFO ] [MAIN ] BIT: Full history saved. [2025-11-04T17:39:09.487972] [INFO ] [MAIN ] Update Progress: 287/2367 symbols completed. BIT took 44.4s. Estimated remaining: 4h 43m 24s. Rate: 7.3 symbols/min [2025-11-04T17:39:09.659223] [INFO ] [MAIN ] BKH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:39:09.699235] [DEBUG] [LOCAL ] BKH: Loaded 37564 existing records from primary storage. [2025-11-04T17:39:09.699521] [INFO ] [MAIN ] BKH: Loaded 37564 existing records. [2025-11-04T17:39:09.773621] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-09. [2025-11-04T17:39:09.876048] [WARN ] [MAIN ] BKH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:39:09.876248] [INFO ] [MAIN ] BKH: Wiping existing primary data before full refresh. [2025-11-04T17:39:09.876294] [INFO ] [LOCAL ] BKH: Starting deletion of primary data. [2025-11-04T17:39:09.876509] [DEBUG] [LOCAL ] BKH: Deleted primary data file. [2025-11-04T17:39:09.876565] [INFO ] [MAIN ] BKH: Starting full primary refresh. [2025-11-04T17:39:10.223455] [INFO ] [REMOTE ] BKH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:39:10.302067] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T17:39:10.567041] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-01. [2025-11-04T17:39:11.369890] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-02. [2025-11-04T17:39:11.381250] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-11-04T17:39:11.827021] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T17:39:12.167085] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-02. [2025-11-04T17:39:12.278968] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T17:39:12.967100] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-03. [2025-11-04T17:39:13.257749] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T17:39:13.767074] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-03. [2025-11-04T17:39:14.108117] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T17:39:14.567086] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-04. [2025-11-04T17:39:14.992082] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T17:39:15.367041] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-04. [2025-11-04T17:39:15.823260] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T17:39:16.167041] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-05. [2025-11-04T17:39:16.482028] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-11-04T17:39:16.967053] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-05. [2025-11-04T17:39:17.115579] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-11-04T17:39:17.767053] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-10. [2025-11-04T17:39:17.938913] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-11-04T17:39:18.567072] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-06. [2025-11-04T17:39:19.234584] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T17:39:19.367086] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-06. [2025-11-04T17:39:19.610584] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T17:39:20.167086] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-07. [2025-11-04T17:39:20.386432] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T17:39:20.967053] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-11. [2025-11-04T17:39:21.454713] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-11-04T17:39:21.587175] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T17:39:21.590098] [INFO ] [REMOTE ] BK: Full fetch complete (77661 records). [2025-11-04T17:39:21.590387] [DEBUG] [VALIDATOR ] BK: Validating primary data. [2025-11-04T17:39:21.775415] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-05. [2025-11-04T17:39:21.985066] [DEBUG] [VALIDATOR ] BK: All expected trading days are present. [2025-11-04T17:39:21.985146] [DEBUG] [VALIDATOR ] BK: Primary data validation passed. [2025-11-04T17:39:21.985958] [INFO ] [MAIN ] BK: Fetched primary data is valid. [2025-11-04T17:39:21.986021] [DEBUG] [LOCAL ] BK: Saving primary data. [2025-11-04T17:39:21.987453] [DEBUG] [LOCAL ] BK: Trimmed 77661 records to 53524 (keeping 200 days of data) [2025-11-04T17:39:22.313440] [DEBUG] [LOCAL ] BK: Primary data saved (53524 records). [2025-11-04T17:39:22.313598] [INFO ] [MAIN ] BK: Full history saved. [2025-11-04T17:39:22.313720] [INFO ] [MAIN ] Update Progress: 288/2367 symbols completed. BK took 36.5s. Estimated remaining: 4h 43m 49s. Rate: 7.3 symbols/min [2025-11-04T17:39:22.477475] [INFO ] [MAIN ] BKKT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:39:22.512287] [DEBUG] [LOCAL ] BKKT: Loaded 38846 existing records from primary storage. [2025-11-04T17:39:22.512563] [INFO ] [MAIN ] BKKT: Loaded 38846 existing records. [2025-11-04T17:39:22.573271] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-08. [2025-11-04T17:39:22.686042] [WARN ] [MAIN ] BKKT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:39:22.686258] [INFO ] [MAIN ] BKKT: Wiping existing primary data before full refresh. [2025-11-04T17:39:22.686304] [INFO ] [LOCAL ] BKKT: Starting deletion of primary data. [2025-11-04T17:39:22.686529] [DEBUG] [LOCAL ] BKKT: Deleted primary data file. [2025-11-04T17:39:22.686585] [INFO ] [MAIN ] BKKT: Starting full primary refresh. [2025-11-04T17:39:23.030112] [INFO ] [REMOTE ] BKKT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:39:23.258734] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T17:39:23.367015] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-06. [2025-11-04T17:39:23.821946] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T17:39:24.167063] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-09. [2025-11-04T17:39:24.410464] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T17:39:24.967029] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-07. [2025-11-04T17:39:25.141868] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-11-04T17:39:25.767040] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-10. [2025-11-04T17:39:25.998769] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T17:39:26.567058] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-01. [2025-11-04T17:39:26.716274] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T17:39:27.367076] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-11. [2025-11-04T17:39:27.885837] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T17:39:28.167042] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-07. [2025-11-04T17:39:28.298160] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T17:39:28.299872] [INFO ] [REMOTE ] BKH: Full fetch complete (53532 records). [2025-11-04T17:39:28.299956] [DEBUG] [VALIDATOR ] BKH: Validating primary data. [2025-11-04T17:39:28.689292] [DEBUG] [VALIDATOR ] BKH: All expected trading days are present. [2025-11-04T17:39:28.689371] [DEBUG] [VALIDATOR ] BKH: Primary data validation passed. [2025-11-04T17:39:28.689959] [INFO ] [MAIN ] BKH: Fetched primary data is valid. [2025-11-04T17:39:28.690020] [DEBUG] [LOCAL ] BKH: Saving primary data. [2025-11-04T17:39:28.691136] [DEBUG] [LOCAL ] BKH: Trimmed 53532 records to 37613 (keeping 200 days of data) [2025-11-04T17:39:28.914210] [DEBUG] [LOCAL ] BKH: Primary data saved (37613 records). [2025-11-04T17:39:28.914338] [INFO ] [MAIN ] BKH: Full history saved. [2025-11-04T17:39:28.915327] [INFO ] [MAIN ] Update Progress: 289/2367 symbols completed. BKH took 19.4s. Estimated remaining: 4h 43m 30s. Rate: 7.3 symbols/min [2025-11-04T17:39:28.978737] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-02. [2025-11-04T17:39:29.089262] [INFO ] [MAIN ] BKN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:39:29.099669] [DEBUG] [LOCAL ] BKN: Loaded 7311 existing records from primary storage. [2025-11-04T17:39:29.099903] [INFO ] [MAIN ] BKN: Loaded 7311 existing records. [2025-11-04T17:39:29.278532] [WARN ] [MAIN ] BKN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:39:29.278754] [INFO ] [MAIN ] BKN: Wiping existing primary data before full refresh. [2025-11-04T17:39:29.278824] [INFO ] [LOCAL ] BKN: Starting deletion of primary data. [2025-11-04T17:39:29.279002] [DEBUG] [LOCAL ] BKN: Deleted primary data file. [2025-11-04T17:39:29.279066] [INFO ] [MAIN ] BKN: Starting full primary refresh. [2025-11-04T17:39:29.643627] [INFO ] [REMOTE ] BKN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:39:29.672166] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T17:39:29.767010] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-08. [2025-11-04T17:39:29.876237] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T17:39:30.567063] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-08. [2025-11-04T17:39:31.000887] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T17:39:31.367094] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-01. [2025-11-04T17:39:31.690155] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-11-04T17:39:32.053519] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T17:39:32.167059] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-03. [2025-11-04T17:39:32.967041] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-02. [2025-11-04T17:39:33.452039] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T17:39:33.470540] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-11-04T17:39:33.767040] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-03. [2025-11-04T17:39:34.267286] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-11-04T17:39:34.567020] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-09. [2025-11-04T17:39:35.367076] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-04. [2025-11-04T17:39:35.424874] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-11-04T17:39:36.167016] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-04. [2025-11-04T17:39:36.288613] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T17:39:36.563873] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-11-04T17:39:36.967105] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-09. [2025-11-04T17:39:37.767041] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-05. [2025-11-04T17:39:38.161084] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-11-04T17:39:38.194295] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T17:39:38.567089] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-10. [2025-11-04T17:39:39.367076] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-06. [2025-11-04T17:39:39.506712] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T17:39:39.858776] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-11-04T17:39:40.167038] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-05. [2025-11-04T17:39:40.967109] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-10. [2025-11-04T17:39:41.187422] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T17:39:41.767063] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-07. [2025-11-04T17:39:42.280669] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T17:39:42.478694] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-11-04T17:39:42.567077] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-08. [2025-11-04T17:39:43.001024] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-11-04T17:39:43.367065] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-09. [2025-11-04T17:39:43.782369] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-11-04T17:39:44.167097] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-06. [2025-11-04T17:39:44.967101] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-11. [2025-11-04T17:39:45.210590] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T17:39:45.767081] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-07. [2025-11-04T17:39:46.047038] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-11 (1.08 seconds). [2025-11-04T17:39:46.048629] [INFO ] [REMOTE ] BKE: Full fetch complete (49592 records). [2025-11-04T17:39:46.048831] [DEBUG] [VALIDATOR ] BKE: Validating primary data. [2025-11-04T17:39:46.420823] [DEBUG] [VALIDATOR ] BKE: All expected trading days are present. [2025-11-04T17:39:46.420903] [DEBUG] [VALIDATOR ] BKE: Primary data validation passed. [2025-11-04T17:39:46.421440] [INFO ] [MAIN ] BKE: Fetched primary data is valid. [2025-11-04T17:39:46.421481] [DEBUG] [LOCAL ] BKE: Saving primary data. [2025-11-04T17:39:46.422497] [DEBUG] [LOCAL ] BKE: Trimmed 49592 records to 32873 (keeping 200 days of data) [2025-11-04T17:39:46.573698] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-10. [2025-11-04T17:39:46.619457] [DEBUG] [LOCAL ] BKE: Primary data saved (32873 records). [2025-11-04T17:39:46.619571] [INFO ] [MAIN ] BKE: Full history saved. [2025-11-04T17:39:46.620211] [INFO ] [MAIN ] Update Progress: 290/2367 symbols completed. BKE took 39.6s. Estimated remaining: 4h 44m 30s. Rate: 7.3 symbols/min [2025-11-04T17:39:46.798174] [INFO ] [MAIN ] BKSY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:39:46.846510] [DEBUG] [LOCAL ] BKSY: Loaded 49227 existing records from primary storage. [2025-11-04T17:39:46.846945] [INFO ] [MAIN ] BKSY: Loaded 49227 existing records. [2025-11-04T17:39:47.039112] [WARN ] [MAIN ] BKSY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:39:47.039445] [INFO ] [MAIN ] BKSY: Wiping existing primary data before full refresh. [2025-11-04T17:39:47.039571] [INFO ] [LOCAL ] BKSY: Starting deletion of primary data. [2025-11-04T17:39:47.039841] [DEBUG] [LOCAL ] BKSY: Deleted primary data file. [2025-11-04T17:39:47.039910] [INFO ] [MAIN ] BKSY: Starting full primary refresh. [2025-11-04T17:39:47.174035] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-11-04T17:39:47.376939] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-11. [2025-11-04T17:39:47.407617] [INFO ] [REMOTE ] BKSY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:39:47.432163] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-11-04T17:39:47.881053] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T17:39:47.882032] [INFO ] [REMOTE ] BKN: Full fetch complete (11184 records). [2025-11-04T17:39:47.882101] [DEBUG] [VALIDATOR ] BKN: Validating primary data. [2025-11-04T17:39:48.168508] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-08. [2025-11-04T17:39:48.250222] [DEBUG] [VALIDATOR ] BKN: All expected trading days are present. [2025-11-04T17:39:48.250387] [DEBUG] [VALIDATOR ] BKN: Primary data validation passed. [2025-11-04T17:39:48.250679] [INFO ] [MAIN ] BKN: Fetched primary data is valid. [2025-11-04T17:39:48.250739] [DEBUG] [LOCAL ] BKN: Saving primary data. [2025-11-04T17:39:48.251238] [DEBUG] [LOCAL ] BKN: Trimmed 11184 records to 7402 (keeping 200 days of data) [2025-11-04T17:39:48.297800] [DEBUG] [LOCAL ] BKN: Primary data saved (7402 records). [2025-11-04T17:39:48.297940] [INFO ] [MAIN ] BKN: Full history saved. [2025-11-04T17:39:48.298022] [INFO ] [MAIN ] Update Progress: 291/2367 symbols completed. BKN took 19.4s. Estimated remaining: 4h 43m 35s. Rate: 7.3 symbols/min [2025-11-04T17:39:48.470027] [INFO ] [MAIN ] BKT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:39:48.484183] [DEBUG] [LOCAL ] BKT: Loaded 12932 existing records from primary storage. [2025-11-04T17:39:48.484379] [INFO ] [MAIN ] BKT: Loaded 12932 existing records. [2025-11-04T17:39:48.661387] [WARN ] [MAIN ] BKT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:39:48.661612] [INFO ] [MAIN ] BKT: Wiping existing primary data before full refresh. [2025-11-04T17:39:48.661692] [INFO ] [LOCAL ] BKT: Starting deletion of primary data. [2025-11-04T17:39:48.661871] [DEBUG] [LOCAL ] BKT: Deleted primary data file. [2025-11-04T17:39:48.661926] [INFO ] [MAIN ] BKT: Starting full primary refresh. [2025-11-04T17:39:48.972249] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-01. [2025-11-04T17:39:49.011583] [INFO ] [REMOTE ] BKT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:39:49.648222] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T17:39:49.767028] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-09. [2025-11-04T17:39:50.363604] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-11-04T17:39:50.567243] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-02. [2025-11-04T17:39:51.229035] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-11-04T17:39:51.367012] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-10. [2025-11-04T17:39:52.026160] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-11-04T17:39:52.167022] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-03. [2025-11-04T17:39:52.967038] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-01. [2025-11-04T17:39:53.205486] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-11-04T17:39:53.419033] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-11-04T17:39:53.487456] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T17:39:53.767049] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-11. [2025-11-04T17:39:54.567049] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-04. [2025-11-04T17:39:55.028144] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-11 (1.26 seconds). [2025-11-04T17:39:55.030445] [INFO ] [REMOTE ] BKD: Full fetch complete (72271 records). [2025-11-04T17:39:55.030614] [DEBUG] [VALIDATOR ] BKD: Validating primary data. [2025-11-04T17:39:55.372224] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-02. [2025-11-04T17:39:55.444082] [DEBUG] [VALIDATOR ] BKD: All expected trading days are present. [2025-11-04T17:39:55.444160] [DEBUG] [VALIDATOR ] BKD: Primary data validation passed. [2025-11-04T17:39:55.444894] [INFO ] [MAIN ] BKD: Fetched primary data is valid. [2025-11-04T17:39:55.444956] [DEBUG] [LOCAL ] BKD: Saving primary data. [2025-11-04T17:39:55.446639] [DEBUG] [LOCAL ] BKD: Trimmed 72271 records to 50249 (keeping 200 days of data) [2025-11-04T17:39:55.726933] [DEBUG] [LOCAL ] BKD: Primary data saved (50249 records). [2025-11-04T17:39:55.727097] [INFO ] [MAIN ] BKD: Full history saved. [2025-11-04T17:39:55.727958] [INFO ] [MAIN ] Update Progress: 292/2367 symbols completed. BKD took 58.5s. Estimated remaining: 4h 43m 21s. Rate: 7.3 symbols/min [2025-11-04T17:39:55.905496] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-11-04T17:39:55.907804] [INFO ] [MAIN ] BKU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:39:55.939622] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-11-04T17:39:55.982574] [DEBUG] [LOCAL ] BKU: Loaded 42266 existing records from primary storage. [2025-11-04T17:39:55.982872] [INFO ] [MAIN ] BKU: Loaded 42266 existing records. [2025-11-04T17:39:56.156786] [WARN ] [MAIN ] BKU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:39:56.157000] [INFO ] [MAIN ] BKU: Wiping existing primary data before full refresh. [2025-11-04T17:39:56.157050] [INFO ] [LOCAL ] BKU: Starting deletion of primary data. [2025-11-04T17:39:56.157251] [DEBUG] [LOCAL ] BKU: Deleted primary data file. [2025-11-04T17:39:56.157309] [INFO ] [MAIN ] BKU: Starting full primary refresh. [2025-11-04T17:39:56.172155] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-11. [2025-11-04T17:39:56.514123] [INFO ] [REMOTE ] BKU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:39:56.967048] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-01. [2025-11-04T17:39:57.693544] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-11 (1.52 seconds). [2025-11-04T17:39:57.695458] [INFO ] [REMOTE ] BKKT: Full fetch complete (53173 records). [2025-11-04T17:39:57.695564] [DEBUG] [VALIDATOR ] BKKT: Validating primary data. [2025-11-04T17:39:57.772150] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-05. [2025-11-04T17:39:58.082553] [DEBUG] [VALIDATOR ] BKKT: All expected trading days are present. [2025-11-04T17:39:58.082686] [DEBUG] [VALIDATOR ] BKKT: Primary data validation passed. [2025-11-04T17:39:58.083438] [INFO ] [MAIN ] BKKT: Fetched primary data is valid. [2025-11-04T17:39:58.083515] [DEBUG] [LOCAL ] BKKT: Saving primary data. [2025-11-04T17:39:58.085019] [DEBUG] [LOCAL ] BKKT: Trimmed 53173 records to 39102 (keeping 200 days of data) [2025-11-04T17:39:58.222221] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T17:39:58.336730] [DEBUG] [LOCAL ] BKKT: Primary data saved (39102 records). [2025-11-04T17:39:58.336880] [INFO ] [MAIN ] BKKT: Full history saved. [2025-11-04T17:39:58.336969] [INFO ] [MAIN ] Update Progress: 293/2367 symbols completed. BKKT took 36.0s. Estimated remaining: 4h 42m 33s. Rate: 7.3 symbols/min [2025-11-04T17:39:58.506549] [INFO ] [MAIN ] BKV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:39:58.533704] [DEBUG] [LOCAL ] BKV: Loaded 29322 existing records from primary storage. [2025-11-04T17:39:58.533973] [INFO ] [MAIN ] BKV: Loaded 29322 existing records. [2025-11-04T17:39:58.576291] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-03. [2025-11-04T17:39:58.716994] [WARN ] [MAIN ] BKV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:39:58.717199] [INFO ] [MAIN ] BKV: Wiping existing primary data before full refresh. [2025-11-04T17:39:58.717247] [INFO ] [LOCAL ] BKV: Starting deletion of primary data. [2025-11-04T17:39:58.717412] [DEBUG] [LOCAL ] BKV: Deleted primary data file. [2025-11-04T17:39:58.717451] [INFO ] [MAIN ] BKV: Starting full primary refresh. [2025-11-04T17:39:59.080197] [INFO ] [REMOTE ] BKV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:39:59.109977] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T17:39:59.128364] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-11-04T17:39:59.367011] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-01. [2025-11-04T17:40:00.167084] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-06. [2025-11-04T17:40:00.717861] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T17:40:00.967055] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-04. [2025-11-04T17:40:01.767050] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-02. [2025-11-04T17:40:02.056331] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T17:40:02.227761] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-11-04T17:40:02.567119] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-07. [2025-11-04T17:40:02.957164] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T17:40:03.367152] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-02. [2025-11-04T17:40:04.167019] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-03. [2025-11-04T17:40:04.320190] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-11-04T17:40:04.372865] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-11-04T17:40:04.967087] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-05. [2025-11-04T17:40:05.238961] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T17:40:05.520157] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T17:40:05.767129] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-04. [2025-11-04T17:40:06.567055] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-06. [2025-11-04T17:40:06.953060] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T17:40:07.167702] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-11-04T17:40:07.367059] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-07. [2025-11-04T17:40:07.910613] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-11-04T17:40:08.167067] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-08. [2025-11-04T17:40:08.967091] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-05. [2025-11-04T17:40:09.704603] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-11-04T17:40:09.767075] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-08. [2025-11-04T17:40:10.339045] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-11-04T17:40:10.453866] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-11-04T17:40:10.567078] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-09. [2025-11-04T17:40:11.233963] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-11-04T17:40:11.367043] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-09. [2025-11-04T17:40:12.167085] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-10. [2025-11-04T17:40:12.958881] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T17:40:12.967012] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-11. [2025-11-04T17:40:13.083028] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-11-04T17:40:13.767072] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-10. [2025-11-04T17:40:14.567047] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-06. [2025-11-04T17:40:15.155461] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-11 (2.19 seconds). [2025-11-04T17:40:15.156475] [INFO ] [REMOTE ] BKT: Full fetch complete (17158 records). [2025-11-04T17:40:15.156547] [DEBUG] [VALIDATOR ] BKT: Validating primary data. [2025-11-04T17:40:15.372455] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-03. [2025-11-04T17:40:15.536611] [DEBUG] [VALIDATOR ] BKT: All expected trading days are present. [2025-11-04T17:40:15.536790] [DEBUG] [VALIDATOR ] BKT: Primary data validation passed. [2025-11-04T17:40:15.537204] [INFO ] [MAIN ] BKT: Fetched primary data is valid. [2025-11-04T17:40:15.537374] [DEBUG] [LOCAL ] BKT: Saving primary data. [2025-11-04T17:40:15.538062] [DEBUG] [LOCAL ] BKT: Trimmed 17158 records to 13031 (keeping 200 days of data) [2025-11-04T17:40:15.624771] [DEBUG] [LOCAL ] BKT: Primary data saved (13031 records). [2025-11-04T17:40:15.625967] [INFO ] [MAIN ] BKT: Full history saved. [2025-11-04T17:40:15.626202] [INFO ] [MAIN ] Update Progress: 294/2367 symbols completed. BKT took 27.3s. Estimated remaining: 4h 43m 29s. Rate: 7.3 symbols/min [2025-11-04T17:40:15.824127] [INFO ] [MAIN ] BLCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:40:15.824414] [DEBUG] [LOCAL ] BLCO: No existing data file found in primary storage. [2025-11-04T17:40:15.824503] [WARN ] [MAIN ] BLCO: No existing data found, performing full refresh instead. [2025-11-04T17:40:15.824845] [INFO ] [MAIN ] BLCO: Wiping existing primary data before full refresh. [2025-11-04T17:40:15.825003] [INFO ] [LOCAL ] BLCO: Starting deletion of primary data. [2025-11-04T17:40:15.825192] [DEBUG] [LOCAL ] BLCO: No primary data file found to delete. [2025-11-04T17:40:15.825251] [INFO ] [MAIN ] BLCO: Starting full primary refresh. [2025-11-04T17:40:15.847831] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T17:40:16.170412] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-07. [2025-11-04T17:40:16.245941] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-10 (2.48 seconds). [2025-11-04T17:40:16.243817] [INFO ] [REMOTE ] BLCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:40:16.558169] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T17:40:16.967100] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-01. [2025-11-04T17:40:17.434257] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T17:40:17.767012] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-11. [2025-11-04T17:40:17.972131] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T17:40:18.567031] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-04. [2025-11-04T17:40:19.182517] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-11 (1.42 seconds). [2025-11-04T17:40:19.184446] [INFO ] [REMOTE ] BKSY: Full fetch complete (71169 records). [2025-11-04T17:40:19.184542] [DEBUG] [VALIDATOR ] BKSY: Validating primary data. [2025-11-04T17:40:19.373872] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-08. [2025-11-04T17:40:19.582183] [DEBUG] [VALIDATOR ] BKSY: All expected trading days are present. [2025-11-04T17:40:19.582281] [DEBUG] [VALIDATOR ] BKSY: Primary data validation passed. [2025-11-04T17:40:19.583024] [INFO ] [MAIN ] BKSY: Fetched primary data is valid. [2025-11-04T17:40:19.583102] [DEBUG] [LOCAL ] BKSY: Saving primary data. [2025-11-04T17:40:19.601757] [DEBUG] [LOCAL ] BKSY: Trimmed 71169 records to 49317 (keeping 200 days of data) [2025-11-04T17:40:19.603218] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-11-04T17:40:19.893410] [DEBUG] [LOCAL ] BKSY: Primary data saved (49317 records). [2025-11-04T17:40:19.893537] [INFO ] [MAIN ] BKSY: Full history saved. [2025-11-04T17:40:19.893639] [INFO ] [MAIN ] Update Progress: 295/2367 symbols completed. BKSY took 33.3s. Estimated remaining: 4h 42m 54s. Rate: 7.3 symbols/min [2025-11-04T17:40:20.064772] [INFO ] [MAIN ] BLD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:40:20.089661] [DEBUG] [LOCAL ] BLD: Loaded 26638 existing records from primary storage. [2025-11-04T17:40:20.089931] [INFO ] [MAIN ] BLD: Loaded 26638 existing records. [2025-11-04T17:40:20.172245] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-02. [2025-11-04T17:40:20.279530] [WARN ] [MAIN ] BLD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:40:20.279800] [INFO ] [MAIN ] BLD: Wiping existing primary data before full refresh. [2025-11-04T17:40:20.279848] [INFO ] [LOCAL ] BLD: Starting deletion of primary data. [2025-11-04T17:40:20.280055] [DEBUG] [LOCAL ] BLD: Deleted primary data file. [2025-11-04T17:40:20.280110] [INFO ] [MAIN ] BLD: Starting full primary refresh. [2025-11-04T17:40:20.630237] [INFO ] [REMOTE ] BLD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:40:20.967119] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-01. [2025-11-04T17:40:21.194415] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T17:40:21.767033] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-03. [2025-11-04T17:40:21.926802] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T17:40:21.988522] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-08 (2.61 seconds). [2025-11-04T17:40:22.567095] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-02. [2025-11-04T17:40:22.842240] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T17:40:23.367566] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-09. [2025-11-04T17:40:23.371289] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-11-04T17:40:24.167113] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-05. [2025-11-04T17:40:24.352915] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T17:40:24.967023] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-03. [2025-11-04T17:40:25.092673] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-11-04T17:40:25.767105] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-06. [2025-11-04T17:40:25.929202] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T17:40:26.567124] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-10. [2025-11-04T17:40:26.855328] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T17:40:27.367040] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-04. [2025-11-04T17:40:27.755823] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-11-04T17:40:28.167015] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-11. [2025-11-04T17:40:28.264733] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-11-04T17:40:28.967021] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-07. [2025-11-04T17:40:29.130174] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-11 (0.96 seconds). [2025-11-04T17:40:29.132230] [INFO ] [REMOTE ] BKU: Full fetch complete (61271 records). [2025-11-04T17:40:29.132369] [DEBUG] [VALIDATOR ] BKU: Validating primary data. [2025-11-04T17:40:29.602329] [DEBUG] [VALIDATOR ] BKU: All expected trading days are present. [2025-11-04T17:40:29.602424] [DEBUG] [VALIDATOR ] BKU: Primary data validation passed. [2025-11-04T17:40:29.603164] [INFO ] [MAIN ] BKU: Fetched primary data is valid. [2025-11-04T17:40:29.603230] [DEBUG] [LOCAL ] BKU: Saving primary data. [2025-11-04T17:40:29.604715] [DEBUG] [LOCAL ] BKU: Trimmed 61271 records to 42272 (keeping 200 days of data) [2025-11-04T17:40:29.800560] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-04. [2025-11-04T17:40:29.895733] [DEBUG] [LOCAL ] BKU: Primary data saved (42272 records). [2025-11-04T17:40:29.895881] [INFO ] [MAIN ] BKU: Full history saved. [2025-11-04T17:40:29.895992] [INFO ] [MAIN ] Update Progress: 296/2367 symbols completed. BKU took 34.2s. Estimated remaining: 4h 42m 58s. Rate: 7.3 symbols/min [2025-11-04T17:40:30.042044] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T17:40:30.112729] [INFO ] [MAIN ] BLDR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:40:30.162246] [DEBUG] [LOCAL ] BLDR: Loaded 51859 existing records from primary storage. [2025-11-04T17:40:30.162580] [INFO ] [MAIN ] BLDR: Loaded 51859 existing records. [2025-11-04T17:40:30.336486] [WARN ] [MAIN ] BLDR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:40:30.336726] [INFO ] [MAIN ] BLDR: Wiping existing primary data before full refresh. [2025-11-04T17:40:30.336785] [INFO ] [LOCAL ] BLDR: Starting deletion of primary data. [2025-11-04T17:40:30.337022] [DEBUG] [LOCAL ] BLDR: Deleted primary data file. [2025-11-04T17:40:30.337078] [INFO ] [MAIN ] BLDR: Starting full primary refresh. [2025-11-04T17:40:30.577592] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-08. [2025-11-04T17:40:30.701988] [INFO ] [REMOTE ] BLDR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:40:31.249511] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T17:40:31.367014] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-05. [2025-11-04T17:40:31.407672] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-11-04T17:40:32.167030] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-05. [2025-11-04T17:40:32.322831] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T17:40:32.967057] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-06. [2025-11-04T17:40:33.135235] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-11-04T17:40:33.767018] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-09. [2025-11-04T17:40:33.807836] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T17:40:34.567021] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-07. [2025-11-04T17:40:34.613721] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-11-04T17:40:35.367062] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-01. [2025-11-04T17:40:35.681483] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T17:40:36.167087] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-08. [2025-11-04T17:40:36.822454] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-11-04T17:40:36.895768] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-11-04T17:40:36.967046] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-10. [2025-11-04T17:40:37.767052] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-06. [2025-11-04T17:40:38.068608] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T17:40:38.567007] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-09. [2025-11-04T17:40:38.782537] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T17:40:39.367059] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-07. [2025-11-04T17:40:39.449019] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T17:40:40.167041] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-11. [2025-11-04T17:40:40.353317] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-11-04T17:40:40.967028] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-02. [2025-11-04T17:40:41.040858] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T17:40:41.042447] [INFO ] [REMOTE ] BKV: Full fetch complete (45252 records). [2025-11-04T17:40:41.042528] [DEBUG] [VALIDATOR ] BKV: Validating primary data. [2025-11-04T17:40:41.407871] [DEBUG] [VALIDATOR ] BKV: All expected trading days are present. [2025-11-04T17:40:41.407977] [DEBUG] [VALIDATOR ] BKV: Primary data validation passed. [2025-11-04T17:40:41.408540] [INFO ] [MAIN ] BKV: Fetched primary data is valid. [2025-11-04T17:40:41.408583] [DEBUG] [LOCAL ] BKV: Saving primary data. [2025-11-04T17:40:41.409576] [DEBUG] [LOCAL ] BKV: Trimmed 45252 records to 29387 (keeping 200 days of data) [2025-11-04T17:40:41.573427] [DEBUG] [LOCAL ] BKV: Primary data saved (29387 records). [2025-11-04T17:40:41.573549] [INFO ] [MAIN ] BKV: Full history saved. [2025-11-04T17:40:41.573625] [INFO ] [MAIN ] Update Progress: 297/2367 symbols completed. BKV took 43.2s. Estimated remaining: 4h 43m 14s. Rate: 7.3 symbols/min [2025-11-04T17:40:41.738355] [INFO ] [MAIN ] BLE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:40:41.752922] [DEBUG] [LOCAL ] BLE: Loaded 13283 existing records from primary storage. [2025-11-04T17:40:41.753155] [INFO ] [MAIN ] BLE: Loaded 13283 existing records. [2025-11-04T17:40:41.773412] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-10. [2025-11-04T17:40:41.930040] [WARN ] [MAIN ] BLE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:40:41.930357] [INFO ] [MAIN ] BLE: Wiping existing primary data before full refresh. [2025-11-04T17:40:41.930423] [INFO ] [LOCAL ] BLE: Starting deletion of primary data. [2025-11-04T17:40:41.930638] [DEBUG] [LOCAL ] BLE: Deleted primary data file. [2025-11-04T17:40:41.930739] [INFO ] [MAIN ] BLE: Starting full primary refresh. [2025-11-04T17:40:42.281900] [INFO ] [REMOTE ] BLE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:40:42.306892] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-11-04T17:40:42.567021] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-08. [2025-11-04T17:40:42.699992] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-11-04T17:40:43.367010] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-03. [2025-11-04T17:40:43.528437] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-11-04T17:40:44.167043] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-09. [2025-11-04T17:40:44.665285] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T17:40:44.967136] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-11. [2025-11-04T17:40:45.062434] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-11-04T17:40:45.688440] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T17:40:45.690116] [INFO ] [REMOTE ] BLD: Full fetch complete (39986 records). [2025-11-04T17:40:45.690197] [DEBUG] [VALIDATOR ] BLD: Validating primary data. [2025-11-04T17:40:45.772116] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-01. [2025-11-04T17:40:46.067916] [DEBUG] [VALIDATOR ] BLD: All expected trading days are present. [2025-11-04T17:40:46.067993] [DEBUG] [VALIDATOR ] BLD: Primary data validation passed. [2025-11-04T17:40:46.068490] [INFO ] [MAIN ] BLD: Fetched primary data is valid. [2025-11-04T17:40:46.068530] [DEBUG] [LOCAL ] BLD: Saving primary data. [2025-11-04T17:40:46.069464] [DEBUG] [LOCAL ] BLD: Trimmed 39986 records to 26828 (keeping 200 days of data) [2025-11-04T17:40:46.224920] [DEBUG] [LOCAL ] BLD: Primary data saved (26828 records). [2025-11-04T17:40:46.225040] [INFO ] [MAIN ] BLD: Full history saved. [2025-11-04T17:40:46.225120] [INFO ] [MAIN ] Update Progress: 298/2367 symbols completed. BLD took 26.3s. Estimated remaining: 4h 42m 41s. Rate: 7.3 symbols/min [2025-11-04T17:40:46.391120] [INFO ] [MAIN ] BLK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:40:46.424991] [DEBUG] [LOCAL ] BLK: Loaded 33489 existing records from primary storage. [2025-11-04T17:40:46.425264] [INFO ] [MAIN ] BLK: Loaded 33489 existing records. [2025-11-04T17:40:46.584583] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-04. [2025-11-04T17:40:46.586734] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T17:40:46.617556] [WARN ] [MAIN ] BLK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:40:46.621193] [INFO ] [MAIN ] BLK: Wiping existing primary data before full refresh. [2025-11-04T17:40:46.621252] [INFO ] [LOCAL ] BLK: Starting deletion of primary data. [2025-11-04T17:40:46.621433] [DEBUG] [LOCAL ] BLK: Deleted primary data file. [2025-11-04T17:40:46.621469] [INFO ] [MAIN ] BLK: Starting full primary refresh. [2025-11-04T17:40:46.964674] [INFO ] [REMOTE ] BLK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:40:47.367100] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-01. [2025-11-04T17:40:48.097224] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T17:40:48.167002] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-05. [2025-11-04T17:40:48.579600] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T17:40:48.967033] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-02. [2025-11-04T17:40:49.499524] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-11-04T17:40:49.499233] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T17:40:49.767030] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-03. [2025-11-04T17:40:50.338447] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T17:40:50.567025] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-04. [2025-11-04T17:40:51.250299] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T17:40:51.367029] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-06. [2025-11-04T17:40:52.167034] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-05. [2025-11-04T17:40:52.706393] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T17:40:52.771761] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-11-04T17:40:52.967089] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-10. [2025-11-04T17:40:53.767056] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-02. [2025-11-04T17:40:54.106493] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T17:40:54.567037] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-07. [2025-11-04T17:40:54.927057] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T17:40:55.367035] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-03. [2025-11-04T17:40:56.022480] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T17:40:56.167021] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-08. [2025-11-04T17:40:56.584929] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T17:40:56.967033] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-06. [2025-11-04T17:40:57.321566] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-11-04T17:40:57.574827] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T17:40:57.767086] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-04. [2025-11-04T17:40:58.567087] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-09. [2025-11-04T17:40:59.253411] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-11-04T17:40:59.367074] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-07. [2025-11-04T17:40:59.917865] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-11-04T17:41:00.123004] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-11-04T17:41:00.167004] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-08. [2025-11-04T17:41:00.834098] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T17:41:00.967105] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-05. [2025-11-04T17:41:01.767116] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-10. [2025-11-04T17:41:02.182547] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T17:41:02.567083] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-09. [2025-11-04T17:41:03.044235] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-11-04T17:41:03.230038] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T17:41:03.367072] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-11. [2025-11-04T17:41:04.167085] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-10. [2025-11-04T17:41:04.511278] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T17:41:04.513490] [INFO ] [REMOTE ] BLDR: Full fetch complete (74792 records). [2025-11-04T17:41:04.513632] [DEBUG] [VALIDATOR ] BLDR: Validating primary data. [2025-11-04T17:41:04.849368] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-11-04T17:41:04.935956] [DEBUG] [VALIDATOR ] BLDR: All expected trading days are present. [2025-11-04T17:41:04.936049] [DEBUG] [VALIDATOR ] BLDR: Primary data validation passed. [2025-11-04T17:41:04.936897] [INFO ] [MAIN ] BLDR: Fetched primary data is valid. [2025-11-04T17:41:04.936973] [DEBUG] [LOCAL ] BLDR: Saving primary data. [2025-11-04T17:41:04.938408] [DEBUG] [LOCAL ] BLDR: Trimmed 74792 records to 51860 (keeping 200 days of data) [2025-11-04T17:41:04.988855] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-06. [2025-11-04T17:41:05.247084] [DEBUG] [LOCAL ] BLDR: Primary data saved (51860 records). [2025-11-04T17:41:05.247239] [INFO ] [MAIN ] BLDR: Full history saved. [2025-11-04T17:41:05.247895] [INFO ] [MAIN ] Update Progress: 299/2367 symbols completed. BLDR took 35.4s. Estimated remaining: 4h 43m 48s. Rate: 7.3 symbols/min [2025-11-04T17:41:05.420616] [INFO ] [MAIN ] BLND: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:41:05.472920] [DEBUG] [LOCAL ] BLND: Loaded 49214 existing records from primary storage. [2025-11-04T17:41:05.473206] [INFO ] [MAIN ] BLND: Loaded 49214 existing records. [2025-11-04T17:41:05.654918] [WARN ] [MAIN ] BLND: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:41:05.655129] [INFO ] [MAIN ] BLND: Wiping existing primary data before full refresh. [2025-11-04T17:41:05.655175] [INFO ] [LOCAL ] BLND: Starting deletion of primary data. [2025-11-04T17:41:05.655351] [DEBUG] [LOCAL ] BLND: Deleted primary data file. [2025-11-04T17:41:05.655388] [INFO ] [MAIN ] BLND: Starting full primary refresh. [2025-11-04T17:41:05.772210] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-11. [2025-11-04T17:41:06.012244] [INFO ] [REMOTE ] BLND: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:41:06.150535] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T17:41:06.567042] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-01. [2025-11-04T17:41:06.626676] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-11 (0.85 seconds). [2025-11-04T17:41:06.628164] [INFO ] [REMOTE ] BLCO: Full fetch complete (52112 records). [2025-11-04T17:41:06.628261] [DEBUG] [VALIDATOR ] BLCO: Validating primary data. [2025-11-04T17:41:06.998242] [DEBUG] [VALIDATOR ] BLCO: All expected trading days are present. [2025-11-04T17:41:06.998321] [DEBUG] [VALIDATOR ] BLCO: Primary data validation passed. [2025-11-04T17:41:06.998901] [INFO ] [MAIN ] BLCO: Fetched primary data is valid. [2025-11-04T17:41:06.998964] [DEBUG] [LOCAL ] BLCO: Saving primary data. [2025-11-04T17:41:07.000024] [DEBUG] [LOCAL ] BLCO: Trimmed 52112 records to 34229 (keeping 200 days of data) [2025-11-04T17:41:07.193058] [DEBUG] [LOCAL ] BLCO: Primary data saved (34229 records). [2025-11-04T17:41:07.193231] [INFO ] [MAIN ] BLCO: Full history saved. [2025-11-04T17:41:07.193308] [INFO ] [MAIN ] Update Progress: 300/2367 symbols completed. BLCO took 51.6s. Estimated remaining: 4h 42m 56s. Rate: 7.3 symbols/min [2025-11-04T17:41:07.363989] [INFO ] [MAIN ] BLSH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:41:07.364171] [DEBUG] [LOCAL ] BLSH: No existing data file found in primary storage. [2025-11-04T17:41:07.364225] [WARN ] [MAIN ] BLSH: No existing data found, performing full refresh instead. [2025-11-04T17:41:07.364331] [INFO ] [MAIN ] BLSH: Wiping existing primary data before full refresh. [2025-11-04T17:41:07.364372] [INFO ] [LOCAL ] BLSH: Starting deletion of primary data. [2025-11-04T17:41:07.364412] [DEBUG] [LOCAL ] BLSH: No primary data file found to delete. [2025-11-04T17:41:07.364444] [INFO ] [MAIN ] BLSH: Starting full primary refresh. [2025-11-04T17:41:07.375609] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-07. [2025-11-04T17:41:07.730040] [INFO ] [REMOTE ] BLSH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:41:07.872465] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T17:41:08.167095] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-01. [2025-11-04T17:41:08.339259] [DEBUG] [REMOTE ] BLSH: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:41:08.339562] [WARN ] [REMOTE ] BLSH: No data returned for month 2025-01 [2025-11-04T17:41:08.339676] [FATAL] [REMOTE ] BLSH: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:41:08.339735] [FATAL] [MAIN ] BLSH: Failed to fetch primary data. [2025-11-04T17:41:08.339830] [WARN ] [MAIN ] BLSH: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:41:08.339918] [INFO ] [LOCAL ] BLSH: Starting deletion of primary data. [2025-11-04T17:41:08.339998] [DEBUG] [LOCAL ] BLSH: No primary data file found to delete. [2025-11-04T17:41:08.340048] [INFO ] [LOCAL ] BLSH: Starting deletion of derivative data. [2025-11-04T17:41:08.340104] [DEBUG] [LOCAL ] BLSH: No derivative data file found to delete. [2025-11-04T17:41:08.340165] [INFO ] [MAIN ] Update Progress: 301/2367 symbols completed. BLSH took 1.1s. Estimated remaining: 4h 41m 60s. Rate: 7.3 symbols/min [2025-11-04T17:41:08.511801] [INFO ] [MAIN ] BLW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:41:08.526067] [DEBUG] [LOCAL ] BLW: Loaded 12668 existing records from primary storage. [2025-11-04T17:41:08.526294] [INFO ] [MAIN ] BLW: Loaded 12668 existing records. [2025-11-04T17:41:08.724592] [WARN ] [MAIN ] BLW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:41:08.725296] [INFO ] [MAIN ] BLW: Wiping existing primary data before full refresh. [2025-11-04T17:41:08.725356] [INFO ] [LOCAL ] BLW: Starting deletion of primary data. [2025-11-04T17:41:08.725542] [DEBUG] [LOCAL ] BLW: Deleted primary data file. [2025-11-04T17:41:08.725611] [INFO ] [MAIN ] BLW: Starting full primary refresh. [2025-11-04T17:41:08.969336] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-02. [2025-11-04T17:41:09.127324] [INFO ] [REMOTE ] BLW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:41:09.149557] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-11-04T17:41:09.767037] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-08. [2025-11-04T17:41:10.218228] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-11-04T17:41:10.567049] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-11. [2025-11-04T17:41:10.797049] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-11-04T17:41:11.253161] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-11 (0.69 seconds). [2025-11-04T17:41:11.254364] [INFO ] [REMOTE ] BLE: Full fetch complete (19926 records). [2025-11-04T17:41:11.254438] [DEBUG] [VALIDATOR ] BLE: Validating primary data. [2025-11-04T17:41:11.372125] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-09. [2025-11-04T17:41:11.636905] [DEBUG] [VALIDATOR ] BLE: All expected trading days are present. [2025-11-04T17:41:11.636981] [DEBUG] [VALIDATOR ] BLE: Primary data validation passed. [2025-11-04T17:41:11.637304] [INFO ] [MAIN ] BLE: Fetched primary data is valid. [2025-11-04T17:41:11.637341] [DEBUG] [LOCAL ] BLE: Saving primary data. [2025-11-04T17:41:11.637900] [DEBUG] [LOCAL ] BLE: Trimmed 19926 records to 13373 (keeping 200 days of data) [2025-11-04T17:41:11.714453] [DEBUG] [LOCAL ] BLE: Primary data saved (13373 records). [2025-11-04T17:41:11.714578] [INFO ] [MAIN ] BLE: Full history saved. [2025-11-04T17:41:11.714677] [INFO ] [MAIN ] Update Progress: 302/2367 symbols completed. BLE took 30.1s. Estimated remaining: 4h 41m 18s. Rate: 7.3 symbols/min [2025-11-04T17:41:11.890298] [INFO ] [MAIN ] BLX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:41:11.890453] [DEBUG] [LOCAL ] BLX: No existing data file found in primary storage. [2025-11-04T17:41:11.890488] [WARN ] [MAIN ] BLX: No existing data found, performing full refresh instead. [2025-11-04T17:41:11.890596] [INFO ] [MAIN ] BLX: Wiping existing primary data before full refresh. [2025-11-04T17:41:11.890635] [INFO ] [LOCAL ] BLX: Starting deletion of primary data. [2025-11-04T17:41:11.890718] [DEBUG] [LOCAL ] BLX: No primary data file found to delete. [2025-11-04T17:41:11.890773] [INFO ] [MAIN ] BLX: Starting full primary refresh. [2025-11-04T17:41:12.173811] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-03. [2025-11-04T17:41:12.258416] [INFO ] [REMOTE ] BLX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:41:12.563560] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T17:41:12.967051] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-10. [2025-11-04T17:41:13.422603] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-11-04T17:41:13.767045] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-04. [2025-11-04T17:41:14.567072] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-01. [2025-11-04T17:41:14.901742] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-11-04T17:41:14.940896] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-11-04T17:41:15.367128] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-01. [2025-11-04T17:41:15.784350] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-11-04T17:41:16.069845] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-11-04T17:41:16.167050] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-11. [2025-11-04T17:41:16.967071] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-05. [2025-11-04T17:41:17.767054] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-02. [2025-11-04T17:41:17.909485] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-11 (1.74 seconds). [2025-11-04T17:41:17.911472] [INFO ] [REMOTE ] BLK: Full fetch complete (52597 records). [2025-11-04T17:41:17.911553] [DEBUG] [VALIDATOR ] BLK: Validating primary data. [2025-11-04T17:41:18.302419] [DEBUG] [VALIDATOR ] BLK: All expected trading days are present. [2025-11-04T17:41:18.302670] [DEBUG] [VALIDATOR ] BLK: Primary data validation passed. [2025-11-04T17:41:18.303312] [INFO ] [MAIN ] BLK: Fetched primary data is valid. [2025-11-04T17:41:18.303391] [DEBUG] [LOCAL ] BLK: Saving primary data. [2025-11-04T17:41:18.304799] [DEBUG] [LOCAL ] BLK: Trimmed 52597 records to 33598 (keeping 200 days of data) [2025-11-04T17:41:18.427095] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T17:41:18.447920] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-11-04T17:41:18.567445] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-03. [2025-11-04T17:41:18.569579] [DEBUG] [LOCAL ] BLK: Primary data saved (33598 records). [2025-11-04T17:41:18.570343] [INFO ] [MAIN ] BLK: Full history saved. [2025-11-04T17:41:18.571531] [INFO ] [MAIN ] Update Progress: 303/2367 symbols completed. BLK took 32.3s. Estimated remaining: 4h 41m 1s. Rate: 7.3 symbols/min [2025-11-04T17:41:18.740694] [INFO ] [MAIN ] BMA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:41:18.777259] [DEBUG] [LOCAL ] BMA: Loaded 36868 existing records from primary storage. [2025-11-04T17:41:18.777522] [INFO ] [MAIN ] BMA: Loaded 36868 existing records. [2025-11-04T17:41:18.952369] [WARN ] [MAIN ] BMA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:41:18.952578] [INFO ] [MAIN ] BMA: Wiping existing primary data before full refresh. [2025-11-04T17:41:18.952624] [INFO ] [LOCAL ] BMA: Starting deletion of primary data. [2025-11-04T17:41:18.952878] [DEBUG] [LOCAL ] BMA: Deleted primary data file. [2025-11-04T17:41:18.952935] [INFO ] [MAIN ] BMA: Starting full primary refresh. [2025-11-04T17:41:19.306002] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T17:41:19.310468] [INFO ] [REMOTE ] BMA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:41:19.367009] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-01. [2025-11-04T17:41:20.167052] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-02. [2025-11-04T17:41:20.528337] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T17:41:20.824616] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-11-04T17:41:20.967056] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-04. [2025-11-04T17:41:21.623961] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-11-04T17:41:21.767092] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-06. [2025-11-04T17:41:22.567040] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-02. [2025-11-04T17:41:22.948961] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T17:41:23.367027] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-07. [2025-11-04T17:41:23.581960] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-11-04T17:41:24.167060] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-05. [2025-11-04T17:41:24.575461] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-11-04T17:41:24.629757] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-11-04T17:41:24.967077] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-03. [2025-11-04T17:41:25.767035] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-08. [2025-11-04T17:41:25.930078] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T17:41:26.567069] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-03. [2025-11-04T17:41:26.986970] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T17:41:27.346554] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-11-04T17:41:27.366999] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-04. [2025-11-04T17:41:28.112177] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T17:41:28.167019] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-09. [2025-11-04T17:41:28.967062] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-04. [2025-11-04T17:41:29.523767] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T17:41:29.767015] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-10. [2025-11-04T17:41:29.920089] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-11-04T17:41:30.567030] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-05. [2025-11-04T17:41:31.117164] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T17:41:31.295112] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T17:41:31.367045] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-05. [2025-11-04T17:41:32.167090] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-06. [2025-11-04T17:41:32.400892] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-11-04T17:41:32.798157] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-11-04T17:41:32.967060] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-06. [2025-11-04T17:41:33.576450] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T17:41:33.767054] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-06. [2025-11-04T17:41:34.567051] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-07. [2025-11-04T17:41:34.727233] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-11-04T17:41:35.119809] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-11-04T17:41:35.367046] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-08. [2025-11-04T17:41:35.984476] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-11-04T17:41:36.167072] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-09. [2025-11-04T17:41:36.788191] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-11-04T17:41:36.967043] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-07. [2025-11-04T17:41:37.767050] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-07. [2025-11-04T17:41:38.019056] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-11-04T17:41:38.281163] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-11-04T17:41:38.567051] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-10. [2025-11-04T17:41:39.367080] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-08. [2025-11-04T17:41:39.425930] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-11-04T17:41:39.911898] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-11-04T17:41:40.167042] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-11. [2025-11-04T17:41:40.825794] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T17:41:40.826806] [INFO ] [REMOTE ] BLX: Full fetch complete (22552 records). [2025-11-04T17:41:40.826919] [DEBUG] [VALIDATOR ] BLX: Validating primary data. [2025-11-04T17:41:40.976076] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-08. [2025-11-04T17:41:41.198363] [DEBUG] [VALIDATOR ] BLX: All expected trading days are present. [2025-11-04T17:41:41.198472] [DEBUG] [VALIDATOR ] BLX: Primary data validation passed. [2025-11-04T17:41:41.198910] [INFO ] [MAIN ] BLX: Fetched primary data is valid. [2025-11-04T17:41:41.199021] [DEBUG] [LOCAL ] BLX: Saving primary data. [2025-11-04T17:41:41.199783] [DEBUG] [LOCAL ] BLX: Trimmed 22552 records to 14949 (keeping 200 days of data) [2025-11-04T17:41:41.287209] [DEBUG] [LOCAL ] BLX: Primary data saved (14949 records). [2025-11-04T17:41:41.287351] [INFO ] [MAIN ] BLX: Full history saved. [2025-11-04T17:41:41.287424] [INFO ] [MAIN ] Update Progress: 304/2367 symbols completed. BLX took 29.6s. Estimated remaining: 4h 42m 32s. Rate: 7.3 symbols/min [2025-11-04T17:41:41.454897] [INFO ] [MAIN ] BME: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:41:41.455026] [DEBUG] [LOCAL ] BME: No existing data file found in primary storage. [2025-11-04T17:41:41.455064] [WARN ] [MAIN ] BME: No existing data found, performing full refresh instead. [2025-11-04T17:41:41.455170] [INFO ] [MAIN ] BME: Wiping existing primary data before full refresh. [2025-11-04T17:41:41.455213] [INFO ] [LOCAL ] BME: Starting deletion of primary data. [2025-11-04T17:41:41.455254] [DEBUG] [LOCAL ] BME: No primary data file found to delete. [2025-11-04T17:41:41.455286] [INFO ] [MAIN ] BME: Starting full primary refresh. [2025-11-04T17:41:41.772163] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-09. [2025-11-04T17:41:41.843915] [INFO ] [REMOTE ] BME: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:41:42.093267] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-11-04T17:41:42.391849] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-11-04T17:41:42.567085] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-10. [2025-11-04T17:41:43.364998] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T17:41:43.368492] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-11. [2025-11-04T17:41:44.167039] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-09. [2025-11-04T17:41:44.531355] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T17:41:44.533376] [INFO ] [REMOTE ] BLND: Full fetch complete (72139 records). [2025-11-04T17:41:44.533461] [DEBUG] [VALIDATOR ] BLND: Validating primary data. [2025-11-04T17:41:44.937526] [DEBUG] [VALIDATOR ] BLND: All expected trading days are present. [2025-11-04T17:41:44.937607] [DEBUG] [VALIDATOR ] BLND: Primary data validation passed. [2025-11-04T17:41:44.938577] [INFO ] [MAIN ] BLND: Fetched primary data is valid. [2025-11-04T17:41:44.938639] [DEBUG] [LOCAL ] BLND: Saving primary data. [2025-11-04T17:41:44.940429] [DEBUG] [LOCAL ] BLND: Trimmed 72139 records to 49200 (keeping 200 days of data) [2025-11-04T17:41:44.987368] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-11. [2025-11-04T17:41:45.210468] [DEBUG] [LOCAL ] BLND: Primary data saved (49200 records). [2025-11-04T17:41:45.210599] [INFO ] [MAIN ] BLND: Full history saved. [2025-11-04T17:41:45.211266] [INFO ] [MAIN ] Update Progress: 305/2367 symbols completed. BLND took 40.0s. Estimated remaining: 4h 41m 55s. Rate: 7.3 symbols/min [2025-11-04T17:41:45.383696] [INFO ] [MAIN ] BMEZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:41:45.407164] [DEBUG] [LOCAL ] BMEZ: Loaded 24406 existing records from primary storage. [2025-11-04T17:41:45.407414] [INFO ] [MAIN ] BMEZ: Loaded 24406 existing records. [2025-11-04T17:41:45.567576] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-11 (0.58 seconds). [2025-11-04T17:41:45.598149] [INFO ] [REMOTE ] BLW: Full fetch complete (19210 records). [2025-11-04T17:41:45.597992] [WARN ] [MAIN ] BMEZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:41:45.621047] [INFO ] [MAIN ] BMEZ: Wiping existing primary data before full refresh. [2025-11-04T17:41:45.620924] [DEBUG] [VALIDATOR ] BLW: Validating primary data. [2025-11-04T17:41:45.621352] [INFO ] [LOCAL ] BMEZ: Starting deletion of primary data. [2025-11-04T17:41:45.623007] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-11-04T17:41:45.623116] [DEBUG] [LOCAL ] BMEZ: Deleted primary data file. [2025-11-04T17:41:45.675621] [INFO ] [MAIN ] BMEZ: Starting full primary refresh. [2025-11-04T17:41:45.775040] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-10. [2025-11-04T17:41:46.357563] [DEBUG] [VALIDATOR ] BLW: All expected trading days are present. [2025-11-04T17:41:46.358484] [DEBUG] [VALIDATOR ] BLW: Primary data validation passed. [2025-11-04T17:41:46.374441] [INFO ] [MAIN ] BLW: Fetched primary data is valid. [2025-11-04T17:41:46.380010] [DEBUG] [LOCAL ] BLW: Saving primary data. [2025-11-04T17:41:46.379958] [INFO ] [REMOTE ] BMEZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:41:46.380788] [DEBUG] [LOCAL ] BLW: Trimmed 19210 records to 12672 (keeping 200 days of data) [2025-11-04T17:41:46.454508] [DEBUG] [LOCAL ] BLW: Primary data saved (12672 records). [2025-11-04T17:41:46.454631] [INFO ] [MAIN ] BLW: Full history saved. [2025-11-04T17:41:46.454745] [INFO ] [MAIN ] Update Progress: 306/2367 symbols completed. BLW took 38.1s. Estimated remaining: 4h 40m 60s. Rate: 7.3 symbols/min [2025-11-04T17:41:46.572148] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-01. [2025-11-04T17:41:46.624275] [INFO ] [MAIN ] BMI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:41:46.647002] [DEBUG] [LOCAL ] BMI: Loaded 24317 existing records from primary storage. [2025-11-04T17:41:46.647258] [INFO ] [MAIN ] BMI: Loaded 24317 existing records. [2025-11-04T17:41:46.823070] [WARN ] [MAIN ] BMI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:41:46.823269] [INFO ] [MAIN ] BMI: Wiping existing primary data before full refresh. [2025-11-04T17:41:46.823318] [INFO ] [LOCAL ] BMI: Starting deletion of primary data. [2025-11-04T17:41:46.823554] [DEBUG] [LOCAL ] BMI: Deleted primary data file. [2025-11-04T17:41:46.823609] [INFO ] [MAIN ] BMI: Starting full primary refresh. [2025-11-04T17:41:47.180583] [INFO ] [REMOTE ] BMI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:41:47.217378] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T17:41:47.323379] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T17:41:47.366994] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-01. [2025-11-04T17:41:48.167017] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-11. [2025-11-04T17:41:48.241228] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T17:41:48.967048] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-01. [2025-11-04T17:41:49.198945] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T17:41:49.200720] [INFO ] [REMOTE ] BMA: Full fetch complete (53007 records). [2025-11-04T17:41:49.200804] [DEBUG] [VALIDATOR ] BMA: Validating primary data. [2025-11-04T17:41:49.580262] [DEBUG] [VALIDATOR ] BMA: All expected trading days are present. [2025-11-04T17:41:49.580343] [DEBUG] [VALIDATOR ] BMA: Primary data validation passed. [2025-11-04T17:41:49.580942] [INFO ] [MAIN ] BMA: Fetched primary data is valid. [2025-11-04T17:41:49.581014] [DEBUG] [LOCAL ] BMA: Saving primary data. [2025-11-04T17:41:49.582223] [DEBUG] [LOCAL ] BMA: Trimmed 53007 records to 36961 (keeping 200 days of data) [2025-11-04T17:41:49.781882] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-02. [2025-11-04T17:41:49.814635] [DEBUG] [LOCAL ] BMA: Primary data saved (36961 records). [2025-11-04T17:41:49.814778] [INFO ] [MAIN ] BMA: Full history saved. [2025-11-04T17:41:49.815481] [INFO ] [MAIN ] Update Progress: 307/2367 symbols completed. BMA took 31.2s. Estimated remaining: 4h 40m 19s. Rate: 7.3 symbols/min [2025-11-04T17:41:49.991550] [INFO ] [MAIN ] BMN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:41:49.991716] [DEBUG] [LOCAL ] BMN: No existing data file found in primary storage. [2025-11-04T17:41:49.991765] [WARN ] [MAIN ] BMN: No existing data found, performing full refresh instead. [2025-11-04T17:41:49.991878] [INFO ] [MAIN ] BMN: Wiping existing primary data before full refresh. [2025-11-04T17:41:49.991920] [INFO ] [LOCAL ] BMN: Starting deletion of primary data. [2025-11-04T17:41:49.991962] [DEBUG] [LOCAL ] BMN: No primary data file found to delete. [2025-11-04T17:41:49.992004] [INFO ] [MAIN ] BMN: Starting full primary refresh. [2025-11-04T17:41:50.388115] [INFO ] [REMOTE ] BMN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:41:50.394753] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-11-04T17:41:50.396797] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T17:41:50.567022] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-02. [2025-11-04T17:41:51.312561] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T17:41:51.367046] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-03. [2025-11-04T17:41:52.167011] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-03. [2025-11-04T17:41:52.246508] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-11-04T17:41:52.709037] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-11-04T17:41:52.967037] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-04. [2025-11-04T17:41:53.520922] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-11-04T17:41:53.767043] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-05. [2025-11-04T17:41:54.372799] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-11-04T17:41:54.567043] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-06. [2025-11-04T17:41:55.093000] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-11-04T17:41:55.367053] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-01. [2025-11-04T17:41:55.784566] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-11-04T17:41:56.167042] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-07. [2025-11-04T17:41:56.690497] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-11-04T17:41:56.967050] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-02. [2025-11-04T17:41:57.313462] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-11-04T17:41:57.767058] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-02. [2025-11-04T17:41:58.568007] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-08. [2025-11-04T17:41:58.571289] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-11-04T17:41:59.343008] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T17:41:59.367004] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-09. [2025-11-04T17:41:59.926143] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T17:42:00.167097] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-10. [2025-11-04T17:42:00.918595] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-11-04T17:42:00.967104] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-03. [2025-11-04T17:42:01.767163] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-03. [2025-11-04T17:42:02.095063] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-11-04T17:42:02.490849] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T17:42:02.567051] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-04. [2025-11-04T17:42:02.938424] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-11-04T17:42:03.367053] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-04. [2025-11-04T17:42:04.167062] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-04. [2025-11-04T17:42:04.438331] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T17:42:04.897931] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T17:42:04.967034] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-05. [2025-11-04T17:42:05.691230] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-11-04T17:42:05.767031] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-11. [2025-11-04T17:42:06.337556] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T17:42:06.338497] [INFO ] [REMOTE ] BME: Full fetch complete (15404 records). [2025-11-04T17:42:06.338567] [DEBUG] [VALIDATOR ] BME: Validating primary data. [2025-11-04T17:42:06.572166] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-06. [2025-11-04T17:42:06.702193] [DEBUG] [VALIDATOR ] BME: All expected trading days are present. [2025-11-04T17:42:06.702268] [DEBUG] [VALIDATOR ] BME: Primary data validation passed. [2025-11-04T17:42:06.702583] [INFO ] [MAIN ] BME: Fetched primary data is valid. [2025-11-04T17:42:06.702623] [DEBUG] [LOCAL ] BME: Saving primary data. [2025-11-04T17:42:06.703151] [DEBUG] [LOCAL ] BME: Trimmed 15404 records to 10981 (keeping 200 days of data) [2025-11-04T17:42:06.768135] [DEBUG] [LOCAL ] BME: Primary data saved (10981 records). [2025-11-04T17:42:06.768262] [INFO ] [MAIN ] BME: Full history saved. [2025-11-04T17:42:06.768328] [INFO ] [MAIN ] Update Progress: 308/2367 symbols completed. BME took 25.5s. Estimated remaining: 4h 41m 10s. Rate: 7.3 symbols/min [2025-11-04T17:42:06.939201] [INFO ] [MAIN ] BMO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:42:06.976852] [DEBUG] [LOCAL ] BMO: Loaded 40320 existing records from primary storage. [2025-11-04T17:42:06.977132] [INFO ] [MAIN ] BMO: Loaded 40320 existing records. [2025-11-04T17:42:07.152390] [WARN ] [MAIN ] BMO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:42:07.152670] [INFO ] [MAIN ] BMO: Wiping existing primary data before full refresh. [2025-11-04T17:42:07.152739] [INFO ] [LOCAL ] BMO: Starting deletion of primary data. [2025-11-04T17:42:07.153058] [DEBUG] [LOCAL ] BMO: Deleted primary data file. [2025-11-04T17:42:07.153114] [INFO ] [MAIN ] BMO: Starting full primary refresh. [2025-11-04T17:42:07.371920] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-05. [2025-11-04T17:42:07.539864] [INFO ] [REMOTE ] BMO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:42:07.541391] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-11-04T17:42:07.892521] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-11-04T17:42:08.167070] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-06. [2025-11-04T17:42:08.477423] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-11-04T17:42:08.967085] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-05. [2025-11-04T17:42:09.767050] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-07. [2025-11-04T17:42:09.791211] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T17:42:10.095618] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-11-04T17:42:10.567052] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-07. [2025-11-04T17:42:11.367066] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-06. [2025-11-04T17:42:11.435015] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-11-04T17:42:12.066456] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T17:42:12.167039] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-07. [2025-11-04T17:42:12.967060] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-01. [2025-11-04T17:42:13.042864] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-11-04T17:42:13.767044] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-08. [2025-11-04T17:42:14.392862] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T17:42:14.567014] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-08. [2025-11-04T17:42:14.697612] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-11-04T17:42:15.367017] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-08. [2025-11-04T17:42:15.388050] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-11-04T17:42:15.698127] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-11-04T17:42:16.167086] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-09. [2025-11-04T17:42:16.967096] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-02. [2025-11-04T17:42:17.212824] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T17:42:17.767037] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-10. [2025-11-04T17:42:18.013879] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T17:42:18.567046] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-03. [2025-11-04T17:42:18.659365] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-11-04T17:42:19.367087] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-09. [2025-11-04T17:42:19.657255] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T17:42:19.749699] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-11-04T17:42:20.167082] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-10. [2025-11-04T17:42:20.529477] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-11-04T17:42:20.967061] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-09. [2025-11-04T17:42:21.767017] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-04. [2025-11-04T17:42:21.793863] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-11-04T17:42:22.567088] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-11. [2025-11-04T17:42:22.790067] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T17:42:22.974904] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T17:42:22.975725] [INFO ] [REMOTE ] BMN: Full fetch complete (3838 records). [2025-11-04T17:42:22.975806] [DEBUG] [VALIDATOR ] BMN: Validating primary data. [2025-11-04T17:42:23.335257] [DEBUG] [VALIDATOR ] BMN: All expected trading days are present. [2025-11-04T17:42:23.335332] [DEBUG] [VALIDATOR ] BMN: Primary data validation passed. [2025-11-04T17:42:23.335461] [INFO ] [MAIN ] BMN: Fetched primary data is valid. [2025-11-04T17:42:23.335499] [DEBUG] [LOCAL ] BMN: Saving primary data. [2025-11-04T17:42:23.335882] [DEBUG] [LOCAL ] BMN: Trimmed 3838 records to 2653 (keeping 200 days of data) [2025-11-04T17:42:23.352981] [DEBUG] [LOCAL ] BMN: Primary data saved (2653 records). [2025-11-04T17:42:23.353060] [INFO ] [MAIN ] BMN: Full history saved. [2025-11-04T17:42:23.353123] [INFO ] [MAIN ] Update Progress: 309/2367 symbols completed. BMN took 33.5s. Estimated remaining: 4h 41m 57s. Rate: 7.3 symbols/min [2025-11-04T17:42:23.372308] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-05. [2025-11-04T17:42:23.530917] [INFO ] [MAIN ] BMY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:42:23.585102] [DEBUG] [LOCAL ] BMY: Loaded 53757 existing records from primary storage. [2025-11-04T17:42:23.585443] [INFO ] [MAIN ] BMY: Loaded 53757 existing records. [2025-11-04T17:42:23.783027] [WARN ] [MAIN ] BMY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:42:23.783238] [INFO ] [MAIN ] BMY: Wiping existing primary data before full refresh. [2025-11-04T17:42:23.783287] [INFO ] [LOCAL ] BMY: Starting deletion of primary data. [2025-11-04T17:42:23.783474] [DEBUG] [LOCAL ] BMY: Deleted primary data file. [2025-11-04T17:42:23.783513] [INFO ] [MAIN ] BMY: Starting full primary refresh. [2025-11-04T17:42:24.139721] [INFO ] [REMOTE ] BMY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:42:24.167044] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-01. [2025-11-04T17:42:24.225410] [DEBUG] [REMOTE ] BMY: Error fetching primary data for 2025-01: 'API call returned an empty response.' [2025-11-04T17:42:24.225755] [WARN ] [REMOTE ] BMY: No data returned for month 2025-01 [2025-11-04T17:42:24.225843] [FATAL] [REMOTE ] BMY: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:42:24.225888] [FATAL] [MAIN ] BMY: Failed to fetch primary data. [2025-11-04T17:42:24.225952] [WARN ] [MAIN ] BMY: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:42:24.226022] [INFO ] [LOCAL ] BMY: Starting deletion of primary data. [2025-11-04T17:42:24.226085] [DEBUG] [LOCAL ] BMY: No primary data file found to delete. [2025-11-04T17:42:24.226138] [INFO ] [LOCAL ] BMY: Starting deletion of derivative data. [2025-11-04T17:42:24.226289] [DEBUG] [LOCAL ] BMY: Deleted derivative data file. [2025-11-04T17:42:24.226777] [INFO ] [MAIN ] Update Progress: 310/2367 symbols completed. BMY took 0.9s. Estimated remaining: 4h 41m 0s. Rate: 7.3 symbols/min [2025-11-04T17:42:24.402812] [INFO ] [MAIN ] BN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:42:24.458301] [DEBUG] [LOCAL ] BN: Loaded 52965 existing records from primary storage. [2025-11-04T17:42:24.458806] [INFO ] [MAIN ] BN: Loaded 52965 existing records. [2025-11-04T17:42:24.647302] [WARN ] [MAIN ] BN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:42:24.647553] [INFO ] [MAIN ] BN: Wiping existing primary data before full refresh. [2025-11-04T17:42:24.647623] [INFO ] [LOCAL ] BN: Starting deletion of primary data. [2025-11-04T17:42:24.647930] [DEBUG] [LOCAL ] BN: Deleted primary data file. [2025-11-04T17:42:24.648049] [INFO ] [MAIN ] BN: Starting full primary refresh. [2025-11-04T17:42:24.800447] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T17:42:24.972308] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-11. [2025-11-04T17:42:25.040892] [INFO ] [REMOTE ] BN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:42:25.767086] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-10. [2025-11-04T17:42:26.216345] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T17:42:26.217680] [INFO ] [REMOTE ] BMI: Full fetch complete (34338 records). [2025-11-04T17:42:26.217768] [DEBUG] [VALIDATOR ] BMI: Validating primary data. [2025-11-04T17:42:26.579049] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-01. [2025-11-04T17:42:26.593779] [DEBUG] [VALIDATOR ] BMI: All expected trading days are present. [2025-11-04T17:42:26.594994] [DEBUG] [VALIDATOR ] BMI: Primary data validation passed. [2025-11-04T17:42:26.595628] [INFO ] [MAIN ] BMI: Fetched primary data is valid. [2025-11-04T17:42:26.595919] [DEBUG] [LOCAL ] BMI: Saving primary data. [2025-11-04T17:42:26.596867] [DEBUG] [LOCAL ] BMI: Trimmed 34338 records to 24177 (keeping 200 days of data) [2025-11-04T17:42:26.778492] [DEBUG] [LOCAL ] BMI: Primary data saved (24177 records). [2025-11-04T17:42:26.780323] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-11-04T17:42:26.779415] [INFO ] [MAIN ] BMI: Full history saved. [2025-11-04T17:42:26.788978] [INFO ] [MAIN ] Update Progress: 311/2367 symbols completed. BMI took 40.3s. Estimated remaining: 4h 40m 15s. Rate: 7.3 symbols/min [2025-11-04T17:42:26.960319] [INFO ] [MAIN ] BNED: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:42:26.985409] [DEBUG] [LOCAL ] BNED: Loaded 27494 existing records from primary storage. [2025-11-04T17:42:26.985682] [INFO ] [MAIN ] BNED: Loaded 27494 existing records. [2025-11-04T17:42:27.160378] [WARN ] [MAIN ] BNED: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:42:27.160594] [INFO ] [MAIN ] BNED: Wiping existing primary data before full refresh. [2025-11-04T17:42:27.160661] [INFO ] [LOCAL ] BNED: Starting deletion of primary data. [2025-11-04T17:42:27.160858] [DEBUG] [LOCAL ] BNED: Deleted primary data file. [2025-11-04T17:42:27.160896] [INFO ] [MAIN ] BNED: Starting full primary refresh. [2025-11-04T17:42:27.374264] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-06. [2025-11-04T17:42:27.512016] [INFO ] [REMOTE ] BNED: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:42:27.957740] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T17:42:28.167059] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-02. [2025-11-04T17:42:28.418948] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T17:42:28.967112] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-11. [2025-11-04T17:42:29.452845] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T17:42:29.603393] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T17:42:29.604755] [INFO ] [REMOTE ] BMEZ: Full fetch complete (38362 records). [2025-11-04T17:42:29.604855] [DEBUG] [VALIDATOR ] BMEZ: Validating primary data. [2025-11-04T17:42:29.769938] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-01. [2025-11-04T17:42:29.997971] [DEBUG] [VALIDATOR ] BMEZ: All expected trading days are present. [2025-11-04T17:42:29.998063] [DEBUG] [VALIDATOR ] BMEZ: Primary data validation passed. [2025-11-04T17:42:29.998572] [INFO ] [MAIN ] BMEZ: Fetched primary data is valid. [2025-11-04T17:42:29.998680] [DEBUG] [LOCAL ] BMEZ: Saving primary data. [2025-11-04T17:42:29.999732] [DEBUG] [LOCAL ] BMEZ: Trimmed 38362 records to 24378 (keeping 200 days of data) [2025-11-04T17:42:30.143373] [DEBUG] [LOCAL ] BMEZ: Primary data saved (24378 records). [2025-11-04T17:42:30.143496] [INFO ] [MAIN ] BMEZ: Full history saved. [2025-11-04T17:42:30.143572] [INFO ] [MAIN ] Update Progress: 312/2367 symbols completed. BMEZ took 44.9s. Estimated remaining: 4h 39m 35s. Rate: 7.4 symbols/min [2025-11-04T17:42:30.315889] [INFO ] [MAIN ] BNH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:42:30.322715] [DEBUG] [LOCAL ] BNH: Loaded 4473 existing records from primary storage. [2025-11-04T17:42:30.322883] [INFO ] [MAIN ] BNH: Loaded 4473 existing records. [2025-11-04T17:42:30.500723] [WARN ] [MAIN ] BNH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:42:30.500926] [INFO ] [MAIN ] BNH: Wiping existing primary data before full refresh. [2025-11-04T17:42:30.500973] [INFO ] [LOCAL ] BNH: Starting deletion of primary data. [2025-11-04T17:42:30.501105] [DEBUG] [LOCAL ] BNH: Deleted primary data file. [2025-11-04T17:42:30.501142] [INFO ] [MAIN ] BNH: Starting full primary refresh. [2025-11-04T17:42:30.575602] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-07. [2025-11-04T17:42:30.872325] [INFO ] [REMOTE ] BNH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:42:30.890536] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T17:42:31.367040] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-02. [2025-11-04T17:42:31.554688] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-11-04T17:42:32.167027] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-08. [2025-11-04T17:42:32.235092] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T17:42:32.967054] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-03. [2025-11-04T17:42:33.253059] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-11-04T17:42:33.767013] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-03. [2025-11-04T17:42:33.878510] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T17:42:34.567030] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-01. [2025-11-04T17:42:35.317378] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T17:42:35.332408] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-11-04T17:42:35.367027] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-04. [2025-11-04T17:42:36.167041] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-04. [2025-11-04T17:42:36.842194] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-11-04T17:42:36.967042] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-05. [2025-11-04T17:42:37.088272] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T17:42:37.767037] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-05. [2025-11-04T17:42:38.394938] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T17:42:38.567007] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-02. [2025-11-04T17:42:38.740639] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-11-04T17:42:38.826302] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-02 (0.26 seconds). [2025-11-04T17:42:39.367042] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-03. [2025-11-04T17:42:39.761200] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-11-04T17:42:40.167039] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-06. [2025-11-04T17:42:40.967032] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-06. [2025-11-04T17:42:41.404870] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T17:42:41.767047] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-07. [2025-11-04T17:42:41.974169] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-11-04T17:42:42.567035] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-04. [2025-11-04T17:42:43.032603] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-11-04T17:42:43.065881] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:42:43.367023] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-09. [2025-11-04T17:42:44.167085] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-05. [2025-11-04T17:42:44.588093] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-11-04T17:42:44.620268] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T17:42:44.967070] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-07. [2025-11-04T17:42:45.767052] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-08. [2025-11-04T17:42:45.946886] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-11-04T17:42:46.567022] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-10. [2025-11-04T17:42:47.225723] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-11-04T17:42:47.367016] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-09. [2025-11-04T17:42:47.676297] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-11-04T17:42:48.167113] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-06. [2025-11-04T17:42:48.676115] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T17:42:48.829367] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-11-04T17:42:48.967027] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-10. [2025-11-04T17:42:49.767045] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-08. [2025-11-04T17:42:50.444300] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-11-04T17:42:50.567007] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-11. [2025-11-04T17:42:50.625948] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-11-04T17:42:51.367043] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-11. [2025-11-04T17:42:51.856328] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-11 (1.29 seconds). [2025-11-04T17:42:51.859014] [INFO ] [REMOTE ] BN: Full fetch complete (76963 records). [2025-11-04T17:42:51.859204] [DEBUG] [VALIDATOR ] BN: Validating primary data. [2025-11-04T17:42:52.176802] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-09. [2025-11-04T17:42:52.291848] [DEBUG] [VALIDATOR ] BN: All expected trading days are present. [2025-11-04T17:42:52.291979] [DEBUG] [VALIDATOR ] BN: Primary data validation passed. [2025-11-04T17:42:52.293108] [INFO ] [MAIN ] BN: Fetched primary data is valid. [2025-11-04T17:42:52.293869] [DEBUG] [LOCAL ] BN: Saving primary data. [2025-11-04T17:42:52.299398] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T17:42:52.301934] [INFO ] [REMOTE ] BMO: Full fetch complete (59768 records). [2025-11-04T17:42:52.302044] [DEBUG] [VALIDATOR ] BMO: Validating primary data. [2025-11-04T17:42:52.302286] [DEBUG] [LOCAL ] BN: Trimmed 76963 records to 52969 (keeping 200 days of data) [2025-11-04T17:42:52.927478] [DEBUG] [VALIDATOR ] BMO: All expected trading days are present. [2025-11-04T17:42:52.927828] [DEBUG] [VALIDATOR ] BMO: Primary data validation passed. [2025-11-04T17:42:52.933185] [INFO ] [MAIN ] BMO: Fetched primary data is valid. [2025-11-04T17:42:52.936720] [DEBUG] [LOCAL ] BMO: Saving primary data. [2025-11-04T17:42:52.960554] [DEBUG] [LOCAL ] BMO: Trimmed 59768 records to 40347 (keeping 200 days of data) [2025-11-04T17:42:52.997137] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-07. [2025-11-04T17:42:53.167263] [DEBUG] [LOCAL ] BN: Primary data saved (52969 records). [2025-11-04T17:42:53.169975] [INFO ] [MAIN ] BN: Full history saved. [2025-11-04T17:42:53.173294] [INFO ] [MAIN ] Update Progress: 313/2367 symbols completed. BN took 28.9s. Estimated remaining: 4h 41m 4s. Rate: 7.3 symbols/min [2025-11-04T17:42:53.508508] [INFO ] [MAIN ] BNJ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:42:53.511837] [DEBUG] [LOCAL ] BMO: Primary data saved (40347 records). [2025-11-04T17:42:53.512158] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-11-04T17:42:53.512542] [INFO ] [MAIN ] BMO: Full history saved. [2025-11-04T17:42:53.514588] [INFO ] [MAIN ] Update Progress: 314/2367 symbols completed. BMO took 46.7s. Estimated remaining: 4h 40m 5s. Rate: 7.3 symbols/min [2025-11-04T17:42:53.603973] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T17:42:53.602844] [DEBUG] [LOCAL ] BNJ: Loaded 2447 existing records from primary storage. [2025-11-04T17:42:53.635679] [INFO ] [MAIN ] BNJ: Loaded 2447 existing records. [2025-11-04T17:42:53.767416] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-10. [2025-11-04T17:42:53.815947] [INFO ] [MAIN ] BNL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:42:53.907232] [WARN ] [MAIN ] BNJ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:42:53.907791] [INFO ] [MAIN ] BNJ: Wiping existing primary data before full refresh. [2025-11-04T17:42:53.907842] [INFO ] [LOCAL ] BNJ: Starting deletion of primary data. [2025-11-04T17:42:53.907961] [DEBUG] [LOCAL ] BNJ: Deleted primary data file. [2025-11-04T17:42:53.907997] [INFO ] [MAIN ] BNJ: Starting full primary refresh. [2025-11-04T17:42:54.197272] [DEBUG] [LOCAL ] BNL: Loaded 46766 existing records from primary storage. [2025-11-04T17:42:54.224722] [INFO ] [MAIN ] BNL: Loaded 46766 existing records. [2025-11-04T17:42:54.436731] [INFO ] [REMOTE ] BNJ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:42:54.495934] [WARN ] [MAIN ] BNL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:42:54.496195] [INFO ] [MAIN ] BNL: Wiping existing primary data before full refresh. [2025-11-04T17:42:54.496249] [INFO ] [LOCAL ] BNL: Starting deletion of primary data. [2025-11-04T17:42:54.496566] [DEBUG] [LOCAL ] BNL: Deleted primary data file. [2025-11-04T17:42:54.496604] [INFO ] [MAIN ] BNL: Starting full primary refresh. [2025-11-04T17:42:54.570145] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-08. [2025-11-04T17:42:54.789522] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T17:42:54.874135] [INFO ] [REMOTE ] BNL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:42:55.057601] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-11-04T17:42:55.367056] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-09. [2025-11-04T17:42:55.794430] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-11-04T17:42:56.167050] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-01. [2025-11-04T17:42:56.728732] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T17:42:56.967043] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-01. [2025-11-04T17:42:57.767035] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-02. [2025-11-04T17:42:58.141789] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T17:42:58.140721] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-11-04T17:42:58.567042] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-11. [2025-11-04T17:42:59.152825] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-11 (0.59 seconds). [2025-11-04T17:42:59.153997] [INFO ] [REMOTE ] BNED: Full fetch complete (40889 records). [2025-11-04T17:42:59.154066] [DEBUG] [VALIDATOR ] BNED: Validating primary data. [2025-11-04T17:42:59.374594] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-10. [2025-11-04T17:42:59.541465] [DEBUG] [VALIDATOR ] BNED: All expected trading days are present. [2025-11-04T17:42:59.541538] [DEBUG] [VALIDATOR ] BNED: Primary data validation passed. [2025-11-04T17:42:59.542033] [INFO ] [MAIN ] BNED: Fetched primary data is valid. [2025-11-04T17:42:59.542093] [DEBUG] [LOCAL ] BNED: Saving primary data. [2025-11-04T17:42:59.542933] [DEBUG] [LOCAL ] BNED: Trimmed 40889 records to 27422 (keeping 200 days of data) [2025-11-04T17:42:59.691510] [DEBUG] [LOCAL ] BNED: Primary data saved (27422 records). [2025-11-04T17:42:59.691631] [INFO ] [MAIN ] BNED: Full history saved. [2025-11-04T17:42:59.691743] [INFO ] [MAIN ] Update Progress: 315/2367 symbols completed. BNED took 32.9s. Estimated remaining: 4h 39m 43s. Rate: 7.3 symbols/min [2025-11-04T17:42:59.864324] [INFO ] [MAIN ] BNS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:42:59.864463] [DEBUG] [LOCAL ] BNS: No existing data file found in primary storage. [2025-11-04T17:42:59.864500] [WARN ] [MAIN ] BNS: No existing data found, performing full refresh instead. [2025-11-04T17:42:59.864605] [INFO ] [MAIN ] BNS: Wiping existing primary data before full refresh. [2025-11-04T17:42:59.864675] [INFO ] [LOCAL ] BNS: Starting deletion of primary data. [2025-11-04T17:42:59.864749] [DEBUG] [LOCAL ] BNS: No primary data file found to delete. [2025-11-04T17:42:59.864785] [INFO ] [MAIN ] BNS: Starting full primary refresh. [2025-11-04T17:42:59.979638] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-11-04T17:43:00.173428] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-03. [2025-11-04T17:43:00.255168] [INFO ] [REMOTE ] BNS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:43:00.713047] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-11-04T17:43:00.967067] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-11. [2025-11-04T17:43:01.353416] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-11 (0.39 seconds). [2025-11-04T17:43:01.354176] [INFO ] [REMOTE ] BNH: Full fetch complete (6807 records). [2025-11-04T17:43:01.354244] [DEBUG] [VALIDATOR ] BNH: Validating primary data. [2025-11-04T17:43:01.354591] [FATAL] [VALIDATOR ] BNH: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:43:01.354705] [FATAL] [MAIN ] BNH: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:43:01.354775] [WARN ] [MAIN ] BNH: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:43:01.354862] [INFO ] [LOCAL ] BNH: Starting deletion of primary data. [2025-11-04T17:43:01.354957] [DEBUG] [LOCAL ] BNH: No primary data file found to delete. [2025-11-04T17:43:01.355013] [INFO ] [LOCAL ] BNH: Starting deletion of derivative data. [2025-11-04T17:43:01.355151] [DEBUG] [LOCAL ] BNH: Deleted derivative data file. [2025-11-04T17:43:01.355236] [INFO ] [MAIN ] Update Progress: 316/2367 symbols completed. BNH took 31.2s. Estimated remaining: 4h 38m 53s. Rate: 7.4 symbols/min [2025-11-04T17:43:01.526116] [INFO ] [MAIN ] BNT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:43:01.526280] [DEBUG] [LOCAL ] BNT: No existing data file found in primary storage. [2025-11-04T17:43:01.526316] [WARN ] [MAIN ] BNT: No existing data found, performing full refresh instead. [2025-11-04T17:43:01.526452] [INFO ] [MAIN ] BNT: Wiping existing primary data before full refresh. [2025-11-04T17:43:01.526523] [INFO ] [LOCAL ] BNT: Starting deletion of primary data. [2025-11-04T17:43:01.526579] [DEBUG] [LOCAL ] BNT: No primary data file found to delete. [2025-11-04T17:43:01.526612] [INFO ] [MAIN ] BNT: Starting full primary refresh. [2025-11-04T17:43:01.772207] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-02. [2025-11-04T17:43:01.917987] [INFO ] [REMOTE ] BNT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:43:02.567099] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-01. [2025-11-04T17:43:03.125228] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T17:43:03.367024] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-04. [2025-11-04T17:43:03.723490] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T17:43:03.876586] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T17:43:04.167060] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-03. [2025-11-04T17:43:04.967086] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-02. [2025-11-04T17:43:05.301138] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-11-04T17:43:05.767101] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-05. [2025-11-04T17:43:05.963886] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T17:43:06.069411] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-11-04T17:43:06.567056] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-01. [2025-11-04T17:43:07.210111] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-11-04T17:43:07.367070] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-02. [2025-11-04T17:43:07.677987] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-11-04T17:43:08.167072] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-03. [2025-11-04T17:43:08.967113] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-04. [2025-11-04T17:43:09.558026] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T17:43:09.767085] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-04. [2025-11-04T17:43:10.028396] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-11-04T17:43:10.567074] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-05. [2025-11-04T17:43:11.071346] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T17:43:11.367078] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-03. [2025-11-04T17:43:11.775750] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-11-04T17:43:11.788368] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T17:43:12.167056] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-04. [2025-11-04T17:43:12.605521] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-11-04T17:43:12.967048] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-06. [2025-11-04T17:43:13.767017] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-05. [2025-11-04T17:43:14.295922] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T17:43:14.567093] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-06. [2025-11-04T17:43:14.985528] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-11-04T17:43:15.299956] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-11-04T17:43:15.367039] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-06. [2025-11-04T17:43:16.167058] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-07. [2025-11-04T17:43:16.676583] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T17:43:16.967085] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-07. [2025-11-04T17:43:17.450493] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T17:43:17.767058] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-05. [2025-11-04T17:43:18.241351] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-11-04T17:43:18.362720] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T17:43:18.567084] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-07. [2025-11-04T17:43:18.892302] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-11-04T17:43:19.367084] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-08. [2025-11-04T17:43:20.167077] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-06. [2025-11-04T17:43:20.470047] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-06 (0.30 seconds). [2025-11-04T17:43:20.626921] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T17:43:20.967022] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-08. [2025-11-04T17:43:21.767040] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-07. [2025-11-04T17:43:22.161336] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T17:43:22.182008] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-11-04T17:43:22.567041] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-09. [2025-11-04T17:43:23.367034] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-08. [2025-11-04T17:43:23.827817] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-11-04T17:43:23.831363] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T17:43:24.167081] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-08. [2025-11-04T17:43:24.401884] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-08 (0.23 seconds). [2025-11-04T17:43:24.967109] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-09. [2025-11-04T17:43:25.378110] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-11-04T17:43:25.767057] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-10. [2025-11-04T17:43:26.567090] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-09. [2025-11-04T17:43:27.203377] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T17:43:27.367047] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-10. [2025-11-04T17:43:27.835195] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-11-04T17:43:27.870497] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T17:43:28.167090] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-11. [2025-11-04T17:43:28.967037] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-11. [2025-11-04T17:43:29.258045] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T17:43:29.260236] [INFO ] [REMOTE ] BNS: Full fetch complete (74516 records). [2025-11-04T17:43:29.260357] [DEBUG] [VALIDATOR ] BNS: Validating primary data. [2025-11-04T17:43:29.325930] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T17:43:29.353247] [INFO ] [REMOTE ] BNJ: Full fetch complete (3858 records). [2025-11-04T17:43:29.367613] [DEBUG] [VALIDATOR ] BNJ: Validating primary data. [2025-11-04T17:43:29.773102] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-09. [2025-11-04T17:43:30.133587] [DEBUG] [VALIDATOR ] BNS: All expected trading days are present. [2025-11-04T17:43:30.139240] [DEBUG] [VALIDATOR ] BNS: Primary data validation passed. [2025-11-04T17:43:30.139351] [DEBUG] [VALIDATOR ] BNJ: All expected trading days are present. [2025-11-04T17:43:30.139421] [DEBUG] [VALIDATOR ] BNJ: Primary data validation passed. [2025-11-04T17:43:30.140536] [INFO ] [MAIN ] BNJ: Fetched primary data is valid. [2025-11-04T17:43:30.140396] [INFO ] [MAIN ] BNS: Fetched primary data is valid. [2025-11-04T17:43:30.140613] [DEBUG] [LOCAL ] BNJ: Saving primary data. [2025-11-04T17:43:30.141152] [DEBUG] [LOCAL ] BNJ: Trimmed 3858 records to 2461 (keeping 200 days of data) [2025-11-04T17:43:30.141206] [DEBUG] [LOCAL ] BNS: Saving primary data. [2025-11-04T17:43:30.156632] [DEBUG] [LOCAL ] BNJ: Primary data saved (2461 records). [2025-11-04T17:43:30.156875] [INFO ] [MAIN ] BNJ: Full history saved. [2025-11-04T17:43:30.157100] [INFO ] [MAIN ] Update Progress: 317/2367 symbols completed. BNJ took 37.0s. Estimated remaining: 4h 40m 58s. Rate: 7.3 symbols/min [2025-11-04T17:43:30.157361] [DEBUG] [LOCAL ] BNS: Trimmed 74516 records to 51208 (keeping 200 days of data) [2025-11-04T17:43:30.495173] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-11-04T17:43:30.515200] [DEBUG] [LOCAL ] BNS: Primary data saved (51208 records). [2025-11-04T17:43:30.515341] [INFO ] [MAIN ] BNS: Full history saved. [2025-11-04T17:43:30.515415] [INFO ] [MAIN ] Update Progress: 318/2367 symbols completed. BNS took 30.8s. Estimated remaining: 4h 39m 59s. Rate: 7.3 symbols/min [2025-11-04T17:43:30.567006] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-10. [2025-11-04T17:43:30.969853] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-11-04T17:43:31.367085] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-10. [2025-11-04T17:43:32.167042] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-11. [2025-11-04T17:43:32.613532] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-11 (0.45 seconds). [2025-11-04T17:43:32.614289] [INFO ] [REMOTE ] BNT: Full fetch complete (4934 records). [2025-11-04T17:43:32.614359] [DEBUG] [VALIDATOR ] BNT: Validating primary data. [2025-11-04T17:43:32.779843] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T17:43:32.972132] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-11. [2025-11-04T17:43:33.075844] [DEBUG] [VALIDATOR ] BNT: All expected trading days are present. [2025-11-04T17:43:33.075918] [DEBUG] [VALIDATOR ] BNT: Primary data validation passed. [2025-11-04T17:43:33.076075] [INFO ] [MAIN ] BNT: Fetched primary data is valid. [2025-11-04T17:43:33.076114] [DEBUG] [LOCAL ] BNT: Saving primary data. [2025-11-04T17:43:33.076489] [DEBUG] [LOCAL ] BNT: Trimmed 4934 records to 3324 (keeping 200 days of data) [2025-11-04T17:43:33.096276] [DEBUG] [LOCAL ] BNT: Primary data saved (3324 records). [2025-11-04T17:43:33.096353] [INFO ] [MAIN ] BNT: Full history saved. [2025-11-04T17:43:33.096414] [INFO ] [MAIN ] Update Progress: 319/2367 symbols completed. BNT took 31.7s. Estimated remaining: 4h 39m 15s. Rate: 7.3 symbols/min [2025-11-04T17:43:33.961597] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T17:43:33.964403] [INFO ] [REMOTE ] BNL: Full fetch complete (67598 records). [2025-11-04T17:43:33.964608] [DEBUG] [VALIDATOR ] BNL: Validating primary data. [2025-11-04T17:43:34.370712] [DEBUG] [VALIDATOR ] BNL: All expected trading days are present. [2025-11-04T17:43:34.370839] [DEBUG] [VALIDATOR ] BNL: Primary data validation passed. [2025-11-04T17:43:34.371779] [INFO ] [MAIN ] BNL: Fetched primary data is valid. [2025-11-04T17:43:34.371851] [DEBUG] [LOCAL ] BNL: Saving primary data. [2025-11-04T17:43:34.373773] [DEBUG] [LOCAL ] BNL: Trimmed 67598 records to 46799 (keeping 200 days of data) [2025-11-04T17:43:34.672130] [DEBUG] [LOCAL ] BNL: Primary data saved (46799 records). [2025-11-04T17:43:34.672344] [INFO ] [MAIN ] BNL: Full history saved. [2025-11-04T17:43:34.672454] [INFO ] [MAIN ] Update Progress: 320/2367 symbols completed. BNL took 41.2s. Estimated remaining: 4h 38m 25s. Rate: 7.4 symbols/min [2025-11-04T17:43:35.447696] [INFO ] [MAIN ] BOC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:43:35.466078] [INFO ] [MAIN ] BOH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:43:35.507735] [INFO ] [MAIN ] BNY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:43:35.488405] [INFO ] [MAIN ] BOE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:43:35.531671] [DEBUG] [LOCAL ] BNY: Loaded 7240 existing records from primary storage. [2025-11-04T17:43:35.532013] [INFO ] [MAIN ] BNY: Loaded 7240 existing records. [2025-11-04T17:43:35.532672] [DEBUG] [LOCAL ] BOC: Loaded 17827 existing records from primary storage. [2025-11-04T17:43:35.567570] [INFO ] [MAIN ] BOC: Loaded 17827 existing records. [2025-11-04T17:43:35.685682] [DEBUG] [LOCAL ] BOE: Loaded 17068 existing records from primary storage. [2025-11-04T17:43:35.741589] [INFO ] [MAIN ] BOE: Loaded 17068 existing records. [2025-11-04T17:43:35.871989] [DEBUG] [LOCAL ] BOH: Loaded 30352 existing records from primary storage. [2025-11-04T17:43:35.936283] [INFO ] [MAIN ] BOH: Loaded 30352 existing records. [2025-11-04T17:43:36.117274] [WARN ] [MAIN ] BNY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:43:36.216933] [INFO ] [MAIN ] BNY: Wiping existing primary data before full refresh. [2025-11-04T17:43:36.211408] [WARN ] [MAIN ] BOC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:43:36.280053] [INFO ] [MAIN ] BOC: Wiping existing primary data before full refresh. [2025-11-04T17:43:36.279762] [INFO ] [LOCAL ] BNY: Starting deletion of primary data. [2025-11-04T17:43:36.310916] [INFO ] [LOCAL ] BOC: Starting deletion of primary data. [2025-11-04T17:43:36.322509] [DEBUG] [LOCAL ] BNY: Deleted primary data file. [2025-11-04T17:43:36.347150] [WARN ] [MAIN ] BOE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:43:36.340710] [INFO ] [MAIN ] BNY: Starting full primary refresh. [2025-11-04T17:43:36.357818] [DEBUG] [LOCAL ] BOC: Deleted primary data file. [2025-11-04T17:43:36.386583] [INFO ] [MAIN ] BOC: Starting full primary refresh. [2025-11-04T17:43:36.418878] [INFO ] [MAIN ] BOE: Wiping existing primary data before full refresh. [2025-11-04T17:43:36.493765] [WARN ] [MAIN ] BOH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:43:36.489638] [INFO ] [LOCAL ] BOE: Starting deletion of primary data. [2025-11-04T17:43:36.511603] [INFO ] [MAIN ] BOH: Wiping existing primary data before full refresh. [2025-11-04T17:43:36.541122] [INFO ] [LOCAL ] BOH: Starting deletion of primary data. [2025-11-04T17:43:36.541033] [DEBUG] [LOCAL ] BOE: Deleted primary data file. [2025-11-04T17:43:36.643410] [DEBUG] [LOCAL ] BOH: Deleted primary data file. [2025-11-04T17:43:36.627875] [INFO ] [MAIN ] BOE: Starting full primary refresh. [2025-11-04T17:43:36.691086] [INFO ] [MAIN ] BOH: Starting full primary refresh. [2025-11-04T17:43:37.832070] [INFO ] [REMOTE ] BNY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:43:37.839488] [INFO ] [REMOTE ] BOC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:43:37.897759] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-01. [2025-11-04T17:43:37.899612] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-01. [2025-11-04T17:43:38.023710] [INFO ] [REMOTE ] BOE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:43:38.029395] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-01. [2025-11-04T17:43:38.106116] [INFO ] [REMOTE ] BOH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:43:38.106530] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-01. [2025-11-04T17:43:38.717632] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-11-04T17:43:38.722075] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-02. [2025-11-04T17:43:38.834172] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-11-04T17:43:38.839282] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-02. [2025-11-04T17:43:38.871248] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-11-04T17:43:38.873363] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-02. [2025-11-04T17:43:39.107983] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T17:43:39.196853] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-11-04T17:43:39.204961] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-11-04T17:43:39.367040] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-03. [2025-11-04T17:43:39.619620] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-11-04T17:43:39.890408] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-11-04T17:43:40.167124] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-03. [2025-11-04T17:43:40.830756] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-11-04T17:43:40.967056] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-03. [2025-11-04T17:43:41.661225] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-11-04T17:43:41.767043] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-04. [2025-11-04T17:43:42.567082] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-04. [2025-11-04T17:43:42.631788] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T17:43:42.986935] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-11-04T17:43:43.367054] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-05. [2025-11-04T17:43:43.926083] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-11-04T17:43:44.167081] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-02. [2025-11-04T17:43:44.967053] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-05. [2025-11-04T17:43:45.080673] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T17:43:45.405044] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-11-04T17:43:45.767039] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-03. [2025-11-04T17:43:46.567040] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-04. [2025-11-04T17:43:46.988325] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T17:43:47.322258] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-11-04T17:43:47.367059] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-06. [2025-11-04T17:43:47.788047] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-11-04T17:43:48.167043] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-05. [2025-11-04T17:43:48.789473] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-11-04T17:43:48.967034] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-06. [2025-11-04T17:43:49.768014] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-07. [2025-11-04T17:43:49.769485] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-11-04T17:43:50.460581] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-11-04T17:43:50.566998] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-08. [2025-11-04T17:43:51.066788] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-11-04T17:43:51.367058] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-07. [2025-11-04T17:43:52.065695] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T17:43:52.167033] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-08. [2025-11-04T17:43:52.803716] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-11-04T17:43:52.967036] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-09. [2025-11-04T17:43:53.412058] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-11-04T17:43:53.767084] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-06. [2025-11-04T17:43:54.567030] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-10. [2025-11-04T17:43:54.619627] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T17:43:54.982881] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-11-04T17:43:55.367035] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-07. [2025-11-04T17:43:56.082732] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-11-04T17:43:56.167037] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-11. [2025-11-04T17:43:56.620308] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-11 (0.45 seconds). [2025-11-04T17:43:56.621230] [INFO ] [REMOTE ] BNY: Full fetch complete (10567 records). [2025-11-04T17:43:56.621298] [DEBUG] [VALIDATOR ] BNY: Validating primary data. [2025-11-04T17:43:56.976525] [DEBUG] [VALIDATOR ] BNY: All expected trading days are present. [2025-11-04T17:43:56.976290] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-09. [2025-11-04T17:43:56.976629] [DEBUG] [VALIDATOR ] BNY: Primary data validation passed. [2025-11-04T17:43:56.976982] [INFO ] [MAIN ] BNY: Fetched primary data is valid. [2025-11-04T17:43:56.977194] [DEBUG] [LOCAL ] BNY: Saving primary data. [2025-11-04T17:43:56.978339] [DEBUG] [LOCAL ] BNY: Trimmed 10567 records to 7312 (keeping 200 days of data) [2025-11-04T17:43:57.031435] [DEBUG] [LOCAL ] BNY: Primary data saved (7312 records). [2025-11-04T17:43:57.031590] [INFO ] [MAIN ] BNY: Full history saved. [2025-11-04T17:43:57.031709] [INFO ] [MAIN ] Update Progress: 321/2367 symbols completed. BNY took 22.4s. Estimated remaining: 4h 39m 47s. Rate: 7.3 symbols/min [2025-11-04T17:43:57.197749] [INFO ] [MAIN ] BOND: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:43:57.228739] [DEBUG] [LOCAL ] BOND: Loaded 32325 existing records from primary storage. [2025-11-04T17:43:57.229025] [INFO ] [MAIN ] BOND: Loaded 32325 existing records. [2025-11-04T17:43:57.402060] [WARN ] [MAIN ] BOND: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:43:57.402274] [INFO ] [MAIN ] BOND: Wiping existing primary data before full refresh. [2025-11-04T17:43:57.402319] [INFO ] [LOCAL ] BOND: Starting deletion of primary data. [2025-11-04T17:43:57.402780] [DEBUG] [LOCAL ] BOND: Deleted primary data file. [2025-11-04T17:43:57.402836] [INFO ] [MAIN ] BOND: Starting full primary refresh. [2025-11-04T17:43:57.755707] [INFO ] [REMOTE ] BOND: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:43:57.767249] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-08. [2025-11-04T17:43:57.768400] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T17:43:58.555282] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T17:43:58.566993] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-09. [2025-11-04T17:43:59.207769] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-11-04T17:43:59.367059] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-04. [2025-11-04T17:44:00.167093] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-10. [2025-11-04T17:44:00.587347] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T17:44:00.967022] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-10. [2025-11-04T17:44:01.107143] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T17:44:01.728413] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-11-04T17:44:01.767074] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-05. [2025-11-04T17:44:02.567057] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-11. [2025-11-04T17:44:02.675269] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-11-04T17:44:03.298291] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T17:44:03.299438] [INFO ] [REMOTE ] BOC: Full fetch complete (25781 records). [2025-11-04T17:44:03.299513] [DEBUG] [VALIDATOR ] BOC: Validating primary data. [2025-11-04T17:44:03.372172] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-11. [2025-11-04T17:44:03.674474] [DEBUG] [VALIDATOR ] BOC: All expected trading days are present. [2025-11-04T17:44:03.674549] [DEBUG] [VALIDATOR ] BOC: Primary data validation passed. [2025-11-04T17:44:03.674997] [INFO ] [MAIN ] BOC: Fetched primary data is valid. [2025-11-04T17:44:03.675056] [DEBUG] [LOCAL ] BOC: Saving primary data. [2025-11-04T17:44:03.675714] [DEBUG] [LOCAL ] BOC: Trimmed 25781 records to 17799 (keeping 200 days of data) [2025-11-04T17:44:03.778874] [DEBUG] [LOCAL ] BOC: Primary data saved (17799 records). [2025-11-04T17:44:03.779012] [INFO ] [MAIN ] BOC: Full history saved. [2025-11-04T17:44:03.779121] [INFO ] [MAIN ] Update Progress: 322/2367 symbols completed. BOC took 29.1s. Estimated remaining: 4h 39m 30s. Rate: 7.3 symbols/min [2025-11-04T17:44:03.959833] [INFO ] [MAIN ] BOOT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:44:03.960020] [DEBUG] [LOCAL ] BOOT: No existing data file found in primary storage. [2025-11-04T17:44:03.960056] [WARN ] [MAIN ] BOOT: No existing data found, performing full refresh instead. [2025-11-04T17:44:03.960189] [INFO ] [MAIN ] BOOT: Wiping existing primary data before full refresh. [2025-11-04T17:44:03.960231] [INFO ] [LOCAL ] BOOT: Starting deletion of primary data. [2025-11-04T17:44:03.960275] [DEBUG] [LOCAL ] BOOT: No primary data file found to delete. [2025-11-04T17:44:03.960306] [INFO ] [MAIN ] BOOT: Starting full primary refresh. [2025-11-04T17:44:04.170562] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-06. [2025-11-04T17:44:04.207603] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T17:44:04.250098] [INFO ] [REMOTE ] BOE: Full fetch complete (26662 records). [2025-11-04T17:44:04.255726] [DEBUG] [VALIDATOR ] BOE: Validating primary data. [2025-11-04T17:44:04.434716] [INFO ] [REMOTE ] BOOT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:44:04.701626] [DEBUG] [VALIDATOR ] BOE: All expected trading days are present. [2025-11-04T17:44:04.701735] [DEBUG] [VALIDATOR ] BOE: Primary data validation passed. [2025-11-04T17:44:04.702141] [INFO ] [MAIN ] BOE: Fetched primary data is valid. [2025-11-04T17:44:04.702180] [DEBUG] [LOCAL ] BOE: Saving primary data. [2025-11-04T17:44:04.702970] [DEBUG] [LOCAL ] BOE: Trimmed 26662 records to 17100 (keeping 200 days of data) [2025-11-04T17:44:04.805310] [DEBUG] [LOCAL ] BOE: Primary data saved (17100 records). [2025-11-04T17:44:04.805439] [INFO ] [MAIN ] BOE: Full history saved. [2025-11-04T17:44:04.805514] [INFO ] [MAIN ] Update Progress: 323/2367 symbols completed. BOE took 30.1s. Estimated remaining: 4h 38m 36s. Rate: 7.3 symbols/min [2025-11-04T17:44:04.972457] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-01. [2025-11-04T17:44:04.976135] [INFO ] [MAIN ] BORR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:44:05.026477] [DEBUG] [LOCAL ] BORR: Loaded 51042 existing records from primary storage. [2025-11-04T17:44:05.026806] [INFO ] [MAIN ] BORR: Loaded 51042 existing records. [2025-11-04T17:44:05.217322] [WARN ] [MAIN ] BORR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:44:05.217675] [INFO ] [MAIN ] BORR: Wiping existing primary data before full refresh. [2025-11-04T17:44:05.217842] [INFO ] [LOCAL ] BORR: Starting deletion of primary data. [2025-11-04T17:44:05.223257] [DEBUG] [LOCAL ] BORR: Deleted primary data file. [2025-11-04T17:44:05.231606] [INFO ] [MAIN ] BORR: Starting full primary refresh. [2025-11-04T17:44:05.298018] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T17:44:05.596986] [INFO ] [REMOTE ] BORR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:44:05.767020] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-01. [2025-11-04T17:44:06.337000] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T17:44:06.567060] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-01. [2025-11-04T17:44:06.992163] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T17:44:07.367239] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-02. [2025-11-04T17:44:07.554509] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T17:44:08.167071] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-02. [2025-11-04T17:44:08.567980] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T17:44:08.967043] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-07. [2025-11-04T17:44:09.345904] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T17:44:09.767014] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-02. [2025-11-04T17:44:09.976690] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T17:44:10.567056] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-08. [2025-11-04T17:44:10.620427] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T17:44:11.367026] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-03. [2025-11-04T17:44:11.565565] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T17:44:12.167101] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-03. [2025-11-04T17:44:12.500800] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-11-04T17:44:12.967094] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-09. [2025-11-04T17:44:13.461169] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-11-04T17:44:13.767047] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-04. [2025-11-04T17:44:14.500476] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-11-04T17:44:14.567093] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-03. [2025-11-04T17:44:14.801217] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-11-04T17:44:15.367027] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-10. [2025-11-04T17:44:15.737429] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-11-04T17:44:16.167090] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-04. [2025-11-04T17:44:16.231460] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-11-04T17:44:16.967054] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-04. [2025-11-04T17:44:17.648269] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-11-04T17:44:17.767073] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-05. [2025-11-04T17:44:18.302517] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T17:44:18.567063] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-11. [2025-11-04T17:44:18.986530] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T17:44:19.367048] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-05. [2025-11-04T17:44:19.434278] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T17:44:19.436346] [INFO ] [REMOTE ] BOH: Full fetch complete (45570 records). [2025-11-04T17:44:19.436475] [DEBUG] [VALIDATOR ] BOH: Validating primary data. [2025-11-04T17:44:19.831571] [DEBUG] [VALIDATOR ] BOH: All expected trading days are present. [2025-11-04T17:44:19.831670] [DEBUG] [VALIDATOR ] BOH: Primary data validation passed. [2025-11-04T17:44:19.832199] [INFO ] [MAIN ] BOH: Fetched primary data is valid. [2025-11-04T17:44:19.832241] [DEBUG] [LOCAL ] BOH: Saving primary data. [2025-11-04T17:44:19.833141] [DEBUG] [LOCAL ] BOH: Trimmed 45570 records to 30281 (keeping 200 days of data) [2025-11-04T17:44:20.010154] [DEBUG] [LOCAL ] BOH: Primary data saved (30281 records). [2025-11-04T17:44:20.010294] [INFO ] [MAIN ] BOH: Full history saved. [2025-11-04T17:44:20.010375] [INFO ] [MAIN ] Update Progress: 324/2367 symbols completed. BOH took 45.3s. Estimated remaining: 4h 39m 12s. Rate: 7.3 symbols/min [2025-11-04T17:44:20.168298] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-05. [2025-11-04T17:44:20.190367] [INFO ] [MAIN ] BOW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:44:20.212243] [DEBUG] [LOCAL ] BOW: Loaded 22332 existing records from primary storage. [2025-11-04T17:44:20.212485] [INFO ] [MAIN ] BOW: Loaded 22332 existing records. [2025-11-04T17:44:20.410258] [WARN ] [MAIN ] BOW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:44:20.410563] [INFO ] [MAIN ] BOW: Wiping existing primary data before full refresh. [2025-11-04T17:44:20.410718] [INFO ] [LOCAL ] BOW: Starting deletion of primary data. [2025-11-04T17:44:20.417927] [DEBUG] [LOCAL ] BOW: Deleted primary data file. [2025-11-04T17:44:20.428316] [INFO ] [MAIN ] BOW: Starting full primary refresh. [2025-11-04T17:44:20.493785] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T17:44:20.792918] [INFO ] [REMOTE ] BOW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:44:20.967013] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-01. [2025-11-04T17:44:21.567772] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T17:44:21.767106] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-06. [2025-11-04T17:44:21.812184] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-11-04T17:44:22.567030] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-06. [2025-11-04T17:44:22.677469] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T17:44:23.367044] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-07. [2025-11-04T17:44:23.558703] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-11-04T17:44:24.167006] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-06. [2025-11-04T17:44:24.282187] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T17:44:24.967070] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-07. [2025-11-04T17:44:25.497210] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T17:44:25.767051] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-02. [2025-11-04T17:44:26.012173] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-11-04T17:44:26.206511] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-11-04T17:44:26.567070] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-08. [2025-11-04T17:44:27.367074] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-03. [2025-11-04T17:44:27.612383] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-11-04T17:44:28.167064] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-09. [2025-11-04T17:44:28.337944] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-11-04T17:44:28.967023] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-07. [2025-11-04T17:44:29.149369] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T17:44:29.767095] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-04. [2025-11-04T17:44:30.442912] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T17:44:30.567110] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-08. [2025-11-04T17:44:30.673576] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-11-04T17:44:31.367097] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-10. [2025-11-04T17:44:31.607691] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T17:44:32.167094] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-08. [2025-11-04T17:44:32.704848] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-11-04T17:44:32.967063] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-11. [2025-11-04T17:44:33.500667] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-11-04T17:44:33.737497] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T17:44:33.739022] [INFO ] [REMOTE ] BOND: Full fetch complete (46708 records). [2025-11-04T17:44:33.739132] [DEBUG] [VALIDATOR ] BOND: Validating primary data. [2025-11-04T17:44:33.772213] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-05. [2025-11-04T17:44:34.127107] [DEBUG] [VALIDATOR ] BOND: All expected trading days are present. [2025-11-04T17:44:34.127188] [DEBUG] [VALIDATOR ] BOND: Primary data validation passed. [2025-11-04T17:44:34.127737] [INFO ] [MAIN ] BOND: Fetched primary data is valid. [2025-11-04T17:44:34.127798] [DEBUG] [LOCAL ] BOND: Saving primary data. [2025-11-04T17:44:34.128878] [DEBUG] [LOCAL ] BOND: Trimmed 46708 records to 32390 (keeping 200 days of data) [2025-11-04T17:44:34.330021] [DEBUG] [LOCAL ] BOND: Primary data saved (32390 records). [2025-11-04T17:44:34.330165] [INFO ] [MAIN ] BOND: Full history saved. [2025-11-04T17:44:34.330245] [INFO ] [MAIN ] Update Progress: 325/2367 symbols completed. BOND took 37.3s. Estimated remaining: 4h 39m 42s. Rate: 7.3 symbols/min [2025-11-04T17:44:34.505698] [INFO ] [MAIN ] BOX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:44:34.552116] [DEBUG] [LOCAL ] BOX: Loaded 49989 existing records from primary storage. [2025-11-04T17:44:34.552441] [INFO ] [MAIN ] BOX: Loaded 49989 existing records. [2025-11-04T17:44:34.575686] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-09. [2025-11-04T17:44:34.741595] [WARN ] [MAIN ] BOX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:44:34.741901] [INFO ] [MAIN ] BOX: Wiping existing primary data before full refresh. [2025-11-04T17:44:34.741973] [INFO ] [LOCAL ] BOX: Starting deletion of primary data. [2025-11-04T17:44:34.742230] [DEBUG] [LOCAL ] BOX: Deleted primary data file. [2025-11-04T17:44:34.747408] [INFO ] [MAIN ] BOX: Starting full primary refresh. [2025-11-04T17:44:34.828603] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T17:44:35.119240] [INFO ] [REMOTE ] BOX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:44:35.367040] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-01. [2025-11-04T17:44:36.045047] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-11-04T17:44:36.167144] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-09. [2025-11-04T17:44:36.529110] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T17:44:36.967009] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-10. [2025-11-04T17:44:37.176379] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T17:44:37.767091] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-02. [2025-11-04T17:44:38.440489] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T17:44:38.567053] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-11. [2025-11-04T17:44:38.844481] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T17:44:39.367027] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-10. [2025-11-04T17:44:39.587028] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T17:44:39.588571] [INFO ] [REMOTE ] BORR: Full fetch complete (74140 records). [2025-11-04T17:44:39.588717] [DEBUG] [VALIDATOR ] BORR: Validating primary data. [2025-11-04T17:44:39.589590] [FATAL] [VALIDATOR ] BORR: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:44:39.589715] [FATAL] [MAIN ] BORR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:44:39.589772] [WARN ] [MAIN ] BORR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:44:39.589825] [INFO ] [LOCAL ] BORR: Starting deletion of primary data. [2025-11-04T17:44:39.589934] [DEBUG] [LOCAL ] BORR: No primary data file found to delete. [2025-11-04T17:44:39.589971] [INFO ] [LOCAL ] BORR: Starting deletion of derivative data. [2025-11-04T17:44:39.590157] [DEBUG] [LOCAL ] BORR: Deleted derivative data file. [2025-11-04T17:44:39.590666] [INFO ] [MAIN ] Update Progress: 326/2367 symbols completed. BORR took 34.8s. Estimated remaining: 4h 39m 16s. Rate: 7.3 symbols/min [2025-11-04T17:44:39.764562] [INFO ] [MAIN ] BP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:44:39.817310] [DEBUG] [LOCAL ] BP: Loaded 53705 existing records from primary storage. [2025-11-04T17:44:39.817663] [INFO ] [MAIN ] BP: Loaded 53705 existing records. [2025-11-04T17:44:40.007135] [WARN ] [MAIN ] BP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:44:40.007377] [INFO ] [MAIN ] BP: Wiping existing primary data before full refresh. [2025-11-04T17:44:40.007424] [INFO ] [LOCAL ] BP: Starting deletion of primary data. [2025-11-04T17:44:40.007630] [DEBUG] [LOCAL ] BP: Deleted primary data file. [2025-11-04T17:44:40.007721] [INFO ] [MAIN ] BP: Starting full primary refresh. [2025-11-04T17:44:40.172157] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-06. [2025-11-04T17:44:40.395240] [INFO ] [REMOTE ] BP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:44:40.425027] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-11-04T17:44:40.918777] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-11-04T17:44:40.967062] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-07. [2025-11-04T17:44:41.623862] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-11-04T17:44:41.767077] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-08. [2025-11-04T17:44:42.567075] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-11. [2025-11-04T17:44:42.616954] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-11-04T17:44:43.367042] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-01. [2025-11-04T17:44:43.752340] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T17:44:43.754038] [INFO ] [REMOTE ] BOOT: Full fetch complete (59578 records). [2025-11-04T17:44:43.754117] [DEBUG] [VALIDATOR ] BOOT: Validating primary data. [2025-11-04T17:44:44.149165] [DEBUG] [VALIDATOR ] BOOT: All expected trading days are present. [2025-11-04T17:44:44.149240] [DEBUG] [VALIDATOR ] BOOT: Primary data validation passed. [2025-11-04T17:44:44.149889] [INFO ] [MAIN ] BOOT: Fetched primary data is valid. [2025-11-04T17:44:44.149963] [DEBUG] [LOCAL ] BOOT: Saving primary data. [2025-11-04T17:44:44.151024] [DEBUG] [LOCAL ] BOOT: Trimmed 59578 records to 38812 (keeping 200 days of data) [2025-11-04T17:44:44.199828] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-09. [2025-11-04T17:44:44.375813] [DEBUG] [LOCAL ] BOOT: Primary data saved (38812 records). [2025-11-04T17:44:44.376144] [INFO ] [MAIN ] BOOT: Full history saved. [2025-11-04T17:44:44.376212] [INFO ] [MAIN ] Update Progress: 327/2367 symbols completed. BOOT took 40.6s. Estimated remaining: 4h 38m 46s. Rate: 7.3 symbols/min [2025-11-04T17:44:44.556452] [INFO ] [MAIN ] BR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:44:44.591392] [DEBUG] [LOCAL ] BR: Loaded 34608 existing records from primary storage. [2025-11-04T17:44:44.591675] [INFO ] [MAIN ] BR: Loaded 34608 existing records. [2025-11-04T17:44:44.774129] [WARN ] [MAIN ] BR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:44:44.774499] [INFO ] [MAIN ] BR: Wiping existing primary data before full refresh. [2025-11-04T17:44:44.774618] [INFO ] [LOCAL ] BR: Starting deletion of primary data. [2025-11-04T17:44:44.774993] [DEBUG] [LOCAL ] BR: Deleted primary data file. [2025-11-04T17:44:44.775136] [INFO ] [MAIN ] BR: Starting full primary refresh. [2025-11-04T17:44:44.968707] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-03. [2025-11-04T17:44:45.105031] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-11-04T17:44:45.201177] [INFO ] [REMOTE ] BR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:44:45.216078] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T17:44:45.767072] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-10. [2025-11-04T17:44:46.120995] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T17:44:46.572227] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-02. [2025-11-04T17:44:46.580810] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-11-04T17:44:47.367038] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-01. [2025-11-04T17:44:48.134905] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-11-04T17:44:48.167018] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-03. [2025-11-04T17:44:48.489187] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T17:44:48.967032] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-11. [2025-11-04T17:44:49.636280] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-11-04T17:44:49.701461] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T17:44:49.702739] [INFO ] [REMOTE ] BOW: Full fetch complete (32484 records). [2025-11-04T17:44:49.702818] [DEBUG] [VALIDATOR ] BOW: Validating primary data. [2025-11-04T17:44:49.772145] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-04. [2025-11-04T17:44:50.076584] [DEBUG] [VALIDATOR ] BOW: All expected trading days are present. [2025-11-04T17:44:50.076693] [DEBUG] [VALIDATOR ] BOW: Primary data validation passed. [2025-11-04T17:44:50.077168] [INFO ] [MAIN ] BOW: Fetched primary data is valid. [2025-11-04T17:44:50.077229] [DEBUG] [LOCAL ] BOW: Saving primary data. [2025-11-04T17:44:50.078109] [DEBUG] [LOCAL ] BOW: Trimmed 32484 records to 22373 (keeping 200 days of data) [2025-11-04T17:44:50.202906] [DEBUG] [LOCAL ] BOW: Primary data saved (22373 records). [2025-11-04T17:44:50.203042] [INFO ] [MAIN ] BOW: Full history saved. [2025-11-04T17:44:50.203120] [INFO ] [MAIN ] Update Progress: 328/2367 symbols completed. BOW took 30.2s. Estimated remaining: 4h 38m 23s. Rate: 7.3 symbols/min [2025-11-04T17:44:50.401756] [INFO ] [MAIN ] BRBR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:44:50.448494] [DEBUG] [LOCAL ] BRBR: Loaded 52050 existing records from primary storage. [2025-11-04T17:44:50.448832] [INFO ] [MAIN ] BRBR: Loaded 52050 existing records. [2025-11-04T17:44:50.572367] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-02. [2025-11-04T17:44:50.636629] [WARN ] [MAIN ] BRBR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:44:50.636892] [INFO ] [MAIN ] BRBR: Wiping existing primary data before full refresh. [2025-11-04T17:44:50.636950] [INFO ] [LOCAL ] BRBR: Starting deletion of primary data. [2025-11-04T17:44:50.637135] [DEBUG] [LOCAL ] BRBR: Deleted primary data file. [2025-11-04T17:44:50.637172] [INFO ] [MAIN ] BRBR: Starting full primary refresh. [2025-11-04T17:44:51.005005] [INFO ] [REMOTE ] BRBR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:44:51.367056] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-01. [2025-11-04T17:44:51.402207] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-11-04T17:44:51.588389] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T17:44:52.167043] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-03. [2025-11-04T17:44:52.760800] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-11-04T17:44:52.967021] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-02. [2025-11-04T17:44:53.358083] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T17:44:53.767083] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-04. [2025-11-04T17:44:53.882953] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-11-04T17:44:54.567069] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-04. [2025-11-04T17:44:54.882004] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-11-04T17:44:55.367082] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-03. [2025-11-04T17:44:55.798549] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-11-04T17:44:56.167090] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-05. [2025-11-04T17:44:56.602407] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T17:44:56.967101] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-04. [2025-11-04T17:44:57.303490] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T17:44:57.767080] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-05. [2025-11-04T17:44:58.148187] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T17:44:58.567043] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-06. [2025-11-04T17:44:58.912986] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T17:44:59.367077] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-06. [2025-11-04T17:44:59.656437] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T17:45:00.167053] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-05. [2025-11-04T17:45:00.460134] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T17:45:00.967063] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-07. [2025-11-04T17:45:01.705203] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T17:45:01.767076] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-05. [2025-11-04T17:45:02.226200] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T17:45:02.567019] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-08. [2025-11-04T17:45:03.367047] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-07. [2025-11-04T17:45:03.573262] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-11-04T17:45:03.769574] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T17:45:04.167081] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-06. [2025-11-04T17:45:04.300959] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-11-04T17:45:04.967063] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-06. [2025-11-04T17:45:05.761852] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-11-04T17:45:05.774504] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-08. [2025-11-04T17:45:06.206456] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T17:45:06.567091] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-07. [2025-11-04T17:45:06.880427] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T17:45:07.367056] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-09. [2025-11-04T17:45:08.172286] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-09. [2025-11-04T17:45:08.192125] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-11-04T17:45:08.436347] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-11-04T17:45:08.967056] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-08. [2025-11-04T17:45:09.428505] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T17:45:09.767051] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-10. [2025-11-04T17:45:10.508409] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-11-04T17:45:10.567088] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-07. [2025-11-04T17:45:10.982914] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T17:45:11.367043] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-10. [2025-11-04T17:45:12.011977] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:45:12.167050] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-11. [2025-11-04T17:45:12.655170] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T17:45:12.967022] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-11. [2025-11-04T17:45:13.378252] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-11 (1.21 seconds). [2025-11-04T17:45:13.380154] [INFO ] [REMOTE ] BOX: Full fetch complete (72396 records). [2025-11-04T17:45:13.380240] [DEBUG] [VALIDATOR ] BOX: Validating primary data. [2025-11-04T17:45:13.768447] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-08. [2025-11-04T17:45:13.791447] [DEBUG] [VALIDATOR ] BOX: All expected trading days are present. [2025-11-04T17:45:13.791510] [DEBUG] [VALIDATOR ] BOX: Primary data validation passed. [2025-11-04T17:45:13.792252] [INFO ] [MAIN ] BOX: Fetched primary data is valid. [2025-11-04T17:45:13.792404] [DEBUG] [LOCAL ] BOX: Saving primary data. [2025-11-04T17:45:13.793823] [DEBUG] [LOCAL ] BOX: Trimmed 72396 records to 50006 (keeping 200 days of data) [2025-11-04T17:45:14.085707] [DEBUG] [LOCAL ] BOX: Primary data saved (50006 records). [2025-11-04T17:45:14.085886] [INFO ] [MAIN ] BOX: Full history saved. [2025-11-04T17:45:14.086846] [INFO ] [MAIN ] Update Progress: 329/2367 symbols completed. BOX took 39.8s. Estimated remaining: 4h 39m 52s. Rate: 7.3 symbols/min [2025-11-04T17:45:14.249927] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-11 (1.28 seconds). [2025-11-04T17:45:14.266802] [INFO ] [MAIN ] BRC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:45:14.268598] [INFO ] [REMOTE ] BR: Full fetch complete (50402 records). [2025-11-04T17:45:14.268702] [DEBUG] [VALIDATOR ] BR: Validating primary data. [2025-11-04T17:45:14.454684] [DEBUG] [LOCAL ] BRC: Loaded 17883 existing records from primary storage. [2025-11-04T17:45:14.513036] [INFO ] [MAIN ] BRC: Loaded 17883 existing records. [2025-11-04T17:45:14.569739] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-09. [2025-11-04T17:45:14.838374] [WARN ] [MAIN ] BRC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:45:14.840997] [INFO ] [MAIN ] BRC: Wiping existing primary data before full refresh. [2025-11-04T17:45:14.841073] [INFO ] [LOCAL ] BRC: Starting deletion of primary data. [2025-11-04T17:45:14.841207] [DEBUG] [VALIDATOR ] BR: All expected trading days are present. [2025-11-04T17:45:14.841343] [DEBUG] [VALIDATOR ] BR: Primary data validation passed. [2025-11-04T17:45:14.841939] [INFO ] [MAIN ] BR: Fetched primary data is valid. [2025-11-04T17:45:14.841984] [DEBUG] [LOCAL ] BRC: Deleted primary data file. [2025-11-04T17:45:14.842089] [INFO ] [MAIN ] BRC: Starting full primary refresh. [2025-11-04T17:45:14.842155] [DEBUG] [LOCAL ] BR: Saving primary data. [2025-11-04T17:45:14.947008] [DEBUG] [LOCAL ] BR: Trimmed 50402 records to 34647 (keeping 200 days of data) [2025-11-04T17:45:15.325364] [INFO ] [REMOTE ] BRC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:45:15.398676] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-01. [2025-11-04T17:45:15.440570] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-11-04T17:45:15.473723] [DEBUG] [LOCAL ] BR: Primary data saved (34647 records). [2025-11-04T17:45:15.473840] [INFO ] [MAIN ] BR: Full history saved. [2025-11-04T17:45:15.474916] [INFO ] [MAIN ] Update Progress: 330/2367 symbols completed. BR took 31.1s. Estimated remaining: 4h 39m 2s. Rate: 7.3 symbols/min [2025-11-04T17:45:15.643901] [INFO ] [MAIN ] BRCC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:45:15.644055] [DEBUG] [LOCAL ] BRCC: No existing data file found in primary storage. [2025-11-04T17:45:15.644091] [WARN ] [MAIN ] BRCC: No existing data found, performing full refresh instead. [2025-11-04T17:45:15.644199] [INFO ] [MAIN ] BRCC: Wiping existing primary data before full refresh. [2025-11-04T17:45:15.644239] [INFO ] [LOCAL ] BRCC: Starting deletion of primary data. [2025-11-04T17:45:15.644278] [DEBUG] [LOCAL ] BRCC: No primary data file found to delete. [2025-11-04T17:45:15.644307] [INFO ] [MAIN ] BRCC: Starting full primary refresh. [2025-11-04T17:45:15.993186] [INFO ] [REMOTE ] BRCC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:45:16.166999] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-01. [2025-11-04T17:45:16.321771] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-11-04T17:45:16.663558] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-11-04T17:45:16.967079] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-02. [2025-11-04T17:45:17.470154] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T17:45:17.624708] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-11-04T17:45:17.767094] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-03. [2025-11-04T17:45:18.567037] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-09. [2025-11-04T17:45:18.605614] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-11-04T17:45:19.367062] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-02. [2025-11-04T17:45:19.921277] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T17:45:20.167102] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-04. [2025-11-04T17:45:20.322527] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T17:45:20.967048] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-03. [2025-11-04T17:45:20.985697] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T17:45:21.767086] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-10. [2025-11-04T17:45:22.271691] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T17:45:22.567057] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-04. [2025-11-04T17:45:23.367016] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-10. [2025-11-04T17:45:23.530097] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-11-04T17:45:23.565226] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-11-04T17:45:24.167092] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-05. [2025-11-04T17:45:24.640498] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T17:45:24.835468] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T17:45:24.967015] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-06. [2025-11-04T17:45:25.646257] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-11-04T17:45:25.767021] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-05. [2025-11-04T17:45:26.567044] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-11. [2025-11-04T17:45:27.031273] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T17:45:27.367027] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-06. [2025-11-04T17:45:27.684774] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T17:45:27.692451] [INFO ] [REMOTE ] BRBR: Full fetch complete (73617 records). [2025-11-04T17:45:27.692753] [DEBUG] [VALIDATOR ] BRBR: Validating primary data. [2025-11-04T17:45:28.093403] [DEBUG] [VALIDATOR ] BRBR: All expected trading days are present. [2025-11-04T17:45:28.093497] [DEBUG] [VALIDATOR ] BRBR: Primary data validation passed. [2025-11-04T17:45:28.094332] [INFO ] [MAIN ] BRBR: Fetched primary data is valid. [2025-11-04T17:45:28.094396] [DEBUG] [LOCAL ] BRBR: Saving primary data. [2025-11-04T17:45:28.095919] [DEBUG] [LOCAL ] BRBR: Trimmed 73617 records to 52123 (keeping 200 days of data) [2025-11-04T17:45:28.176857] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-11. [2025-11-04T17:45:28.396953] [DEBUG] [LOCAL ] BRBR: Primary data saved (52123 records). [2025-11-04T17:45:28.397209] [INFO ] [MAIN ] BRBR: Full history saved. [2025-11-04T17:45:28.397301] [INFO ] [MAIN ] Update Progress: 331/2367 symbols completed. BRBR took 38.2s. Estimated remaining: 4h 39m 22s. Rate: 7.3 symbols/min [2025-11-04T17:45:28.566978] [INFO ] [MAIN ] BRCE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:45:28.567314] [DEBUG] [LOCAL ] BRCE: No existing data file found in primary storage. [2025-11-04T17:45:28.567373] [WARN ] [MAIN ] BRCE: No existing data found, performing full refresh instead. [2025-11-04T17:45:28.587798] [INFO ] [MAIN ] BRCE: Wiping existing primary data before full refresh. [2025-11-04T17:45:28.588141] [INFO ] [LOCAL ] BRCE: Starting deletion of primary data. [2025-11-04T17:45:28.588603] [DEBUG] [LOCAL ] BRCE: No primary data file found to delete. [2025-11-04T17:45:28.589302] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-11-04T17:45:28.589410] [INFO ] [MAIN ] BRCE: Starting full primary refresh. [2025-11-04T17:45:28.942054] [INFO ] [REMOTE ] BRCE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:45:28.967001] [INFO ] [REMOTE ] BRCE: Fetching primary data for 2025-01. [2025-11-04T17:45:29.123066] [DEBUG] [REMOTE ] BRCE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:45:29.123700] [WARN ] [REMOTE ] BRCE: No data returned for month 2025-01 [2025-11-04T17:45:29.123830] [FATAL] [REMOTE ] BRCE: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:45:29.123896] [FATAL] [MAIN ] BRCE: Failed to fetch primary data. [2025-11-04T17:45:29.123969] [WARN ] [MAIN ] BRCE: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:45:29.124064] [INFO ] [LOCAL ] BRCE: Starting deletion of primary data. [2025-11-04T17:45:29.124158] [DEBUG] [LOCAL ] BRCE: No primary data file found to delete. [2025-11-04T17:45:29.124272] [INFO ] [LOCAL ] BRCE: Starting deletion of derivative data. [2025-11-04T17:45:29.124335] [DEBUG] [LOCAL ] BRCE: No derivative data file found to delete. [2025-11-04T17:45:29.124407] [INFO ] [MAIN ] Update Progress: 332/2367 symbols completed. BRCE took 0.7s. Estimated remaining: 4h 38m 28s. Rate: 7.3 symbols/min [2025-11-04T17:45:29.340061] [INFO ] [MAIN ] BRIE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:45:29.340320] [DEBUG] [LOCAL ] BRIE: No existing data file found in primary storage. [2025-11-04T17:45:29.340377] [WARN ] [MAIN ] BRIE: No existing data found, performing full refresh instead. [2025-11-04T17:45:29.340496] [INFO ] [MAIN ] BRIE: Wiping existing primary data before full refresh. [2025-11-04T17:45:29.340538] [INFO ] [LOCAL ] BRIE: Starting deletion of primary data. [2025-11-04T17:45:29.340579] [DEBUG] [LOCAL ] BRIE: No primary data file found to delete. [2025-11-04T17:45:29.340611] [INFO ] [MAIN ] BRIE: Starting full primary refresh. [2025-11-04T17:45:29.731307] [INFO ] [REMOTE ] BRIE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:45:29.736298] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-11 (1.56 seconds). [2025-11-04T17:45:29.738859] [INFO ] [REMOTE ] BP: Full fetch complete (77880 records). [2025-11-04T17:45:29.738977] [DEBUG] [VALIDATOR ] BP: Validating primary data. [2025-11-04T17:45:29.772117] [INFO ] [REMOTE ] BRIE: Fetching primary data for 2025-01. [2025-11-04T17:45:30.009840] [DEBUG] [REMOTE ] BRIE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:45:30.034471] [WARN ] [REMOTE ] BRIE: No data returned for month 2025-01 [2025-11-04T17:45:30.037175] [FATAL] [REMOTE ] BRIE: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:45:30.037253] [FATAL] [MAIN ] BRIE: Failed to fetch primary data. [2025-11-04T17:45:30.043121] [WARN ] [MAIN ] BRIE: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:45:30.053502] [INFO ] [LOCAL ] BRIE: Starting deletion of primary data. [2025-11-04T17:45:30.058766] [DEBUG] [LOCAL ] BRIE: No primary data file found to delete. [2025-11-04T17:45:30.076074] [INFO ] [LOCAL ] BRIE: Starting deletion of derivative data. [2025-11-04T17:45:30.086066] [DEBUG] [LOCAL ] BRIE: No derivative data file found to delete. [2025-11-04T17:45:30.110118] [INFO ] [MAIN ] Update Progress: 333/2367 symbols completed. BRIE took 1.0s. Estimated remaining: 4h 37m 36s. Rate: 7.3 symbols/min [2025-11-04T17:45:30.199032] [DEBUG] [VALIDATOR ] BP: All expected trading days are present. [2025-11-04T17:45:30.227074] [DEBUG] [VALIDATOR ] BP: Primary data validation passed. [2025-11-04T17:45:30.238933] [INFO ] [MAIN ] BP: Fetched primary data is valid. [2025-11-04T17:45:30.239196] [DEBUG] [LOCAL ] BP: Saving primary data. [2025-11-04T17:45:30.294577] [DEBUG] [LOCAL ] BP: Trimmed 77880 records to 53705 (keeping 200 days of data) [2025-11-04T17:45:30.313318] [INFO ] [MAIN ] BRO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:45:30.571813] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-07. [2025-11-04T17:45:30.648270] [DEBUG] [LOCAL ] BRO: Loaded 52143 existing records from primary storage. [2025-11-04T17:45:30.657976] [INFO ] [MAIN ] BRO: Loaded 52143 existing records. [2025-11-04T17:45:30.750460] [DEBUG] [LOCAL ] BP: Primary data saved (53705 records). [2025-11-04T17:45:30.767816] [INFO ] [MAIN ] BP: Full history saved. [2025-11-04T17:45:30.777043] [INFO ] [MAIN ] Update Progress: 334/2367 symbols completed. BP took 51.2s. Estimated remaining: 4h 36m 42s. Rate: 7.3 symbols/min [2025-11-04T17:45:31.002363] [WARN ] [MAIN ] BRO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:45:31.032542] [INFO ] [MAIN ] BRO: Wiping existing primary data before full refresh. [2025-11-04T17:45:31.033949] [INFO ] [MAIN ] BROS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:45:31.034233] [DEBUG] [LOCAL ] BROS: No existing data file found in primary storage. [2025-11-04T17:45:31.034165] [INFO ] [LOCAL ] BRO: Starting deletion of primary data. [2025-11-04T17:45:31.034304] [WARN ] [MAIN ] BROS: No existing data found, performing full refresh instead. [2025-11-04T17:45:31.034507] [INFO ] [MAIN ] BROS: Wiping existing primary data before full refresh. [2025-11-04T17:45:31.034630] [INFO ] [LOCAL ] BROS: Starting deletion of primary data. [2025-11-04T17:45:31.034729] [DEBUG] [LOCAL ] BROS: No primary data file found to delete. [2025-11-04T17:45:31.034770] [INFO ] [MAIN ] BROS: Starting full primary refresh. [2025-11-04T17:45:31.034804] [DEBUG] [LOCAL ] BRO: Deleted primary data file. [2025-11-04T17:45:31.034930] [INFO ] [MAIN ] BRO: Starting full primary refresh. [2025-11-04T17:45:31.385736] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-07. [2025-11-04T17:45:31.766013] [INFO ] [REMOTE ] BROS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:45:31.806720] [INFO ] [REMOTE ] BRO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:45:31.831832] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T17:45:32.167031] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-08. [2025-11-04T17:45:32.264845] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-11-04T17:45:32.967059] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-08. [2025-11-04T17:45:33.167918] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T17:45:33.672840] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T17:45:33.767080] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-01. [2025-11-04T17:45:34.567074] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-01. [2025-11-04T17:45:35.043012] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T17:45:35.367048] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-09. [2025-11-04T17:45:35.846569] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T17:45:36.167103] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-09. [2025-11-04T17:45:36.513186] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T17:45:36.710843] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T17:45:36.967051] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-10. [2025-11-04T17:45:37.619492] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-11-04T17:45:37.767051] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-02. [2025-11-04T17:45:38.567044] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-11. [2025-11-04T17:45:39.015761] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-11-04T17:45:39.324060] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-11 (0.76 seconds). [2025-11-04T17:45:39.325232] [INFO ] [REMOTE ] BRC: Full fetch complete (28221 records). [2025-11-04T17:45:39.325505] [DEBUG] [VALIDATOR ] BRC: Validating primary data. [2025-11-04T17:45:39.372246] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-03. [2025-11-04T17:45:39.715234] [DEBUG] [VALIDATOR ] BRC: All expected trading days are present. [2025-11-04T17:45:39.715312] [DEBUG] [VALIDATOR ] BRC: Primary data validation passed. [2025-11-04T17:45:39.715752] [INFO ] [MAIN ] BRC: Fetched primary data is valid. [2025-11-04T17:45:39.715816] [DEBUG] [LOCAL ] BRC: Saving primary data. [2025-11-04T17:45:39.716959] [DEBUG] [LOCAL ] BRC: Trimmed 28221 records to 17834 (keeping 200 days of data) [2025-11-04T17:45:39.825754] [DEBUG] [LOCAL ] BRC: Primary data saved (17834 records). [2025-11-04T17:45:39.825899] [INFO ] [MAIN ] BRC: Full history saved. [2025-11-04T17:45:39.825975] [INFO ] [MAIN ] Update Progress: 335/2367 symbols completed. BRC took 25.7s. Estimated remaining: 4h 36m 39s. Rate: 7.3 symbols/min [2025-11-04T17:45:39.995924] [INFO ] [MAIN ] BRSL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:45:39.996105] [DEBUG] [LOCAL ] BRSL: No existing data file found in primary storage. [2025-11-04T17:45:39.996140] [WARN ] [MAIN ] BRSL: No existing data found, performing full refresh instead. [2025-11-04T17:45:39.996250] [INFO ] [MAIN ] BRSL: Wiping existing primary data before full refresh. [2025-11-04T17:45:39.996291] [INFO ] [LOCAL ] BRSL: Starting deletion of primary data. [2025-11-04T17:45:39.996333] [DEBUG] [LOCAL ] BRSL: No primary data file found to delete. [2025-11-04T17:45:39.996364] [INFO ] [MAIN ] BRSL: Starting full primary refresh. [2025-11-04T17:45:40.174072] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-02. [2025-11-04T17:45:40.361047] [INFO ] [REMOTE ] BRSL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:45:40.737391] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T17:45:40.967074] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-10. [2025-11-04T17:45:41.770952] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-01. [2025-11-04T17:45:41.789113] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-11-04T17:45:41.942471] [DEBUG] [REMOTE ] BRSL: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:45:41.942816] [WARN ] [REMOTE ] BRSL: No data returned for month 2025-01 [2025-11-04T17:45:41.942904] [FATAL] [REMOTE ] BRSL: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:45:41.942949] [FATAL] [MAIN ] BRSL: Failed to fetch primary data. [2025-11-04T17:45:41.942983] [WARN ] [MAIN ] BRSL: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:45:41.943045] [INFO ] [LOCAL ] BRSL: Starting deletion of primary data. [2025-11-04T17:45:41.943106] [DEBUG] [LOCAL ] BRSL: No primary data file found to delete. [2025-11-04T17:45:41.943140] [INFO ] [LOCAL ] BRSL: Starting deletion of derivative data. [2025-11-04T17:45:41.943179] [DEBUG] [LOCAL ] BRSL: No derivative data file found to delete. [2025-11-04T17:45:41.943227] [INFO ] [MAIN ] Update Progress: 336/2367 symbols completed. BRSL took 2.1s. Estimated remaining: 4h 35m 54s. Rate: 7.4 symbols/min [2025-11-04T17:45:42.134108] [INFO ] [MAIN ] BRSP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:45:42.165731] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T17:45:42.206226] [DEBUG] [LOCAL ] BRSP: Loaded 35774 existing records from primary storage. [2025-11-04T17:45:42.206496] [INFO ] [MAIN ] BRSP: Loaded 35774 existing records. [2025-11-04T17:45:42.379619] [WARN ] [MAIN ] BRSP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:45:42.379863] [INFO ] [MAIN ] BRSP: Wiping existing primary data before full refresh. [2025-11-04T17:45:42.379911] [INFO ] [LOCAL ] BRSP: Starting deletion of primary data. [2025-11-04T17:45:42.380087] [DEBUG] [LOCAL ] BRSP: Deleted primary data file. [2025-11-04T17:45:42.380126] [INFO ] [MAIN ] BRSP: Starting full primary refresh. [2025-11-04T17:45:42.572195] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-04. [2025-11-04T17:45:42.739507] [INFO ] [REMOTE ] BRSP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:45:43.367069] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-11. [2025-11-04T17:45:44.075712] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-11-04T17:45:44.167032] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-05. [2025-11-04T17:45:44.247180] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-11 (0.88 seconds). [2025-11-04T17:45:44.248910] [INFO ] [REMOTE ] BRCC: Full fetch complete (57403 records). [2025-11-04T17:45:44.249022] [DEBUG] [VALIDATOR ] BRCC: Validating primary data. [2025-11-04T17:45:44.708801] [DEBUG] [VALIDATOR ] BRCC: All expected trading days are present. [2025-11-04T17:45:44.708894] [DEBUG] [VALIDATOR ] BRCC: Primary data validation passed. [2025-11-04T17:45:44.709731] [INFO ] [MAIN ] BRCC: Fetched primary data is valid. [2025-11-04T17:45:44.709800] [DEBUG] [LOCAL ] BRCC: Saving primary data. [2025-11-04T17:45:44.711038] [DEBUG] [LOCAL ] BRCC: Trimmed 57403 records to 40286 (keeping 200 days of data) [2025-11-04T17:45:44.951022] [DEBUG] [LOCAL ] BRCC: Primary data saved (40286 records). [2025-11-04T17:45:44.951168] [INFO ] [MAIN ] BRCC: Full history saved. [2025-11-04T17:45:44.951242] [INFO ] [MAIN ] Update Progress: 337/2367 symbols completed. BRCC took 29.5s. Estimated remaining: 4h 35m 15s. Rate: 7.4 symbols/min [2025-11-04T17:45:44.972166] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-03. [2025-11-04T17:45:45.120687] [INFO ] [MAIN ] BRT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:45:45.127958] [DEBUG] [LOCAL ] BRT: Loaded 5241 existing records from primary storage. [2025-11-04T17:45:45.128129] [INFO ] [MAIN ] BRT: Loaded 5241 existing records. [2025-11-04T17:45:45.298627] [WARN ] [MAIN ] BRT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:45:45.298906] [INFO ] [MAIN ] BRT: Wiping existing primary data before full refresh. [2025-11-04T17:45:45.298954] [INFO ] [LOCAL ] BRT: Starting deletion of primary data. [2025-11-04T17:45:45.299131] [DEBUG] [LOCAL ] BRT: Deleted primary data file. [2025-11-04T17:45:45.299187] [INFO ] [MAIN ] BRT: Starting full primary refresh. [2025-11-04T17:45:45.648581] [INFO ] [REMOTE ] BRT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:45:45.690616] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-11-04T17:45:45.767022] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-01. [2025-11-04T17:45:46.567060] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-01. [2025-11-04T17:45:46.677070] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-11-04T17:45:46.911189] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T17:45:47.127399] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T17:45:47.367046] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-02. [2025-11-04T17:45:47.704972] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-11-04T17:45:48.167047] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-06. [2025-11-04T17:45:48.967134] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-03. [2025-11-04T17:45:49.360960] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-11-04T17:45:49.484420] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T17:45:49.767055] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-04. [2025-11-04T17:45:50.224534] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-11-04T17:45:50.567061] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-05. [2025-11-04T17:45:51.006368] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-11-04T17:45:51.367098] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-04. [2025-11-04T17:45:52.167088] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-07. [2025-11-04T17:45:52.919514] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-11-04T17:45:52.967017] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-05. [2025-11-04T17:45:53.490507] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T17:45:53.767094] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-06. [2025-11-04T17:45:54.162891] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-11-04T17:45:54.447804] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T17:45:54.567045] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-02. [2025-11-04T17:45:55.367056] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-08. [2025-11-04T17:45:55.440248] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T17:45:56.167057] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-06. [2025-11-04T17:45:56.790095] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T17:45:56.967045] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-03. [2025-11-04T17:45:57.767088] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-07. [2025-11-04T17:45:58.171344] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T17:45:58.269857] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-11-04T17:45:58.567069] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-04. [2025-11-04T17:45:59.367079] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-09. [2025-11-04T17:45:59.541618] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-11-04T17:46:00.167080] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-05. [2025-11-04T17:46:00.839609] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-11-04T17:46:00.967044] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-10. [2025-11-04T17:46:01.329837] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T17:46:01.767069] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-06. [2025-11-04T17:46:02.426452] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T17:46:02.567097] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-08. [2025-11-04T17:46:02.754855] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-11-04T17:46:02.992632] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-11-04T17:46:03.367024] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-07. [2025-11-04T17:46:04.167114] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-09. [2025-11-04T17:46:04.413545] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-11-04T17:46:04.595737] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-11-04T17:46:04.967024] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-10. [2025-11-04T17:46:05.441693] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-06 (9.27 seconds). [2025-11-04T17:46:05.492195] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-11-04T17:46:05.767083] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-11. [2025-11-04T17:46:06.266638] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T17:46:06.267790] [INFO ] [REMOTE ] BRT: Full fetch complete (7114 records). [2025-11-04T17:46:06.267864] [DEBUG] [VALIDATOR ] BRT: Validating primary data. [2025-11-04T17:46:06.572246] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-07. [2025-11-04T17:46:06.651563] [DEBUG] [VALIDATOR ] BRT: All expected trading days are present. [2025-11-04T17:46:06.651737] [DEBUG] [VALIDATOR ] BRT: Primary data validation passed. [2025-11-04T17:46:06.652248] [INFO ] [MAIN ] BRT: Fetched primary data is valid. [2025-11-04T17:46:06.652312] [DEBUG] [LOCAL ] BRT: Saving primary data. [2025-11-04T17:46:06.652932] [DEBUG] [LOCAL ] BRT: Trimmed 7114 records to 5241 (keeping 200 days of data) [2025-11-04T17:46:06.686083] [DEBUG] [LOCAL ] BRT: Primary data saved (5241 records). [2025-11-04T17:46:06.686222] [INFO ] [MAIN ] BRT: Full history saved. [2025-11-04T17:46:06.686317] [INFO ] [MAIN ] Update Progress: 338/2367 symbols completed. BRT took 21.7s. Estimated remaining: 4h 36m 29s. Rate: 7.3 symbols/min [2025-11-04T17:46:06.865085] [INFO ] [MAIN ] BRW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:46:06.885619] [DEBUG] [LOCAL ] BRW: Loaded 20936 existing records from primary storage. [2025-11-04T17:46:06.885937] [INFO ] [MAIN ] BRW: Loaded 20936 existing records. [2025-11-04T17:46:07.087067] [WARN ] [MAIN ] BRW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:46:07.087338] [INFO ] [MAIN ] BRW: Wiping existing primary data before full refresh. [2025-11-04T17:46:07.087388] [INFO ] [LOCAL ] BRW: Starting deletion of primary data. [2025-11-04T17:46:07.087768] [DEBUG] [LOCAL ] BRW: Deleted primary data file. [2025-11-04T17:46:07.087825] [INFO ] [MAIN ] BRW: Starting full primary refresh. [2025-11-04T17:46:07.374012] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-08. [2025-11-04T17:46:07.475972] [INFO ] [REMOTE ] BRW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:46:08.167037] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-11. [2025-11-04T17:46:08.299987] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-11-04T17:46:08.566512] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T17:46:08.967017] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-01. [2025-11-04T17:46:09.621238] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-11 (1.45 seconds). [2025-11-04T17:46:09.623468] [INFO ] [REMOTE ] BRO: Full fetch complete (75221 records). [2025-11-04T17:46:09.623614] [DEBUG] [VALIDATOR ] BRO: Validating primary data. [2025-11-04T17:46:09.768867] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-09. [2025-11-04T17:46:10.059407] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-11-04T17:46:10.089755] [DEBUG] [VALIDATOR ] BRO: All expected trading days are present. [2025-11-04T17:46:10.089864] [DEBUG] [VALIDATOR ] BRO: Primary data validation passed. [2025-11-04T17:46:10.090711] [INFO ] [MAIN ] BRO: Fetched primary data is valid. [2025-11-04T17:46:10.090782] [DEBUG] [LOCAL ] BRO: Saving primary data. [2025-11-04T17:46:10.092350] [DEBUG] [LOCAL ] BRO: Trimmed 75221 records to 52156 (keeping 200 days of data) [2025-11-04T17:46:10.413843] [DEBUG] [LOCAL ] BRO: Primary data saved (52156 records). [2025-11-04T17:46:10.413989] [INFO ] [MAIN ] BRO: Full history saved. [2025-11-04T17:46:10.414089] [INFO ] [MAIN ] Update Progress: 339/2367 symbols completed. BRO took 40.3s. Estimated remaining: 4h 35m 54s. Rate: 7.4 symbols/min [2025-11-04T17:46:10.582704] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-08. [2025-11-04T17:46:10.586394] [INFO ] [MAIN ] BRX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:46:10.586684] [DEBUG] [LOCAL ] BRX: No existing data file found in primary storage. [2025-11-04T17:46:10.586739] [WARN ] [MAIN ] BRX: No existing data found, performing full refresh instead. [2025-11-04T17:46:10.586876] [INFO ] [MAIN ] BRX: Wiping existing primary data before full refresh. [2025-11-04T17:46:10.586974] [INFO ] [LOCAL ] BRX: Starting deletion of primary data. [2025-11-04T17:46:10.587026] [DEBUG] [LOCAL ] BRX: No primary data file found to delete. [2025-11-04T17:46:10.587059] [INFO ] [MAIN ] BRX: Starting full primary refresh. [2025-11-04T17:46:10.945662] [INFO ] [REMOTE ] BRX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:46:10.966835] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T17:46:11.367090] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-10. [2025-11-04T17:46:12.167058] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-01. [2025-11-04T17:46:12.224109] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-11-04T17:46:12.413991] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-11-04T17:46:12.967053] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-11. [2025-11-04T17:46:13.507089] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T17:46:13.767063] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-02. [2025-11-04T17:46:14.019297] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T17:46:14.020927] [INFO ] [REMOTE ] BRSP: Full fetch complete (54026 records). [2025-11-04T17:46:14.021015] [DEBUG] [VALIDATOR ] BRSP: Validating primary data. [2025-11-04T17:46:14.404150] [DEBUG] [VALIDATOR ] BRSP: All expected trading days are present. [2025-11-04T17:46:14.404236] [DEBUG] [VALIDATOR ] BRSP: Primary data validation passed. [2025-11-04T17:46:14.404856] [INFO ] [MAIN ] BRSP: Fetched primary data is valid. [2025-11-04T17:46:14.405015] [DEBUG] [LOCAL ] BRSP: Saving primary data. [2025-11-04T17:46:14.406197] [DEBUG] [LOCAL ] BRSP: Trimmed 54026 records to 35732 (keeping 200 days of data) [2025-11-04T17:46:14.505090] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-11-04T17:46:14.573845] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-03. [2025-11-04T17:46:14.629106] [DEBUG] [LOCAL ] BRSP: Primary data saved (35732 records). [2025-11-04T17:46:14.629255] [INFO ] [MAIN ] BRSP: Full history saved. [2025-11-04T17:46:14.629994] [INFO ] [MAIN ] Update Progress: 340/2367 symbols completed. BRSP took 32.7s. Estimated remaining: 4h 35m 22s. Rate: 7.4 symbols/min [2025-11-04T17:46:14.798770] [INFO ] [MAIN ] BSAC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:46:14.830909] [DEBUG] [LOCAL ] BSAC: Loaded 31986 existing records from primary storage. [2025-11-04T17:46:14.831176] [INFO ] [MAIN ] BSAC: Loaded 31986 existing records. [2025-11-04T17:46:15.012612] [WARN ] [MAIN ] BSAC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:46:15.012876] [INFO ] [MAIN ] BSAC: Wiping existing primary data before full refresh. [2025-11-04T17:46:15.012927] [INFO ] [LOCAL ] BSAC: Starting deletion of primary data. [2025-11-04T17:46:15.013342] [DEBUG] [LOCAL ] BSAC: Deleted primary data file. [2025-11-04T17:46:15.013402] [INFO ] [MAIN ] BSAC: Starting full primary refresh. [2025-11-04T17:46:15.324519] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-11-04T17:46:15.372182] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-02. [2025-11-04T17:46:15.401477] [INFO ] [REMOTE ] BSAC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:46:16.167036] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-09. [2025-11-04T17:46:16.574566] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T17:46:16.967040] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-04. [2025-11-04T17:46:17.707597] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-11-04T17:46:17.735042] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-11-04T17:46:17.766998] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-10. [2025-11-04T17:46:18.567072] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-03. [2025-11-04T17:46:19.367008] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-01. [2025-11-04T17:46:19.744099] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-11-04T17:46:19.891486] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T17:46:20.060725] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T17:46:20.167077] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-05. [2025-11-04T17:46:20.967023] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-11. [2025-11-04T17:46:21.065555] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T17:46:21.767074] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-06. [2025-11-04T17:46:22.567045] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-02. [2025-11-04T17:46:22.622175] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-11 (1.66 seconds). [2025-11-04T17:46:22.624418] [INFO ] [REMOTE ] BROS: Full fetch complete (77284 records). [2025-11-04T17:46:22.624565] [DEBUG] [VALIDATOR ] BROS: Validating primary data. [2025-11-04T17:46:22.771030] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T17:46:23.072133] [DEBUG] [VALIDATOR ] BROS: All expected trading days are present. [2025-11-04T17:46:23.072230] [DEBUG] [VALIDATOR ] BROS: Primary data validation passed. [2025-11-04T17:46:23.073141] [INFO ] [MAIN ] BROS: Fetched primary data is valid. [2025-11-04T17:46:23.073222] [DEBUG] [LOCAL ] BROS: Saving primary data. [2025-11-04T17:46:23.075097] [DEBUG] [LOCAL ] BROS: Trimmed 77284 records to 53349 (keeping 200 days of data) [2025-11-04T17:46:23.367540] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-07. [2025-11-04T17:46:23.520294] [DEBUG] [LOCAL ] BROS: Primary data saved (53349 records). [2025-11-04T17:46:23.520484] [INFO ] [MAIN ] BROS: Full history saved. [2025-11-04T17:46:23.520590] [INFO ] [MAIN ] Update Progress: 341/2367 symbols completed. BROS took 52.7s. Estimated remaining: 4h 35m 18s. Rate: 7.4 symbols/min [2025-11-04T17:46:23.756096] [INFO ] [MAIN ] BSBR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:46:23.756386] [DEBUG] [LOCAL ] BSBR: No existing data file found in primary storage. [2025-11-04T17:46:23.756503] [WARN ] [MAIN ] BSBR: No existing data found, performing full refresh instead. [2025-11-04T17:46:23.756685] [INFO ] [MAIN ] BSBR: Wiping existing primary data before full refresh. [2025-11-04T17:46:23.756916] [INFO ] [LOCAL ] BSBR: Starting deletion of primary data. [2025-11-04T17:46:23.757087] [DEBUG] [LOCAL ] BSBR: No primary data file found to delete. [2025-11-04T17:46:23.757266] [INFO ] [MAIN ] BSBR: Starting full primary refresh. [2025-11-04T17:46:23.881115] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T17:46:24.172141] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-03. [2025-11-04T17:46:24.210919] [INFO ] [REMOTE ] BSBR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:46:24.284532] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T17:46:24.967088] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-04. [2025-11-04T17:46:25.661808] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-11-04T17:46:25.767058] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-04. [2025-11-04T17:46:26.567093] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-08. [2025-11-04T17:46:27.003021] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-11-04T17:46:27.142253] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T17:46:27.305768] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-11-04T17:46:27.367063] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-09. [2025-11-04T17:46:28.167050] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-05. [2025-11-04T17:46:28.224007] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-11-04T17:46:28.967064] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-10. [2025-11-04T17:46:29.131938] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T17:46:29.772170] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-01. [2025-11-04T17:46:29.781478] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-11-04T17:46:30.567077] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-05. [2025-11-04T17:46:30.810691] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-11-04T17:46:31.367062] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-02. [2025-11-04T17:46:31.730499] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T17:46:32.167044] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-06. [2025-11-04T17:46:32.296071] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-11-04T17:46:32.967063] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-06. [2025-11-04T17:46:33.210821] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T17:46:33.767035] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-07. [2025-11-04T17:46:34.106077] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-11-04T17:46:34.567017] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-11. [2025-11-04T17:46:34.665419] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T17:46:35.308667] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T17:46:35.309639] [INFO ] [REMOTE ] BRW: Full fetch complete (29344 records). [2025-11-04T17:46:35.309751] [DEBUG] [VALIDATOR ] BRW: Validating primary data. [2025-11-04T17:46:35.310251] [FATAL] [VALIDATOR ] BRW: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:46:35.310345] [FATAL] [MAIN ] BRW: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:46:35.310387] [WARN ] [MAIN ] BRW: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:46:35.310424] [INFO ] [LOCAL ] BRW: Starting deletion of primary data. [2025-11-04T17:46:35.310478] [DEBUG] [LOCAL ] BRW: No primary data file found to delete. [2025-11-04T17:46:35.310512] [INFO ] [LOCAL ] BRW: Starting deletion of derivative data. [2025-11-04T17:46:35.310696] [DEBUG] [LOCAL ] BRW: Deleted derivative data file. [2025-11-04T17:46:35.310798] [INFO ] [MAIN ] Update Progress: 342/2367 symbols completed. BRW took 28.6s. Estimated remaining: 4h 35m 32s. Rate: 7.3 symbols/min [2025-11-04T17:46:35.375229] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-03. [2025-11-04T17:46:35.485396] [INFO ] [MAIN ] BSL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:46:35.485539] [DEBUG] [LOCAL ] BSL: No existing data file found in primary storage. [2025-11-04T17:46:35.485576] [WARN ] [MAIN ] BSL: No existing data found, performing full refresh instead. [2025-11-04T17:46:35.485714] [INFO ] [MAIN ] BSL: Wiping existing primary data before full refresh. [2025-11-04T17:46:35.485764] [INFO ] [LOCAL ] BSL: Starting deletion of primary data. [2025-11-04T17:46:35.485826] [DEBUG] [LOCAL ] BSL: No primary data file found to delete. [2025-11-04T17:46:35.485868] [INFO ] [MAIN ] BSL: Starting full primary refresh. [2025-11-04T17:46:35.842209] [INFO ] [REMOTE ] BSL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:46:36.167055] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-07. [2025-11-04T17:46:36.384446] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T17:46:36.967050] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-08. [2025-11-04T17:46:37.639466] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-11-04T17:46:37.767039] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-08. [2025-11-04T17:46:37.913749] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T17:46:38.567062] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-09. [2025-11-04T17:46:39.019914] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T17:46:39.367029] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-04. [2025-11-04T17:46:39.581631] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T17:46:40.167019] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-09. [2025-11-04T17:46:40.357261] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-11-04T17:46:40.967049] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-01. [2025-11-04T17:46:41.767015] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-05. [2025-11-04T17:46:41.893786] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-11-04T17:46:41.914993] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T17:46:42.567076] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-02. [2025-11-04T17:46:42.760157] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T17:46:43.043678] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T17:46:43.367023] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-03. [2025-11-04T17:46:43.941960] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T17:46:44.167043] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-06. [2025-11-04T17:46:44.967047] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-10. [2025-11-04T17:46:45.145098] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T17:46:45.767044] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-04. [2025-11-04T17:46:46.246358] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-11-04T17:46:46.269892] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-11-04T17:46:46.567059] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-10. [2025-11-04T17:46:47.367042] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-11. [2025-11-04T17:46:47.636159] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T17:46:48.167118] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-11. [2025-11-04T17:46:48.549023] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T17:46:48.552183] [INFO ] [REMOTE ] BRX: Full fetch complete (75207 records). [2025-11-04T17:46:48.552543] [DEBUG] [VALIDATOR ] BRX: Validating primary data. [2025-11-04T17:46:48.953919] [DEBUG] [VALIDATOR ] BRX: All expected trading days are present. [2025-11-04T17:46:48.954041] [DEBUG] [VALIDATOR ] BRX: Primary data validation passed. [2025-11-04T17:46:48.954858] [INFO ] [MAIN ] BRX: Fetched primary data is valid. [2025-11-04T17:46:48.958723] [DEBUG] [LOCAL ] BRX: Saving primary data. [2025-11-04T17:46:48.959036] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-11 (0.79 seconds). [2025-11-04T17:46:48.961146] [INFO ] [REMOTE ] BSAC: Full fetch complete (45805 records). [2025-11-04T17:46:48.961386] [DEBUG] [VALIDATOR ] BSAC: Validating primary data. [2025-11-04T17:46:48.967407] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-07. [2025-11-04T17:46:48.967669] [DEBUG] [LOCAL ] BRX: Trimmed 75207 records to 51844 (keeping 200 days of data) [2025-11-04T17:46:49.592494] [DEBUG] [VALIDATOR ] BSAC: All expected trading days are present. [2025-11-04T17:46:49.595493] [DEBUG] [VALIDATOR ] BSAC: Primary data validation passed. [2025-11-04T17:46:49.596981] [INFO ] [MAIN ] BSAC: Fetched primary data is valid. [2025-11-04T17:46:49.599669] [DEBUG] [LOCAL ] BSAC: Saving primary data. [2025-11-04T17:46:49.603546] [DEBUG] [LOCAL ] BSAC: Trimmed 45805 records to 31883 (keeping 200 days of data) [2025-11-04T17:46:49.769780] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-05. [2025-11-04T17:46:49.887354] [DEBUG] [LOCAL ] BRX: Primary data saved (51844 records). [2025-11-04T17:46:49.893789] [DEBUG] [LOCAL ] BSAC: Primary data saved (31883 records). [2025-11-04T17:46:49.893897] [INFO ] [MAIN ] BRX: Full history saved. [2025-11-04T17:46:49.894092] [INFO ] [MAIN ] Update Progress: 343/2367 symbols completed. BRX took 39.5s. Estimated remaining: 4h 36m 1s. Rate: 7.3 symbols/min [2025-11-04T17:46:49.894506] [INFO ] [MAIN ] BSAC: Full history saved. [2025-11-04T17:46:49.927284] [INFO ] [MAIN ] Update Progress: 344/2367 symbols completed. BSAC took 35.3s. Estimated remaining: 4h 35m 5s. Rate: 7.4 symbols/min [2025-11-04T17:46:50.240977] [INFO ] [MAIN ] BSM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:46:50.264814] [INFO ] [MAIN ] BST: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:46:50.290621] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T17:46:50.314355] [DEBUG] [LOCAL ] BST: Loaded 17860 existing records from primary storage. [2025-11-04T17:46:50.314603] [INFO ] [MAIN ] BST: Loaded 17860 existing records. [2025-11-04T17:46:50.417525] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-11-04T17:46:50.465817] [DEBUG] [LOCAL ] BSM: Loaded 31597 existing records from primary storage. [2025-11-04T17:46:50.490049] [INFO ] [MAIN ] BSM: Loaded 31597 existing records. [2025-11-04T17:46:50.567829] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-08. [2025-11-04T17:46:50.604314] [WARN ] [MAIN ] BST: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:46:50.615841] [INFO ] [MAIN ] BST: Wiping existing primary data before full refresh. [2025-11-04T17:46:50.629485] [INFO ] [LOCAL ] BST: Starting deletion of primary data. [2025-11-04T17:46:50.665194] [DEBUG] [LOCAL ] BST: Deleted primary data file. [2025-11-04T17:46:50.665314] [INFO ] [MAIN ] BST: Starting full primary refresh. [2025-11-04T17:46:50.884524] [WARN ] [MAIN ] BSM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:46:50.913918] [INFO ] [MAIN ] BSM: Wiping existing primary data before full refresh. [2025-11-04T17:46:50.924532] [INFO ] [LOCAL ] BSM: Starting deletion of primary data. [2025-11-04T17:46:50.940329] [DEBUG] [LOCAL ] BSM: Deleted primary data file. [2025-11-04T17:46:50.974632] [INFO ] [MAIN ] BSM: Starting full primary refresh. [2025-11-04T17:46:51.277447] [INFO ] [REMOTE ] BST: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:46:51.372343] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-06. [2025-11-04T17:46:51.565607] [INFO ] [REMOTE ] BSM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:46:51.710422] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T17:46:52.028051] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-11-04T17:46:52.167033] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-09. [2025-11-04T17:46:52.967073] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-01. [2025-11-04T17:46:53.317472] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T17:46:53.759074] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-11-04T17:46:53.766986] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-02. [2025-11-04T17:46:54.514863] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T17:46:54.567052] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-07. [2025-11-04T17:46:54.985366] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-11-04T17:46:55.367208] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-03. [2025-11-04T17:46:56.167839] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-08. [2025-11-04T17:46:56.171252] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-11-04T17:46:56.589901] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-11-04T17:46:56.967057] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-04. [2025-11-04T17:46:57.767052] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-09. [2025-11-04T17:46:58.014912] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-11-04T17:46:58.287048] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-11-04T17:46:58.567065] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-10. [2025-11-04T17:46:59.367035] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-10. [2025-11-04T17:46:59.551946] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-11-04T17:46:59.902141] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-11-04T17:47:00.167070] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-11. [2025-11-04T17:47:00.967058] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-11. [2025-11-04T17:47:01.210254] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T17:47:01.211403] [INFO ] [REMOTE ] BSBR: Full fetch complete (50571 records). [2025-11-04T17:47:01.211474] [DEBUG] [VALIDATOR ] BSBR: Validating primary data. [2025-11-04T17:47:01.212084] [FATAL] [VALIDATOR ] BSBR: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:47:01.212166] [FATAL] [MAIN ] BSBR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:47:01.212207] [WARN ] [MAIN ] BSBR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:47:01.212388] [INFO ] [LOCAL ] BSBR: Starting deletion of primary data. [2025-11-04T17:47:01.212472] [DEBUG] [LOCAL ] BSBR: No primary data file found to delete. [2025-11-04T17:47:01.212522] [INFO ] [LOCAL ] BSBR: Starting deletion of derivative data. [2025-11-04T17:47:01.212578] [DEBUG] [LOCAL ] BSBR: No derivative data file found to delete. [2025-11-04T17:47:01.212680] [INFO ] [MAIN ] Update Progress: 345/2367 symbols completed. BSBR took 37.7s. Estimated remaining: 4h 35m 15s. Rate: 7.3 symbols/min [2025-11-04T17:47:01.384101] [INFO ] [MAIN ] BSTZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:47:01.407047] [DEBUG] [LOCAL ] BSTZ: Loaded 24246 existing records from primary storage. [2025-11-04T17:47:01.407318] [INFO ] [MAIN ] BSTZ: Loaded 24246 existing records. [2025-11-04T17:47:01.502129] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T17:47:01.513923] [INFO ] [REMOTE ] BSL: Full fetch complete (13016 records). [2025-11-04T17:47:01.524520] [DEBUG] [VALIDATOR ] BSL: Validating primary data. [2025-11-04T17:47:01.697532] [WARN ] [MAIN ] BSTZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:47:01.697851] [INFO ] [MAIN ] BSTZ: Wiping existing primary data before full refresh. [2025-11-04T17:47:01.703496] [INFO ] [LOCAL ] BSTZ: Starting deletion of primary data. [2025-11-04T17:47:01.713954] [DEBUG] [LOCAL ] BSTZ: Deleted primary data file. [2025-11-04T17:47:01.719114] [INFO ] [MAIN ] BSTZ: Starting full primary refresh. [2025-11-04T17:47:01.775164] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-01. [2025-11-04T17:47:02.240540] [DEBUG] [VALIDATOR ] BSL: All expected trading days are present. [2025-11-04T17:47:02.245747] [DEBUG] [VALIDATOR ] BSL: Primary data validation passed. [2025-11-04T17:47:02.256934] [INFO ] [MAIN ] BSL: Fetched primary data is valid. [2025-11-04T17:47:02.264990] [DEBUG] [LOCAL ] BSL: Saving primary data. [2025-11-04T17:47:02.278735] [DEBUG] [LOCAL ] BSL: Trimmed 13016 records to 7931 (keeping 200 days of data) [2025-11-04T17:47:02.381381] [INFO ] [REMOTE ] BSTZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:47:02.394842] [DEBUG] [LOCAL ] BSL: Primary data saved (7931 records). [2025-11-04T17:47:02.394928] [INFO ] [MAIN ] BSL: Full history saved. [2025-11-04T17:47:02.394990] [INFO ] [MAIN ] Update Progress: 346/2367 symbols completed. BSL took 27.1s. Estimated remaining: 4h 34m 26s. Rate: 7.4 symbols/min [2025-11-04T17:47:02.559708] [INFO ] [MAIN ] BSX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:47:02.567022] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-01. [2025-11-04T17:47:02.609635] [DEBUG] [LOCAL ] BSX: Loaded 53688 existing records from primary storage. [2025-11-04T17:47:02.609955] [INFO ] [MAIN ] BSX: Loaded 53688 existing records. [2025-11-04T17:47:02.783531] [WARN ] [MAIN ] BSX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:47:02.783776] [INFO ] [MAIN ] BSX: Wiping existing primary data before full refresh. [2025-11-04T17:47:02.783824] [INFO ] [LOCAL ] BSX: Starting deletion of primary data. [2025-11-04T17:47:02.784197] [DEBUG] [LOCAL ] BSX: Deleted primary data file. [2025-11-04T17:47:02.784253] [INFO ] [MAIN ] BSX: Starting full primary refresh. [2025-11-04T17:47:03.143915] [INFO ] [REMOTE ] BSX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:47:03.159552] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T17:47:03.367035] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-01. [2025-11-04T17:47:03.504727] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-11-04T17:47:04.167075] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-05. [2025-11-04T17:47:04.813694] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T17:47:04.967010] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-02. [2025-11-04T17:47:04.995885] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-11-04T17:47:05.767056] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-02. [2025-11-04T17:47:06.313909] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T17:47:06.515220] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T17:47:06.567088] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-03. [2025-11-04T17:47:07.367027] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-06. [2025-11-04T17:47:08.019730] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-11-04T17:47:08.072004] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-11-04T17:47:08.167080] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-04. [2025-11-04T17:47:08.967092] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-02. [2025-11-04T17:47:09.631442] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-11-04T17:47:09.767172] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-05. [2025-11-04T17:47:09.956371] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T17:47:10.567062] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-07. [2025-11-04T17:47:11.181458] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T17:47:11.289868] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-11-04T17:47:11.367034] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-08. [2025-11-04T17:47:12.145997] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-11-04T17:47:12.167035] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-09. [2025-11-04T17:47:12.945111] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-11-04T17:47:12.967033] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-10. [2025-11-04T17:47:13.767555] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T17:47:13.767668] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-03. [2025-11-04T17:47:14.567016] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-03. [2025-11-04T17:47:14.668231] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-11-04T17:47:15.367123] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-04. [2025-11-04T17:47:15.715889] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T17:47:16.167044] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-04. [2025-11-04T17:47:16.386671] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T17:47:16.967017] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-05. [2025-11-04T17:47:17.108197] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-11-04T17:47:17.768182] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-11. [2025-11-04T17:47:17.772473] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T17:47:18.567027] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-06. [2025-11-04T17:47:18.575177] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T17:47:18.576257] [INFO ] [REMOTE ] BST: Full fetch complete (28220 records). [2025-11-04T17:47:18.576325] [DEBUG] [VALIDATOR ] BST: Validating primary data. [2025-11-04T17:47:18.939177] [DEBUG] [VALIDATOR ] BST: All expected trading days are present. [2025-11-04T17:47:18.939254] [DEBUG] [VALIDATOR ] BST: Primary data validation passed. [2025-11-04T17:47:18.939617] [INFO ] [MAIN ] BST: Fetched primary data is valid. [2025-11-04T17:47:18.939686] [DEBUG] [LOCAL ] BST: Saving primary data. [2025-11-04T17:47:18.940321] [DEBUG] [LOCAL ] BST: Trimmed 28220 records to 17811 (keeping 200 days of data) [2025-11-04T17:47:19.043980] [DEBUG] [LOCAL ] BST: Primary data saved (17811 records). [2025-11-04T17:47:19.044106] [INFO ] [MAIN ] BST: Full history saved. [2025-11-04T17:47:19.044183] [INFO ] [MAIN ] Update Progress: 347/2367 symbols completed. BST took 29.1s. Estimated remaining: 4h 35m 8s. Rate: 7.3 symbols/min [2025-11-04T17:47:19.213456] [INFO ] [MAIN ] BTA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:47:19.213627] [DEBUG] [LOCAL ] BTA: No existing data file found in primary storage. [2025-11-04T17:47:19.213742] [WARN ] [MAIN ] BTA: No existing data found, performing full refresh instead. [2025-11-04T17:47:19.213858] [INFO ] [MAIN ] BTA: Wiping existing primary data before full refresh. [2025-11-04T17:47:19.213899] [INFO ] [LOCAL ] BTA: Starting deletion of primary data. [2025-11-04T17:47:19.213939] [DEBUG] [LOCAL ] BTA: No primary data file found to delete. [2025-11-04T17:47:19.213970] [INFO ] [MAIN ] BTA: Starting full primary refresh. [2025-11-04T17:47:19.384233] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-06. [2025-11-04T17:47:19.563464] [INFO ] [REMOTE ] BTA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:47:20.167375] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-05. [2025-11-04T17:47:20.327712] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-11-04T17:47:20.360282] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T17:47:20.967141] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-01. [2025-11-04T17:47:21.055227] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T17:47:21.651201] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-11-04T17:47:21.767023] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-02. [2025-11-04T17:47:22.250505] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T17:47:22.567038] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-03. [2025-11-04T17:47:22.965467] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T17:47:23.367076] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-04. [2025-11-04T17:47:23.884980] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-11-04T17:47:24.167035] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-06. [2025-11-04T17:47:24.967054] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-07. [2025-11-04T17:47:25.006891] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T17:47:25.767043] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-07. [2025-11-04T17:47:26.567062] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-05. [2025-11-04T17:47:26.634911] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-11-04T17:47:26.880688] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T17:47:27.001355] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T17:47:27.367034] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-08. [2025-11-04T17:47:28.167076] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-08. [2025-11-04T17:47:28.886055] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-11-04T17:47:28.967020] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-06. [2025-11-04T17:47:29.163127] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T17:47:29.413397] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-11-04T17:47:29.767040] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-09. [2025-11-04T17:47:30.567007] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-07. [2025-11-04T17:47:30.934924] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-11-04T17:47:31.372223] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-09. [2025-11-04T17:47:31.392743] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-11-04T17:47:32.167073] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-08. [2025-11-04T17:47:32.383891] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T17:47:32.626046] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-11-04T17:47:32.967049] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-07. [2025-11-04T17:47:33.767045] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-09. [2025-11-04T17:47:33.911108] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-11-04T17:47:34.173579] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-11-04T17:47:34.567042] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-10. [2025-11-04T17:47:35.367038] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-10. [2025-11-04T17:47:36.167014] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-10. [2025-11-04T17:47:36.260484] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T17:47:36.291053] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-11-04T17:47:36.547296] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-11-04T17:47:36.967074] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-11. [2025-11-04T17:47:37.767071] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-08. [2025-11-04T17:47:37.886385] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T17:47:37.887898] [INFO ] [REMOTE ] BSM: Full fetch complete (46565 records). [2025-11-04T17:47:37.887974] [DEBUG] [VALIDATOR ] BSM: Validating primary data. [2025-11-04T17:47:38.261900] [DEBUG] [VALIDATOR ] BSM: All expected trading days are present. [2025-11-04T17:47:38.261978] [DEBUG] [VALIDATOR ] BSM: Primary data validation passed. [2025-11-04T17:47:38.262500] [INFO ] [MAIN ] BSM: Fetched primary data is valid. [2025-11-04T17:47:38.262538] [DEBUG] [LOCAL ] BSM: Saving primary data. [2025-11-04T17:47:38.263450] [DEBUG] [LOCAL ] BSM: Trimmed 46565 records to 31666 (keeping 200 days of data) [2025-11-04T17:47:38.453102] [DEBUG] [LOCAL ] BSM: Primary data saved (31666 records). [2025-11-04T17:47:38.453406] [INFO ] [MAIN ] BSM: Full history saved. [2025-11-04T17:47:38.453510] [INFO ] [MAIN ] Update Progress: 348/2367 symbols completed. BSM took 48.6s. Estimated remaining: 4h 36m 5s. Rate: 7.3 symbols/min [2025-11-04T17:47:38.567129] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-11. [2025-11-04T17:47:38.635015] [INFO ] [MAIN ] BTE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:47:38.654843] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T17:47:38.706131] [DEBUG] [LOCAL ] BTE: Loaded 52656 existing records from primary storage. [2025-11-04T17:47:38.706439] [INFO ] [MAIN ] BTE: Loaded 52656 existing records. [2025-11-04T17:47:38.885376] [WARN ] [MAIN ] BTE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:47:38.885608] [INFO ] [MAIN ] BTE: Wiping existing primary data before full refresh. [2025-11-04T17:47:38.885696] [INFO ] [LOCAL ] BTE: Starting deletion of primary data. [2025-11-04T17:47:38.886116] [DEBUG] [LOCAL ] BTE: Deleted primary data file. [2025-11-04T17:47:38.886172] [INFO ] [MAIN ] BTE: Starting full primary refresh. [2025-11-04T17:47:38.999982] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T17:47:39.036714] [INFO ] [REMOTE ] BSX: Full fetch complete (77865 records). [2025-11-04T17:47:39.107208] [DEBUG] [VALIDATOR ] BSX: Validating primary data. [2025-11-04T17:47:39.367266] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-09. [2025-11-04T17:47:39.426409] [INFO ] [REMOTE ] BTE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:47:39.665890] [DEBUG] [VALIDATOR ] BSX: All expected trading days are present. [2025-11-04T17:47:39.665973] [DEBUG] [VALIDATOR ] BSX: Primary data validation passed. [2025-11-04T17:47:39.666899] [INFO ] [MAIN ] BSX: Fetched primary data is valid. [2025-11-04T17:47:39.666963] [DEBUG] [LOCAL ] BSX: Saving primary data. [2025-11-04T17:47:39.668421] [DEBUG] [LOCAL ] BSX: Trimmed 77865 records to 53688 (keeping 200 days of data) [2025-11-04T17:47:39.982814] [DEBUG] [LOCAL ] BSX: Primary data saved (53688 records). [2025-11-04T17:47:39.982950] [INFO ] [MAIN ] BSX: Full history saved. [2025-11-04T17:47:39.983769] [INFO ] [MAIN ] Update Progress: 349/2367 symbols completed. BSX took 37.6s. Estimated remaining: 4h 35m 18s. Rate: 7.3 symbols/min [2025-11-04T17:47:40.150902] [INFO ] [MAIN ] BTI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:47:40.167024] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-01. [2025-11-04T17:47:40.202770] [DEBUG] [LOCAL ] BTI: Loaded 53585 existing records from primary storage. [2025-11-04T17:47:40.203076] [INFO ] [MAIN ] BTI: Loaded 53585 existing records. [2025-11-04T17:47:40.386943] [WARN ] [MAIN ] BTI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:47:40.387280] [INFO ] [MAIN ] BTI: Wiping existing primary data before full refresh. [2025-11-04T17:47:40.387360] [INFO ] [LOCAL ] BTI: Starting deletion of primary data. [2025-11-04T17:47:40.402917] [DEBUG] [LOCAL ] BTI: Deleted primary data file. [2025-11-04T17:47:40.403613] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-11-04T17:47:40.409062] [INFO ] [MAIN ] BTI: Starting full primary refresh. [2025-11-04T17:47:40.759932] [INFO ] [REMOTE ] BTI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:47:40.967029] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-10. [2025-11-04T17:47:41.621679] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T17:47:41.683909] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-11-04T17:47:41.767021] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-02. [2025-11-04T17:47:42.567090] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-11. [2025-11-04T17:47:42.869781] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T17:47:43.193188] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T17:47:43.194872] [INFO ] [REMOTE ] BSTZ: Full fetch complete (36108 records). [2025-11-04T17:47:43.194950] [DEBUG] [VALIDATOR ] BSTZ: Validating primary data. [2025-11-04T17:47:43.373962] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-11. [2025-11-04T17:47:43.595965] [DEBUG] [VALIDATOR ] BSTZ: All expected trading days are present. [2025-11-04T17:47:43.596038] [DEBUG] [VALIDATOR ] BSTZ: Primary data validation passed. [2025-11-04T17:47:43.596485] [INFO ] [MAIN ] BSTZ: Fetched primary data is valid. [2025-11-04T17:47:43.596525] [DEBUG] [LOCAL ] BSTZ: Saving primary data. [2025-11-04T17:47:43.597335] [DEBUG] [LOCAL ] BSTZ: Trimmed 36108 records to 24270 (keeping 200 days of data) [2025-11-04T17:47:43.737859] [DEBUG] [LOCAL ] BSTZ: Primary data saved (24270 records). [2025-11-04T17:47:43.737984] [INFO ] [MAIN ] BSTZ: Full history saved. [2025-11-04T17:47:43.738058] [INFO ] [MAIN ] Update Progress: 350/2367 symbols completed. BSTZ took 42.5s. Estimated remaining: 4h 34m 44s. Rate: 7.3 symbols/min [2025-11-04T17:47:43.902438] [INFO ] [MAIN ] BTO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:47:43.912276] [DEBUG] [LOCAL ] BTO: Loaded 8590 existing records from primary storage. [2025-11-04T17:47:43.912439] [INFO ] [MAIN ] BTO: Loaded 8590 existing records. [2025-11-04T17:47:44.013918] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T17:47:44.020341] [INFO ] [REMOTE ] BTA: Full fetch complete (8789 records). [2025-11-04T17:47:44.035832] [DEBUG] [VALIDATOR ] BTA: Validating primary data. [2025-11-04T17:47:44.059312] [FATAL] [VALIDATOR ] BTA: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:47:44.070673] [FATAL] [MAIN ] BTA: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:47:44.090186] [WARN ] [MAIN ] BTA: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:47:44.093678] [WARN ] [MAIN ] BTO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:47:44.093789] [INFO ] [LOCAL ] BTA: Starting deletion of primary data. [2025-11-04T17:47:44.093930] [INFO ] [MAIN ] BTO: Wiping existing primary data before full refresh. [2025-11-04T17:47:44.094097] [DEBUG] [LOCAL ] BTA: No primary data file found to delete. [2025-11-04T17:47:44.094045] [INFO ] [LOCAL ] BTO: Starting deletion of primary data. [2025-11-04T17:47:44.094205] [INFO ] [LOCAL ] BTA: Starting deletion of derivative data. [2025-11-04T17:47:44.094285] [DEBUG] [LOCAL ] BTO: Deleted primary data file. [2025-11-04T17:47:44.094351] [INFO ] [MAIN ] BTO: Starting full primary refresh. [2025-11-04T17:47:44.094273] [DEBUG] [LOCAL ] BTA: No derivative data file found to delete. [2025-11-04T17:47:44.125314] [INFO ] [MAIN ] Update Progress: 351/2367 symbols completed. BTA took 25.1s. Estimated remaining: 4h 33m 51s. Rate: 7.4 symbols/min [2025-11-04T17:47:44.168532] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-03. [2025-11-04T17:47:44.562993] [INFO ] [MAIN ] BTT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:47:44.637080] [INFO ] [REMOTE ] BTO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:47:44.648356] [DEBUG] [LOCAL ] BTT: Loaded 14839 existing records from primary storage. [2025-11-04T17:47:44.648548] [INFO ] [MAIN ] BTT: Loaded 14839 existing records. [2025-11-04T17:47:44.829699] [WARN ] [MAIN ] BTT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:47:44.829905] [INFO ] [MAIN ] BTT: Wiping existing primary data before full refresh. [2025-11-04T17:47:44.829956] [INFO ] [LOCAL ] BTT: Starting deletion of primary data. [2025-11-04T17:47:44.830138] [DEBUG] [LOCAL ] BTT: Deleted primary data file. [2025-11-04T17:47:44.830195] [INFO ] [MAIN ] BTT: Starting full primary refresh. [2025-11-04T17:47:44.972109] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-01. [2025-11-04T17:47:45.182156] [INFO ] [REMOTE ] BTT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:47:45.540953] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T17:47:45.718930] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T17:47:45.767001] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-02. [2025-11-04T17:47:46.341713] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-11-04T17:47:46.567132] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-01. [2025-11-04T17:47:47.367069] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-03. [2025-11-04T17:47:47.413484] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-11-04T17:47:47.945787] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-11-04T17:47:48.167032] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-02. [2025-11-04T17:47:48.762354] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-11-04T17:47:48.967086] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-01. [2025-11-04T17:47:49.767059] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-04. [2025-11-04T17:47:50.567062] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-03. [2025-11-04T17:47:50.638190] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-11-04T17:47:50.916286] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T17:47:51.241719] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-11-04T17:47:51.367078] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-02. [2025-11-04T17:47:52.167086] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-04. [2025-11-04T17:47:52.770728] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-11-04T17:47:52.948896] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-11-04T17:47:52.967003] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-05. [2025-11-04T17:47:53.492198] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-11-04T17:47:53.767107] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-03. [2025-11-04T17:47:54.567074] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-04. [2025-11-04T17:47:55.172432] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-11-04T17:47:55.367123] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-06. [2025-11-04T17:47:55.939686] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-11-04T17:47:56.167062] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-05. [2025-11-04T17:47:56.967090] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-07. [2025-11-04T17:47:57.665820] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-11-04T17:47:57.689268] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-11-04T17:47:57.767040] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-05. [2025-11-04T17:47:58.568556] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-06. [2025-11-04T17:47:58.570340] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-11-04T17:47:59.367096] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-06. [2025-11-04T17:47:59.846513] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T17:48:00.167082] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-07. [2025-11-04T17:48:00.967063] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-08. [2025-11-04T17:48:01.762879] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-11-04T17:48:01.774808] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-08. [2025-11-04T17:48:02.216077] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T17:48:02.567178] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-09. [2025-11-04T17:48:03.088730] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-11-04T17:48:03.120613] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-11-04T17:48:03.367046] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-09. [2025-11-04T17:48:04.167079] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-10. [2025-11-04T17:48:04.719149] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T17:48:04.749009] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-11-04T17:48:04.967093] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-11. [2025-11-04T17:48:05.616352] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-11 (0.65 seconds). [2025-11-04T17:48:05.617465] [INFO ] [REMOTE ] BTT: Full fetch complete (22473 records). [2025-11-04T17:48:05.617536] [DEBUG] [VALIDATOR ] BTT: Validating primary data. [2025-11-04T17:48:05.771456] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-10. [2025-11-04T17:48:05.997565] [DEBUG] [VALIDATOR ] BTT: All expected trading days are present. [2025-11-04T17:48:05.997638] [DEBUG] [VALIDATOR ] BTT: Primary data validation passed. [2025-11-04T17:48:05.998022] [INFO ] [MAIN ] BTT: Fetched primary data is valid. [2025-11-04T17:48:05.998063] [DEBUG] [LOCAL ] BTT: Saving primary data. [2025-11-04T17:48:05.998667] [DEBUG] [LOCAL ] BTT: Trimmed 22473 records to 14833 (keeping 200 days of data) [2025-11-04T17:48:06.084536] [DEBUG] [LOCAL ] BTT: Primary data saved (14833 records). [2025-11-04T17:48:06.084687] [INFO ] [MAIN ] BTT: Full history saved. [2025-11-04T17:48:06.084779] [INFO ] [MAIN ] Update Progress: 352/2367 symbols completed. BTT took 21.9s. Estimated remaining: 4h 35m 2s. Rate: 7.3 symbols/min [2025-11-04T17:48:06.252502] [INFO ] [MAIN ] BTU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:48:06.303017] [DEBUG] [LOCAL ] BTU: Loaded 53319 existing records from primary storage. [2025-11-04T17:48:06.303335] [INFO ] [MAIN ] BTU: Loaded 53319 existing records. [2025-11-04T17:48:06.481616] [WARN ] [MAIN ] BTU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:48:06.481813] [INFO ] [MAIN ] BTU: Wiping existing primary data before full refresh. [2025-11-04T17:48:06.481878] [INFO ] [LOCAL ] BTU: Starting deletion of primary data. [2025-11-04T17:48:06.482075] [DEBUG] [LOCAL ] BTU: Deleted primary data file. [2025-11-04T17:48:06.482132] [INFO ] [MAIN ] BTU: Starting full primary refresh. [2025-11-04T17:48:06.843208] [INFO ] [REMOTE ] BTU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:48:06.843482] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-01. [2025-11-04T17:48:08.254854] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T17:48:08.270902] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-02. [2025-11-04T17:48:09.454546] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T17:48:09.464567] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-03. [2025-11-04T17:48:09.755157] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-03 (15.99 seconds). [2025-11-04T17:48:09.767445] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-04. [2025-11-04T17:48:10.843069] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T17:48:10.854601] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-04. [2025-11-04T17:48:11.414141] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-11-04T17:48:11.425466] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-05. [2025-11-04T17:48:12.106676] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-06 (12.74 seconds). [2025-11-04T17:48:12.109234] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-07. [2025-11-04T17:48:12.384329] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-11-04T17:48:12.395679] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-05. [2025-11-04T17:48:12.679888] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-11-04T17:48:12.967063] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-08. [2025-11-04T17:48:13.093120] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-11-04T17:48:13.464869] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-11-04T17:48:13.767053] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-09. [2025-11-04T17:48:13.872583] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T17:48:14.276067] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-11-04T17:48:14.567088] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-10. [2025-11-04T17:48:15.084035] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-11-04T17:48:15.367100] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-11. [2025-11-04T17:48:15.940569] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T17:48:15.941785] [INFO ] [REMOTE ] BTO: Full fetch complete (13261 records). [2025-11-04T17:48:15.941858] [DEBUG] [VALIDATOR ] BTO: Validating primary data. [2025-11-04T17:48:16.172346] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-06. [2025-11-04T17:48:16.341966] [DEBUG] [VALIDATOR ] BTO: All expected trading days are present. [2025-11-04T17:48:16.342083] [DEBUG] [VALIDATOR ] BTO: Primary data validation passed. [2025-11-04T17:48:16.342817] [INFO ] [MAIN ] BTO: Fetched primary data is valid. [2025-11-04T17:48:16.342883] [DEBUG] [LOCAL ] BTO: Saving primary data. [2025-11-04T17:48:16.343504] [DEBUG] [LOCAL ] BTO: Trimmed 13261 records to 8663 (keeping 200 days of data) [2025-11-04T17:48:16.407398] [DEBUG] [LOCAL ] BTO: Primary data saved (8663 records). [2025-11-04T17:48:16.407563] [INFO ] [MAIN ] BTO: Full history saved. [2025-11-04T17:48:16.407718] [INFO ] [MAIN ] Update Progress: 353/2367 symbols completed. BTO took 32.7s. Estimated remaining: 4h 35m 6s. Rate: 7.3 symbols/min [2025-11-04T17:48:16.580966] [INFO ] [MAIN ] BTX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:48:16.581228] [DEBUG] [LOCAL ] BTX: No existing data file found in primary storage. [2025-11-04T17:48:16.581268] [WARN ] [MAIN ] BTX: No existing data found, performing full refresh instead. [2025-11-04T17:48:16.581395] [INFO ] [MAIN ] BTX: Wiping existing primary data before full refresh. [2025-11-04T17:48:16.581437] [INFO ] [LOCAL ] BTX: Starting deletion of primary data. [2025-11-04T17:48:16.581480] [DEBUG] [LOCAL ] BTX: No primary data file found to delete. [2025-11-04T17:48:16.581522] [INFO ] [MAIN ] BTX: Starting full primary refresh. [2025-11-04T17:48:16.961456] [INFO ] [REMOTE ] BTX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:48:16.967028] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-01. [2025-11-04T17:48:17.117938] [DEBUG] [REMOTE ] BTX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:48:17.118500] [WARN ] [REMOTE ] BTX: No data returned for month 2025-01 [2025-11-04T17:48:17.118601] [FATAL] [REMOTE ] BTX: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:48:17.118685] [FATAL] [MAIN ] BTX: Failed to fetch primary data. [2025-11-04T17:48:17.118755] [WARN ] [MAIN ] BTX: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:48:17.118807] [INFO ] [LOCAL ] BTX: Starting deletion of primary data. [2025-11-04T17:48:17.118898] [DEBUG] [LOCAL ] BTX: No primary data file found to delete. [2025-11-04T17:48:17.118948] [INFO ] [LOCAL ] BTX: Starting deletion of derivative data. [2025-11-04T17:48:17.118991] [DEBUG] [LOCAL ] BTX: No derivative data file found to delete. [2025-11-04T17:48:17.119049] [INFO ] [MAIN ] Update Progress: 354/2367 symbols completed. BTX took 0.7s. Estimated remaining: 4h 34m 16s. Rate: 7.3 symbols/min [2025-11-04T17:48:17.320695] [INFO ] [MAIN ] BTZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:48:17.349613] [DEBUG] [LOCAL ] BTZ: Loaded 21646 existing records from primary storage. [2025-11-04T17:48:17.350055] [INFO ] [MAIN ] BTZ: Loaded 21646 existing records. [2025-11-04T17:48:17.574637] [WARN ] [MAIN ] BTZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:48:17.575232] [INFO ] [MAIN ] BTZ: Wiping existing primary data before full refresh. [2025-11-04T17:48:17.575293] [INFO ] [LOCAL ] BTZ: Starting deletion of primary data. [2025-11-04T17:48:17.575686] [DEBUG] [LOCAL ] BTZ: Deleted primary data file. [2025-11-04T17:48:17.575745] [INFO ] [MAIN ] BTZ: Starting full primary refresh. [2025-11-04T17:48:17.767115] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-06. [2025-11-04T17:48:17.975630] [INFO ] [REMOTE ] BTZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:48:18.007913] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-11-04T17:48:18.567093] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-07. [2025-11-04T17:48:19.368031] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-01. [2025-11-04T17:48:19.373868] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-11-04T17:48:20.064263] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-11-04T17:48:20.112763] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-11-04T17:48:20.167080] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-07. [2025-11-04T17:48:20.967044] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-02. [2025-11-04T17:48:21.649367] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T17:48:21.691875] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T17:48:21.767002] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-03. [2025-11-04T17:48:22.440153] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-11-04T17:48:22.567053] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-08. [2025-11-04T17:48:23.367021] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-08. [2025-11-04T17:48:23.961850] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T17:48:24.167083] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-09. [2025-11-04T17:48:24.967086] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-04. [2025-11-04T17:48:25.012594] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-11-04T17:48:25.465361] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T17:48:25.591064] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-10 (19.82 seconds). [2025-11-04T17:48:25.767249] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-10. [2025-11-04T17:48:25.883885] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T17:48:26.567073] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-09. [2025-11-04T17:48:27.288868] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-11-04T17:48:27.367033] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-05. [2025-11-04T17:48:28.057144] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-11-04T17:48:28.097845] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T17:48:28.167004] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-06. [2025-11-04T17:48:28.967047] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-11. [2025-11-04T17:48:29.248912] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T17:48:29.767059] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-10. [2025-11-04T17:48:30.417631] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-11 (1.45 seconds). [2025-11-04T17:48:30.420109] [INFO ] [REMOTE ] BTE: Full fetch complete (76348 records). [2025-11-04T17:48:30.420238] [DEBUG] [VALIDATOR ] BTE: Validating primary data. [2025-11-04T17:48:30.577110] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-11. [2025-11-04T17:48:30.853288] [DEBUG] [VALIDATOR ] BTE: All expected trading days are present. [2025-11-04T17:48:30.853399] [DEBUG] [VALIDATOR ] BTE: Primary data validation passed. [2025-11-04T17:48:30.854245] [INFO ] [MAIN ] BTE: Fetched primary data is valid. [2025-11-04T17:48:30.854309] [DEBUG] [LOCAL ] BTE: Saving primary data. [2025-11-04T17:48:30.855807] [DEBUG] [LOCAL ] BTE: Trimmed 76348 records to 52650 (keeping 200 days of data) [2025-11-04T17:48:31.164219] [DEBUG] [LOCAL ] BTE: Primary data saved (52650 records). [2025-11-04T17:48:31.164363] [INFO ] [MAIN ] BTE: Full history saved. [2025-11-04T17:48:31.164440] [INFO ] [MAIN ] Update Progress: 355/2367 symbols completed. BTE took 52.7s. Estimated remaining: 4h 34m 41s. Rate: 7.3 symbols/min [2025-11-04T17:48:31.333059] [INFO ] [MAIN ] BUD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:48:31.378436] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-07. [2025-11-04T17:48:31.381498] [DEBUG] [LOCAL ] BUD: Loaded 52579 existing records from primary storage. [2025-11-04T17:48:31.381842] [INFO ] [MAIN ] BUD: Loaded 52579 existing records. [2025-11-04T17:48:31.591003] [WARN ] [MAIN ] BUD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:48:31.592165] [INFO ] [MAIN ] BUD: Wiping existing primary data before full refresh. [2025-11-04T17:48:31.592227] [INFO ] [LOCAL ] BUD: Starting deletion of primary data. [2025-11-04T17:48:31.592960] [DEBUG] [LOCAL ] BUD: Deleted primary data file. [2025-11-04T17:48:31.593020] [INFO ] [MAIN ] BUD: Starting full primary refresh. [2025-11-04T17:48:31.762506] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-11-04T17:48:32.014306] [INFO ] [REMOTE ] BUD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:48:32.019468] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-11 (1.44 seconds). [2025-11-04T17:48:32.021590] [INFO ] [REMOTE ] BTU: Full fetch complete (77317 records). [2025-11-04T17:48:32.021754] [DEBUG] [VALIDATOR ] BTU: Validating primary data. [2025-11-04T17:48:32.168898] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-01. [2025-11-04T17:48:32.343230] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T17:48:32.444073] [DEBUG] [VALIDATOR ] BTU: All expected trading days are present. [2025-11-04T17:48:32.444167] [DEBUG] [VALIDATOR ] BTU: Primary data validation passed. [2025-11-04T17:48:32.445010] [INFO ] [MAIN ] BTU: Fetched primary data is valid. [2025-11-04T17:48:32.445079] [DEBUG] [LOCAL ] BTU: Saving primary data. [2025-11-04T17:48:32.446578] [DEBUG] [LOCAL ] BTU: Trimmed 77317 records to 53319 (keeping 200 days of data) [2025-11-04T17:48:32.759760] [DEBUG] [LOCAL ] BTU: Primary data saved (53319 records). [2025-11-04T17:48:32.759900] [INFO ] [MAIN ] BTU: Full history saved. [2025-11-04T17:48:32.760635] [INFO ] [MAIN ] Update Progress: 356/2367 symbols completed. BTU took 26.7s. Estimated remaining: 4h 33m 55s. Rate: 7.3 symbols/min [2025-11-04T17:48:32.943705] [INFO ] [MAIN ] BUI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:48:32.957543] [DEBUG] [LOCAL ] BUI: Loaded 11477 existing records from primary storage. [2025-11-04T17:48:32.957787] [INFO ] [MAIN ] BUI: Loaded 11477 existing records. [2025-11-04T17:48:32.972337] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-11. [2025-11-04T17:48:33.137088] [WARN ] [MAIN ] BUI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:48:33.137324] [INFO ] [MAIN ] BUI: Wiping existing primary data before full refresh. [2025-11-04T17:48:33.137371] [INFO ] [LOCAL ] BUI: Starting deletion of primary data. [2025-11-04T17:48:33.137544] [DEBUG] [LOCAL ] BUI: Deleted primary data file. [2025-11-04T17:48:33.137581] [INFO ] [MAIN ] BUI: Starting full primary refresh. [2025-11-04T17:48:33.490537] [INFO ] [REMOTE ] BUI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:48:33.778131] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-01. [2025-11-04T17:48:33.780971] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-11-04T17:48:34.278814] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-11 (1.31 seconds). [2025-11-04T17:48:34.280951] [INFO ] [REMOTE ] BTI: Full fetch complete (77663 records). [2025-11-04T17:48:34.281061] [DEBUG] [VALIDATOR ] BTI: Validating primary data. [2025-11-04T17:48:34.572935] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-02. [2025-11-04T17:48:34.669566] [DEBUG] [VALIDATOR ] BTI: All expected trading days are present. [2025-11-04T17:48:34.669693] [DEBUG] [VALIDATOR ] BTI: Primary data validation passed. [2025-11-04T17:48:34.670555] [INFO ] [MAIN ] BTI: Fetched primary data is valid. [2025-11-04T17:48:34.670633] [DEBUG] [LOCAL ] BTI: Saving primary data. [2025-11-04T17:48:34.678981] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T17:48:34.682952] [DEBUG] [LOCAL ] BTI: Trimmed 77663 records to 53585 (keeping 200 days of data) [2025-11-04T17:48:35.009138] [DEBUG] [LOCAL ] BTI: Primary data saved (53585 records). [2025-11-04T17:48:35.009283] [INFO ] [MAIN ] BTI: Full history saved. [2025-11-04T17:48:35.010006] [INFO ] [MAIN ] Update Progress: 357/2367 symbols completed. BTI took 55.0s. Estimated remaining: 4h 33m 14s. Rate: 7.4 symbols/min [2025-11-04T17:48:35.367032] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-08. [2025-11-04T17:48:35.964340] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-11-04T17:48:36.072787] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T17:48:36.167046] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-03. [2025-11-04T17:48:36.967021] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-02. [2025-11-04T17:48:37.523801] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-11-04T17:48:37.567851] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T17:48:37.767047] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-03. [2025-11-04T17:48:38.273520] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-11-04T17:48:38.567048] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-04. [2025-11-04T17:48:39.125173] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-11-04T17:48:39.367045] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-05. [2025-11-04T17:48:39.965852] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-11-04T17:48:40.167094] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-04. [2025-11-04T17:48:40.967040] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-09. [2025-11-04T17:48:41.723133] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T17:48:41.740947] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-11-04T17:48:41.767016] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-05. [2025-11-04T17:48:42.567088] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-10. [2025-11-04T17:48:43.167201] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T17:48:43.308449] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-11-04T17:48:43.367031] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-11. [2025-11-04T17:48:43.981785] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T17:48:43.983167] [INFO ] [REMOTE ] BTZ: Full fetch complete (32894 records). [2025-11-04T17:48:43.983261] [DEBUG] [VALIDATOR ] BTZ: Validating primary data. [2025-11-04T17:48:43.984195] [FATAL] [VALIDATOR ] BTZ: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:48:43.984284] [FATAL] [MAIN ] BTZ: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:48:43.984327] [WARN ] [MAIN ] BTZ: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:48:43.984367] [INFO ] [LOCAL ] BTZ: Starting deletion of primary data. [2025-11-04T17:48:43.984429] [DEBUG] [LOCAL ] BTZ: No primary data file found to delete. [2025-11-04T17:48:43.984466] [INFO ] [LOCAL ] BTZ: Starting deletion of derivative data. [2025-11-04T17:48:43.984673] [DEBUG] [LOCAL ] BTZ: Deleted derivative data file. [2025-11-04T17:48:43.984782] [INFO ] [MAIN ] Update Progress: 358/2367 symbols completed. BTZ took 26.9s. Estimated remaining: 4h 33m 10s. Rate: 7.4 symbols/min [2025-11-04T17:48:44.167117] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-06. [2025-11-04T17:48:44.875374] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T17:48:44.967076] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-07. [2025-11-04T17:48:45.571568] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T17:48:45.767035] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-08. [2025-11-04T17:48:46.343439] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-11-04T17:48:46.567042] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-09. [2025-11-04T17:48:47.135076] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-11-04T17:48:47.367079] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-10. [2025-11-04T17:48:47.954737] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-11-04T17:48:48.167098] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-06. [2025-11-04T17:48:48.967094] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-11. [2025-11-04T17:48:49.486163] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-11 (0.52 seconds). [2025-11-04T17:48:49.487082] [INFO ] [REMOTE ] BUI: Full fetch complete (16405 records). [2025-11-04T17:48:49.487154] [DEBUG] [VALIDATOR ] BUI: Validating primary data. [2025-11-04T17:48:49.864939] [DEBUG] [VALIDATOR ] BUI: All expected trading days are present. [2025-11-04T17:48:49.865034] [DEBUG] [VALIDATOR ] BUI: Primary data validation passed. [2025-11-04T17:48:49.865439] [INFO ] [MAIN ] BUI: Fetched primary data is valid. [2025-11-04T17:48:49.865512] [DEBUG] [LOCAL ] BUI: Saving primary data. [2025-11-04T17:48:49.875624] [DEBUG] [LOCAL ] BUI: Trimmed 16405 records to 11487 (keeping 200 days of data) [2025-11-04T17:48:49.894009] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-11-04T17:48:49.954440] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-07. [2025-11-04T17:48:49.975928] [DEBUG] [LOCAL ] BUI: Primary data saved (11487 records). [2025-11-04T17:48:49.976192] [INFO ] [MAIN ] BUI: Full history saved. [2025-11-04T17:48:49.976288] [INFO ] [MAIN ] Update Progress: 359/2367 symbols completed. BUI took 17.2s. Estimated remaining: 4h 32m 50s. Rate: 7.4 symbols/min [2025-11-04T17:48:51.367331] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T17:48:51.380006] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-08. [2025-11-04T17:48:52.841998] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-11-04T17:48:52.853825] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-09. [2025-11-04T17:48:54.209388] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T17:48:54.220703] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-10. [2025-11-04T17:48:55.767615] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-11-04T17:48:55.779848] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-11. [2025-11-04T17:48:57.006162] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T17:48:57.008285] [INFO ] [REMOTE ] BUD: Full fetch complete (76492 records). [2025-11-04T17:48:57.008382] [DEBUG] [VALIDATOR ] BUD: Validating primary data. [2025-11-04T17:48:57.393830] [DEBUG] [VALIDATOR ] BUD: All expected trading days are present. [2025-11-04T17:48:57.393908] [DEBUG] [VALIDATOR ] BUD: Primary data validation passed. [2025-11-04T17:48:57.394696] [INFO ] [MAIN ] BUD: Fetched primary data is valid. [2025-11-04T17:48:57.394758] [DEBUG] [LOCAL ] BUD: Saving primary data. [2025-11-04T17:48:57.396118] [DEBUG] [LOCAL ] BUD: Trimmed 76492 records to 52576 (keeping 200 days of data) [2025-11-04T17:48:57.695377] [DEBUG] [LOCAL ] BUD: Primary data saved (52576 records). [2025-11-04T17:48:57.695517] [INFO ] [MAIN ] BUD: Full history saved. [2025-11-04T17:48:57.695597] [INFO ] [MAIN ] Update Progress: 360/2367 symbols completed. BUD took 26.5s. Estimated remaining: 4h 32m 39s. Rate: 7.4 symbols/min [2025-11-04T17:48:58.243131] [INFO ] [MAIN ] BUR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:48:58.419094] [INFO ] [MAIN ] BUXX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:48:58.459479] [INFO ] [MAIN ] BV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:48:58.462367] [INFO ] [MAIN ] BURL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:48:58.463515] [DEBUG] [LOCAL ] BURL: No existing data file found in primary storage. [2025-11-04T17:48:58.463586] [WARN ] [MAIN ] BURL: No existing data found, performing full refresh instead. [2025-11-04T17:48:58.464129] [INFO ] [MAIN ] BURL: Wiping existing primary data before full refresh. [2025-11-04T17:48:58.464219] [INFO ] [LOCAL ] BURL: Starting deletion of primary data. [2025-11-04T17:48:58.464290] [DEBUG] [LOCAL ] BURL: No primary data file found to delete. [2025-11-04T17:48:58.464327] [INFO ] [MAIN ] BURL: Starting full primary refresh. [2025-11-04T17:48:58.582413] [DEBUG] [LOCAL ] BUXX: Loaded 15026 existing records from primary storage. [2025-11-04T17:48:58.592615] [INFO ] [MAIN ] BUXX: Loaded 15026 existing records. [2025-11-04T17:48:58.844034] [DEBUG] [LOCAL ] BUR: Loaded 45527 existing records from primary storage. [2025-11-04T17:48:58.854686] [INFO ] [MAIN ] BUR: Loaded 45527 existing records. [2025-11-04T17:48:58.907477] [DEBUG] [LOCAL ] BV: Loaded 38370 existing records from primary storage. [2025-11-04T17:48:59.001474] [INFO ] [MAIN ] BV: Loaded 38370 existing records. [2025-11-04T17:48:59.262359] [WARN ] [MAIN ] BUXX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:48:59.377482] [INFO ] [MAIN ] BUXX: Wiping existing primary data before full refresh. [2025-11-04T17:48:59.383055] [INFO ] [LOCAL ] BUXX: Starting deletion of primary data. [2025-11-04T17:48:59.369298] [INFO ] [REMOTE ] BURL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:48:59.405025] [WARN ] [MAIN ] BUR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:48:59.405469] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-01. [2025-11-04T17:48:59.405614] [INFO ] [MAIN ] BUR: Wiping existing primary data before full refresh. [2025-11-04T17:48:59.458611] [INFO ] [LOCAL ] BUR: Starting deletion of primary data. [2025-11-04T17:48:59.441963] [DEBUG] [LOCAL ] BUXX: Deleted primary data file. [2025-11-04T17:48:59.460747] [INFO ] [MAIN ] BUXX: Starting full primary refresh. [2025-11-04T17:48:59.463735] [WARN ] [MAIN ] BV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:48:59.484808] [DEBUG] [LOCAL ] BUR: Deleted primary data file. [2025-11-04T17:48:59.507183] [INFO ] [MAIN ] BV: Wiping existing primary data before full refresh. [2025-11-04T17:48:59.528434] [INFO ] [LOCAL ] BV: Starting deletion of primary data. [2025-11-04T17:48:59.528553] [INFO ] [MAIN ] BUR: Starting full primary refresh. [2025-11-04T17:48:59.596429] [DEBUG] [LOCAL ] BV: Deleted primary data file. [2025-11-04T17:48:59.608361] [INFO ] [MAIN ] BV: Starting full primary refresh. [2025-11-04T17:49:00.474728] [INFO ] [REMOTE ] BUXX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:49:00.505761] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-01. [2025-11-04T17:49:00.567849] [INFO ] [REMOTE ] BUR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:49:00.594934] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-01. [2025-11-04T17:49:00.594627] [INFO ] [REMOTE ] BV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:49:00.595244] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-01. [2025-11-04T17:49:00.862460] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-11-04T17:49:00.870902] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-02. [2025-11-04T17:49:01.290459] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-11-04T17:49:01.293383] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-02. [2025-11-04T17:49:01.701561] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-11-04T17:49:01.711349] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-02. [2025-11-04T17:49:01.756832] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T17:49:01.765177] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-02. [2025-11-04T17:49:01.932954] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T17:49:01.941588] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-03. [2025-11-04T17:49:01.961392] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-11-04T17:49:01.964077] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-03. [2025-11-04T17:49:02.600577] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T17:49:02.603785] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-04. [2025-11-04T17:49:02.715667] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-11-04T17:49:02.724211] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-03. [2025-11-04T17:49:02.799327] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T17:49:03.242094] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-11-04T17:49:03.299746] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T17:49:03.367057] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-05. [2025-11-04T17:49:03.910820] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T17:49:03.966531] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-11-04T17:49:04.167081] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-03. [2025-11-04T17:49:04.967090] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-04. [2025-11-04T17:49:05.255873] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T17:49:05.767039] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-04. [2025-11-04T17:49:06.302798] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T17:49:06.567079] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-06. [2025-11-04T17:49:07.031026] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T17:49:07.096740] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-11-04T17:49:07.367048] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-07. [2025-11-04T17:49:08.038932] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T17:49:08.167029] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-05. [2025-11-04T17:49:08.967058] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-05. [2025-11-04T17:49:09.436203] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-11-04T17:49:09.767042] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-04. [2025-11-04T17:49:10.056453] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-11-04T17:49:10.567085] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-06. [2025-11-04T17:49:10.812124] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-11-04T17:49:11.367045] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-05. [2025-11-04T17:49:11.527975] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-11-04T17:49:12.167143] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-06. [2025-11-04T17:49:12.451461] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-11-04T17:49:12.967130] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-06. [2025-11-04T17:49:13.592714] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-11-04T17:49:13.775681] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-07. [2025-11-04T17:49:13.779919] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T17:49:14.567063] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-07. [2025-11-04T17:49:15.129057] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T17:49:15.367111] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-08. [2025-11-04T17:49:15.624455] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-11-04T17:49:16.136849] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T17:49:16.167029] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-08. [2025-11-04T17:49:16.967110] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-08. [2025-11-04T17:49:17.562877] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T17:49:17.767141] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-07. [2025-11-04T17:49:18.155553] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T17:49:18.567140] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-09. [2025-11-04T17:49:19.206781] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:49:19.328773] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-11-04T17:49:19.367070] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-09. [2025-11-04T17:49:20.167134] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-09. [2025-11-04T17:49:20.547905] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T17:49:20.967042] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-10. [2025-11-04T17:49:21.310612] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T17:49:21.767096] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-08. [2025-11-04T17:49:22.189835] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T17:49:22.567056] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-10. [2025-11-04T17:49:22.936933] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T17:49:23.367041] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-09. [2025-11-04T17:49:24.020923] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T17:49:24.167036] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-11. [2025-11-04T17:49:24.645491] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T17:49:24.967066] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-10. [2025-11-04T17:49:25.392782] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T17:49:25.394671] [INFO ] [REMOTE ] BURL: Full fetch complete (65892 records). [2025-11-04T17:49:25.394751] [DEBUG] [VALIDATOR ] BURL: Validating primary data. [2025-11-04T17:49:25.770564] [DEBUG] [VALIDATOR ] BURL: All expected trading days are present. [2025-11-04T17:49:25.770280] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-10. [2025-11-04T17:49:25.770731] [DEBUG] [VALIDATOR ] BURL: Primary data validation passed. [2025-11-04T17:49:25.772231] [INFO ] [MAIN ] BURL: Fetched primary data is valid. [2025-11-04T17:49:25.772377] [DEBUG] [LOCAL ] BURL: Saving primary data. [2025-11-04T17:49:25.773709] [DEBUG] [LOCAL ] BURL: Trimmed 65892 records to 44993 (keeping 200 days of data) [2025-11-04T17:49:26.039760] [DEBUG] [LOCAL ] BURL: Primary data saved (44993 records). [2025-11-04T17:49:26.039897] [INFO ] [MAIN ] BURL: Full history saved. [2025-11-04T17:49:26.039993] [INFO ] [MAIN ] Update Progress: 361/2367 symbols completed. BURL took 28.3s. Estimated remaining: 4h 34m 23s. Rate: 7.3 symbols/min [2025-11-04T17:49:26.209591] [INFO ] [MAIN ] BVN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:49:26.262541] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-11-04T17:49:26.288070] [DEBUG] [LOCAL ] BVN: Loaded 44730 existing records from primary storage. [2025-11-04T17:49:26.288587] [INFO ] [MAIN ] BVN: Loaded 44730 existing records. [2025-11-04T17:49:26.465863] [WARN ] [MAIN ] BVN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:49:26.466135] [INFO ] [MAIN ] BVN: Wiping existing primary data before full refresh. [2025-11-04T17:49:26.466186] [INFO ] [LOCAL ] BVN: Starting deletion of primary data. [2025-11-04T17:49:26.466369] [DEBUG] [LOCAL ] BVN: Deleted primary data file. [2025-11-04T17:49:26.466406] [INFO ] [MAIN ] BVN: Starting full primary refresh. [2025-11-04T17:49:26.572312] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-11. [2025-11-04T17:49:26.637517] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-11-04T17:49:26.838135] [INFO ] [REMOTE ] BVN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:49:27.367045] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-01. [2025-11-04T17:49:27.761400] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T17:49:27.763289] [INFO ] [REMOTE ] BUR: Full fetch complete (65144 records). [2025-11-04T17:49:27.763478] [DEBUG] [VALIDATOR ] BUR: Validating primary data. [2025-11-04T17:49:28.141291] [DEBUG] [VALIDATOR ] BUR: All expected trading days are present. [2025-11-04T17:49:28.141366] [DEBUG] [VALIDATOR ] BUR: Primary data validation passed. [2025-11-04T17:49:28.142041] [INFO ] [MAIN ] BUR: Fetched primary data is valid. [2025-11-04T17:49:28.142102] [DEBUG] [LOCAL ] BUR: Saving primary data. [2025-11-04T17:49:28.143308] [DEBUG] [LOCAL ] BUR: Trimmed 65144 records to 45609 (keeping 200 days of data) [2025-11-04T17:49:28.197618] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-11. [2025-11-04T17:49:28.422185] [DEBUG] [LOCAL ] BUR: Primary data saved (45609 records). [2025-11-04T17:49:28.422370] [INFO ] [MAIN ] BUR: Full history saved. [2025-11-04T17:49:28.422556] [INFO ] [MAIN ] Update Progress: 362/2367 symbols completed. BUR took 30.7s. Estimated remaining: 4h 33m 43s. Rate: 7.3 symbols/min [2025-11-04T17:49:28.591814] [INFO ] [MAIN ] BW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:49:28.613840] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T17:49:28.655729] [DEBUG] [LOCAL ] BW: Loaded 40518 existing records from primary storage. [2025-11-04T17:49:28.656013] [INFO ] [MAIN ] BW: Loaded 40518 existing records. [2025-11-04T17:49:28.812499] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T17:49:28.833160] [INFO ] [REMOTE ] BUXX: Full fetch complete (20815 records). [2025-11-04T17:49:28.837095] [DEBUG] [VALIDATOR ] BUXX: Validating primary data. [2025-11-04T17:49:28.836959] [WARN ] [MAIN ] BW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:49:28.837815] [FATAL] [VALIDATOR ] BUXX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:49:28.837945] [INFO ] [MAIN ] BW: Wiping existing primary data before full refresh. [2025-11-04T17:49:28.838113] [INFO ] [LOCAL ] BW: Starting deletion of primary data. [2025-11-04T17:49:28.838055] [FATAL] [MAIN ] BUXX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:49:28.838270] [WARN ] [MAIN ] BUXX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:49:28.838347] [INFO ] [LOCAL ] BUXX: Starting deletion of primary data. [2025-11-04T17:49:28.838387] [DEBUG] [LOCAL ] BW: Deleted primary data file. [2025-11-04T17:49:28.838636] [INFO ] [MAIN ] BW: Starting full primary refresh. [2025-11-04T17:49:28.838574] [DEBUG] [LOCAL ] BUXX: No primary data file found to delete. [2025-11-04T17:49:28.881588] [INFO ] [LOCAL ] BUXX: Starting deletion of derivative data. [2025-11-04T17:49:28.918715] [DEBUG] [LOCAL ] BUXX: Deleted derivative data file. [2025-11-04T17:49:28.937037] [INFO ] [MAIN ] Update Progress: 363/2367 symbols completed. BUXX took 31.2s. Estimated remaining: 4h 32m 52s. Rate: 7.3 symbols/min [2025-11-04T17:49:28.971376] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-02. [2025-11-04T17:49:29.308316] [INFO ] [MAIN ] BWA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:49:29.387894] [INFO ] [REMOTE ] BW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:49:29.430055] [DEBUG] [LOCAL ] BWA: Loaded 52015 existing records from primary storage. [2025-11-04T17:49:29.430368] [INFO ] [MAIN ] BWA: Loaded 52015 existing records. [2025-11-04T17:49:29.634518] [WARN ] [MAIN ] BWA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:49:29.634771] [INFO ] [MAIN ] BWA: Wiping existing primary data before full refresh. [2025-11-04T17:49:29.634822] [INFO ] [LOCAL ] BWA: Starting deletion of primary data. [2025-11-04T17:49:29.635047] [DEBUG] [LOCAL ] BWA: Deleted primary data file. [2025-11-04T17:49:29.635104] [INFO ] [MAIN ] BWA: Starting full primary refresh. [2025-11-04T17:49:29.779001] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-01. [2025-11-04T17:49:30.000801] [INFO ] [REMOTE ] BWA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:49:30.217567] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-11-04T17:49:30.567090] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-03. [2025-11-04T17:49:31.104780] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T17:49:31.367043] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-02. [2025-11-04T17:49:31.630046] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-11-04T17:49:32.167041] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-04. [2025-11-04T17:49:32.370234] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T17:49:32.967052] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-03. [2025-11-04T17:49:33.144945] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-11-04T17:49:33.767038] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-01. [2025-11-04T17:49:33.987893] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-11-04T17:49:34.567045] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-05. [2025-11-04T17:49:35.134416] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-11-04T17:49:35.367092] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-04. [2025-11-04T17:49:35.746190] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-11-04T17:49:36.167085] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-02. [2025-11-04T17:49:36.685260] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T17:49:36.967036] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-11. [2025-11-04T17:49:37.438044] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T17:49:37.767031] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-06. [2025-11-04T17:49:38.223211] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-11 (1.26 seconds). [2025-11-04T17:49:38.224768] [INFO ] [REMOTE ] BV: Full fetch complete (55448 records). [2025-11-04T17:49:38.224872] [DEBUG] [VALIDATOR ] BV: Validating primary data. [2025-11-04T17:49:38.588556] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-05. [2025-11-04T17:49:38.608468] [DEBUG] [VALIDATOR ] BV: All expected trading days are present. [2025-11-04T17:49:38.608546] [DEBUG] [VALIDATOR ] BV: Primary data validation passed. [2025-11-04T17:49:38.609204] [INFO ] [MAIN ] BV: Fetched primary data is valid. [2025-11-04T17:49:38.609266] [DEBUG] [LOCAL ] BV: Saving primary data. [2025-11-04T17:49:38.610353] [DEBUG] [LOCAL ] BV: Trimmed 55448 records to 38357 (keeping 200 days of data) [2025-11-04T17:49:38.834531] [DEBUG] [LOCAL ] BV: Primary data saved (38357 records). [2025-11-04T17:49:38.834704] [INFO ] [MAIN ] BV: Full history saved. [2025-11-04T17:49:38.834793] [INFO ] [MAIN ] Update Progress: 364/2367 symbols completed. BV took 41.1s. Estimated remaining: 4h 32m 54s. Rate: 7.3 symbols/min [2025-11-04T17:49:39.030432] [INFO ] [MAIN ] BWG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:49:39.028126] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-11-04T17:49:39.050744] [DEBUG] [LOCAL ] BWG: Loaded 12380 existing records from primary storage. [2025-11-04T17:49:39.050966] [INFO ] [MAIN ] BWG: Loaded 12380 existing records. [2025-11-04T17:49:39.223218] [WARN ] [MAIN ] BWG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:49:39.223432] [INFO ] [MAIN ] BWG: Wiping existing primary data before full refresh. [2025-11-04T17:49:39.223478] [INFO ] [LOCAL ] BWG: Starting deletion of primary data. [2025-11-04T17:49:39.223624] [DEBUG] [LOCAL ] BWG: Deleted primary data file. [2025-11-04T17:49:39.223716] [INFO ] [MAIN ] BWG: Starting full primary refresh. [2025-11-04T17:49:39.394512] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-07. [2025-11-04T17:49:39.583416] [INFO ] [REMOTE ] BWG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:49:39.702227] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T17:49:40.167059] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-03. [2025-11-04T17:49:40.653592] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T17:49:40.967045] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-08. [2025-11-04T17:49:41.434561] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T17:49:41.767040] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-01. [2025-11-04T17:49:42.205878] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T17:49:42.491983] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-11-04T17:49:42.567014] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-02. [2025-11-04T17:49:43.105960] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-11-04T17:49:43.367047] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-04. [2025-11-04T17:49:44.167058] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-09. [2025-11-04T17:49:44.635788] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T17:49:44.967051] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-05. [2025-11-04T17:49:45.363997] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T17:49:45.767030] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-10. [2025-11-04T17:49:46.134744] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T17:49:46.567034] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-06. [2025-11-04T17:49:47.120401] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T17:49:47.367038] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-06. [2025-11-04T17:49:47.919686] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-11-04T17:49:48.167082] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-03. [2025-11-04T17:49:48.521068] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T17:49:48.665931] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-11-04T17:49:48.967077] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-07. [2025-11-04T17:49:49.767059] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-04. [2025-11-04T17:49:49.926560] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T17:49:50.293327] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T17:49:50.567098] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-05. [2025-11-04T17:49:51.133212] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-11-04T17:49:51.367069] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-06. [2025-11-04T17:49:51.905531] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-11-04T17:49:52.167057] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-08. [2025-11-04T17:49:52.967038] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-07. [2025-11-04T17:49:53.373359] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T17:49:53.728446] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-11-04T17:49:53.767105] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-08. [2025-11-04T17:49:54.365120] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-11-04T17:49:54.567047] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-09. [2025-11-04T17:49:55.367050] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-11. [2025-11-04T17:49:55.991624] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T17:49:56.167028] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-10. [2025-11-04T17:49:56.532882] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-11 (1.17 seconds). [2025-11-04T17:49:56.534211] [INFO ] [REMOTE ] BVN: Full fetch complete (63798 records). [2025-11-04T17:49:56.534300] [DEBUG] [VALIDATOR ] BVN: Validating primary data. [2025-11-04T17:49:56.535024] [FATAL] [VALIDATOR ] BVN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:49:56.535113] [FATAL] [MAIN ] BVN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:49:56.535156] [WARN ] [MAIN ] BVN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:49:56.535193] [INFO ] [LOCAL ] BVN: Starting deletion of primary data. [2025-11-04T17:49:56.535247] [DEBUG] [LOCAL ] BVN: No primary data file found to delete. [2025-11-04T17:49:56.535280] [INFO ] [LOCAL ] BVN: Starting deletion of derivative data. [2025-11-04T17:49:56.535444] [DEBUG] [LOCAL ] BVN: Deleted derivative data file. [2025-11-04T17:49:56.536184] [INFO ] [MAIN ] Update Progress: 365/2367 symbols completed. BVN took 30.5s. Estimated remaining: 4h 33m 38s. Rate: 7.3 symbols/min [2025-11-04T17:49:56.702660] [INFO ] [MAIN ] BWLP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:49:56.735670] [DEBUG] [LOCAL ] BWLP: Loaded 31767 existing records from primary storage. [2025-11-04T17:49:56.735933] [INFO ] [MAIN ] BWLP: Loaded 31767 existing records. [2025-11-04T17:49:56.908984] [WARN ] [MAIN ] BWLP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:49:56.909180] [INFO ] [MAIN ] BWLP: Wiping existing primary data before full refresh. [2025-11-04T17:49:56.909226] [INFO ] [LOCAL ] BWLP: Starting deletion of primary data. [2025-11-04T17:49:56.909418] [DEBUG] [LOCAL ] BWLP: Deleted primary data file. [2025-11-04T17:49:56.909473] [INFO ] [MAIN ] BWLP: Starting full primary refresh. [2025-11-04T17:49:56.972307] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-07. [2025-11-04T17:49:57.288920] [INFO ] [REMOTE ] BWLP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:49:57.402595] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-11-04T17:49:57.767050] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-11. [2025-11-04T17:49:58.410786] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:49:58.567009] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-08. [2025-11-04T17:49:58.889246] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T17:49:58.891365] [INFO ] [REMOTE ] BW: Full fetch complete (59404 records). [2025-11-04T17:49:58.891492] [DEBUG] [VALIDATOR ] BW: Validating primary data. [2025-11-04T17:49:59.374280] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-09. [2025-11-04T17:49:59.430809] [DEBUG] [VALIDATOR ] BW: All expected trading days are present. [2025-11-04T17:49:59.430988] [DEBUG] [VALIDATOR ] BW: Primary data validation passed. [2025-11-04T17:49:59.432272] [INFO ] [MAIN ] BW: Fetched primary data is valid. [2025-11-04T17:49:59.432358] [DEBUG] [LOCAL ] BW: Saving primary data. [2025-11-04T17:49:59.434179] [DEBUG] [LOCAL ] BW: Trimmed 59404 records to 40588 (keeping 200 days of data) [2025-11-04T17:49:59.760929] [DEBUG] [LOCAL ] BW: Primary data saved (40588 records). [2025-11-04T17:49:59.761179] [INFO ] [MAIN ] BW: Full history saved. [2025-11-04T17:49:59.761366] [INFO ] [MAIN ] Update Progress: 366/2367 symbols completed. BW took 31.3s. Estimated remaining: 4h 33m 2s. Rate: 7.3 symbols/min [2025-11-04T17:49:59.996728] [INFO ] [MAIN ] BWMX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:50:00.037680] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-11-04T17:50:00.052961] [DEBUG] [LOCAL ] BWMX: Loaded 8148 existing records from primary storage. [2025-11-04T17:50:00.053491] [INFO ] [MAIN ] BWMX: Loaded 8148 existing records. [2025-11-04T17:50:00.172155] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-09. [2025-11-04T17:50:00.239883] [WARN ] [MAIN ] BWMX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:50:00.240424] [INFO ] [MAIN ] BWMX: Wiping existing primary data before full refresh. [2025-11-04T17:50:00.240490] [INFO ] [LOCAL ] BWMX: Starting deletion of primary data. [2025-11-04T17:50:00.240707] [DEBUG] [LOCAL ] BWMX: Deleted primary data file. [2025-11-04T17:50:00.240774] [INFO ] [MAIN ] BWMX: Starting full primary refresh. [2025-11-04T17:50:00.368481] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T17:50:00.611510] [INFO ] [REMOTE ] BWMX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:50:00.967073] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-01. [2025-11-04T17:50:01.570486] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T17:50:01.767081] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-10. [2025-11-04T17:50:02.098981] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T17:50:02.295403] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-11-04T17:50:02.567056] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-11. [2025-11-04T17:50:03.240360] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T17:50:03.241737] [INFO ] [REMOTE ] BWG: Full fetch complete (17333 records). [2025-11-04T17:50:03.241817] [DEBUG] [VALIDATOR ] BWG: Validating primary data. [2025-11-04T17:50:03.372127] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-02. [2025-11-04T17:50:03.633088] [DEBUG] [VALIDATOR ] BWG: All expected trading days are present. [2025-11-04T17:50:03.633199] [DEBUG] [VALIDATOR ] BWG: Primary data validation passed. [2025-11-04T17:50:03.633697] [INFO ] [MAIN ] BWG: Fetched primary data is valid. [2025-11-04T17:50:03.633765] [DEBUG] [LOCAL ] BWG: Saving primary data. [2025-11-04T17:50:03.634839] [DEBUG] [LOCAL ] BWG: Trimmed 17333 records to 12491 (keeping 200 days of data) [2025-11-04T17:50:03.711430] [DEBUG] [LOCAL ] BWG: Primary data saved (12491 records). [2025-11-04T17:50:03.711614] [INFO ] [MAIN ] BWG: Full history saved. [2025-11-04T17:50:03.711737] [INFO ] [MAIN ] Update Progress: 367/2367 symbols completed. BWG took 24.9s. Estimated remaining: 4h 32m 31s. Rate: 7.3 symbols/min [2025-11-04T17:50:03.878843] [INFO ] [MAIN ] BWNB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:50:03.885366] [DEBUG] [LOCAL ] BWNB: Loaded 3995 existing records from primary storage. [2025-11-04T17:50:03.885566] [INFO ] [MAIN ] BWNB: Loaded 3995 existing records. [2025-11-04T17:50:04.058865] [WARN ] [MAIN ] BWNB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:50:04.059108] [INFO ] [MAIN ] BWNB: Wiping existing primary data before full refresh. [2025-11-04T17:50:04.059156] [INFO ] [LOCAL ] BWNB: Starting deletion of primary data. [2025-11-04T17:50:04.059313] [DEBUG] [LOCAL ] BWNB: Deleted primary data file. [2025-11-04T17:50:04.059381] [INFO ] [MAIN ] BWNB: Starting full primary refresh. [2025-11-04T17:50:04.172319] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-10. [2025-11-04T17:50:04.415513] [INFO ] [REMOTE ] BWNB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:50:04.475146] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T17:50:04.967042] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-01. [2025-11-04T17:50:05.426006] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T17:50:05.589437] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-11-04T17:50:05.767088] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-02. [2025-11-04T17:50:06.105248] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-11-04T17:50:06.567064] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-11. [2025-11-04T17:50:07.367024] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-01. [2025-11-04T17:50:07.632698] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T17:50:07.635213] [INFO ] [REMOTE ] BWA: Full fetch complete (75869 records). [2025-11-04T17:50:07.635339] [DEBUG] [VALIDATOR ] BWA: Validating primary data. [2025-11-04T17:50:07.981288] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-11-04T17:50:08.087459] [DEBUG] [VALIDATOR ] BWA: All expected trading days are present. [2025-11-04T17:50:08.087538] [DEBUG] [VALIDATOR ] BWA: Primary data validation passed. [2025-11-04T17:50:08.088368] [INFO ] [MAIN ] BWA: Fetched primary data is valid. [2025-11-04T17:50:08.088430] [DEBUG] [LOCAL ] BWA: Saving primary data. [2025-11-04T17:50:08.089805] [DEBUG] [LOCAL ] BWA: Trimmed 75869 records to 52050 (keeping 200 days of data) [2025-11-04T17:50:08.180990] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-02. [2025-11-04T17:50:08.401032] [DEBUG] [LOCAL ] BWA: Primary data saved (52050 records). [2025-11-04T17:50:08.401177] [INFO ] [MAIN ] BWA: Full history saved. [2025-11-04T17:50:08.401256] [INFO ] [MAIN ] Update Progress: 368/2367 symbols completed. BWA took 39.5s. Estimated remaining: 4h 32m 4s. Rate: 7.3 symbols/min [2025-11-04T17:50:08.566090] [INFO ] [MAIN ] BWSN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:50:08.573129] [DEBUG] [LOCAL ] BWSN: Loaded 5022 existing records from primary storage. [2025-11-04T17:50:08.573292] [INFO ] [MAIN ] BWSN: Loaded 5022 existing records. [2025-11-04T17:50:08.688252] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T17:50:08.768300] [WARN ] [MAIN ] BWSN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:50:08.768512] [INFO ] [MAIN ] BWSN: Wiping existing primary data before full refresh. [2025-11-04T17:50:08.768560] [INFO ] [LOCAL ] BWSN: Starting deletion of primary data. [2025-11-04T17:50:08.768725] [DEBUG] [LOCAL ] BWSN: Deleted primary data file. [2025-11-04T17:50:08.768780] [INFO ] [MAIN ] BWSN: Starting full primary refresh. [2025-11-04T17:50:08.977100] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-03. [2025-11-04T17:50:09.148979] [INFO ] [REMOTE ] BWSN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:50:09.427332] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-11-04T17:50:09.767044] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-03. [2025-11-04T17:50:10.253607] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-11-04T17:50:10.567045] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-01. [2025-11-04T17:50:10.923934] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-01 (0.36 seconds). [2025-11-04T17:50:11.367044] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-03. [2025-11-04T17:50:12.167048] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-04. [2025-11-04T17:50:12.671735] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-11-04T17:50:12.893064] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T17:50:12.967011] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-04. [2025-11-04T17:50:13.767012] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-04. [2025-11-04T17:50:13.839182] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-11-04T17:50:14.130755] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T17:50:14.567030] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-05. [2025-11-04T17:50:15.076693] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-11-04T17:50:15.367083] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-06. [2025-11-04T17:50:16.080679] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T17:50:16.167108] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-07. [2025-11-04T17:50:16.473052] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-07 (0.31 seconds). [2025-11-04T17:50:16.967043] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-08. [2025-11-04T17:50:17.392169] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-11-04T17:50:17.767074] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-05. [2025-11-04T17:50:18.349577] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-11-04T17:50:18.567043] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-09. [2025-11-04T17:50:19.078485] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-11-04T17:50:19.367039] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-06. [2025-11-04T17:50:20.150101] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T17:50:20.167002] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-07. [2025-11-04T17:50:20.722556] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-11-04T17:50:20.967040] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-02. [2025-11-04T17:50:21.716287] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T17:50:21.766996] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-03. [2025-11-04T17:50:22.134490] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-11-04T17:50:22.567074] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-10. [2025-11-04T17:50:22.881247] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-10 (0.31 seconds). [2025-11-04T17:50:23.367059] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-08. [2025-11-04T17:50:23.892966] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-11-04T17:50:24.167030] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-05. [2025-11-04T17:50:24.967103] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-04. [2025-11-04T17:50:25.305446] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T17:50:25.375327] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-11-04T17:50:25.767027] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-11. [2025-11-04T17:50:26.319373] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T17:50:26.320083] [INFO ] [REMOTE ] BWNB: Full fetch complete (5587 records). [2025-11-04T17:50:26.320152] [DEBUG] [VALIDATOR ] BWNB: Validating primary data. [2025-11-04T17:50:26.572212] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-05. [2025-11-04T17:50:26.710677] [DEBUG] [VALIDATOR ] BWNB: All expected trading days are present. [2025-11-04T17:50:26.710762] [DEBUG] [VALIDATOR ] BWNB: Primary data validation passed. [2025-11-04T17:50:26.710916] [INFO ] [MAIN ] BWNB: Fetched primary data is valid. [2025-11-04T17:50:26.710952] [DEBUG] [LOCAL ] BWNB: Saving primary data. [2025-11-04T17:50:26.711313] [DEBUG] [LOCAL ] BWNB: Trimmed 5587 records to 3998 (keeping 200 days of data) [2025-11-04T17:50:26.735126] [DEBUG] [LOCAL ] BWNB: Primary data saved (3998 records). [2025-11-04T17:50:26.735214] [INFO ] [MAIN ] BWNB: Full history saved. [2025-11-04T17:50:26.735284] [INFO ] [MAIN ] Update Progress: 369/2367 symbols completed. BWNB took 23.0s. Estimated remaining: 4h 32m 51s. Rate: 7.3 symbols/min [2025-11-04T17:50:26.907622] [INFO ] [MAIN ] BWXT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:50:26.950395] [DEBUG] [LOCAL ] BWXT: Loaded 47265 existing records from primary storage. [2025-11-04T17:50:26.950717] [INFO ] [MAIN ] BWXT: Loaded 47265 existing records. [2025-11-04T17:50:27.130964] [WARN ] [MAIN ] BWXT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:50:27.131285] [INFO ] [MAIN ] BWXT: Wiping existing primary data before full refresh. [2025-11-04T17:50:27.131341] [INFO ] [LOCAL ] BWXT: Starting deletion of primary data. [2025-11-04T17:50:27.131530] [DEBUG] [LOCAL ] BWXT: Deleted primary data file. [2025-11-04T17:50:27.131571] [INFO ] [MAIN ] BWXT: Starting full primary refresh. [2025-11-04T17:50:27.240261] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T17:50:27.372179] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-06. [2025-11-04T17:50:27.495474] [INFO ] [REMOTE ] BWXT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:50:28.167077] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-09. [2025-11-04T17:50:28.543148] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T17:50:28.669547] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-11-04T17:50:28.967061] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-01. [2025-11-04T17:50:29.767065] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-06. [2025-11-04T17:50:30.295457] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-11-04T17:50:30.440312] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T17:50:30.567077] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-07. [2025-11-04T17:50:30.883332] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-11-04T17:50:31.367046] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-02. [2025-11-04T17:50:32.167045] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-08. [2025-11-04T17:50:32.642763] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T17:50:32.718719] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-11-04T17:50:32.967050] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-09. [2025-11-04T17:50:33.336307] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-11-04T17:50:33.767045] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-10. [2025-11-04T17:50:34.241585] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-11-04T17:50:34.567082] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-10. [2025-11-04T17:50:34.962362] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-11-04T17:50:35.367002] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-11. [2025-11-04T17:50:35.895129] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-11 (0.53 seconds). [2025-11-04T17:50:35.895804] [INFO ] [REMOTE ] BWSN: Full fetch complete (6456 records). [2025-11-04T17:50:35.895872] [DEBUG] [VALIDATOR ] BWSN: Validating primary data. [2025-11-04T17:50:36.174587] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-11. [2025-11-04T17:50:36.259936] [DEBUG] [VALIDATOR ] BWSN: All expected trading days are present. [2025-11-04T17:50:36.260003] [DEBUG] [VALIDATOR ] BWSN: Primary data validation passed. [2025-11-04T17:50:36.260173] [INFO ] [MAIN ] BWSN: Fetched primary data is valid. [2025-11-04T17:50:36.260211] [DEBUG] [LOCAL ] BWSN: Saving primary data. [2025-11-04T17:50:36.260691] [DEBUG] [LOCAL ] BWSN: Trimmed 6456 records to 4995 (keeping 200 days of data) [2025-11-04T17:50:36.289842] [DEBUG] [LOCAL ] BWSN: Primary data saved (4995 records). [2025-11-04T17:50:36.289923] [INFO ] [MAIN ] BWSN: Full history saved. [2025-11-04T17:50:36.289995] [INFO ] [MAIN ] Update Progress: 370/2367 symbols completed. BWSN took 27.9s. Estimated remaining: 4h 32m 50s. Rate: 7.3 symbols/min [2025-11-04T17:50:36.470233] [INFO ] [MAIN ] BX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:50:36.522839] [DEBUG] [LOCAL ] BX: Loaded 53481 existing records from primary storage. [2025-11-04T17:50:36.523169] [INFO ] [MAIN ] BX: Loaded 53481 existing records. [2025-11-04T17:50:36.703809] [WARN ] [MAIN ] BX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:50:36.704082] [INFO ] [MAIN ] BX: Wiping existing primary data before full refresh. [2025-11-04T17:50:36.704137] [INFO ] [LOCAL ] BX: Starting deletion of primary data. [2025-11-04T17:50:36.704357] [DEBUG] [LOCAL ] BX: Deleted primary data file. [2025-11-04T17:50:36.704414] [INFO ] [MAIN ] BX: Starting full primary refresh. [2025-11-04T17:50:36.932624] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-11 (0.76 seconds). [2025-11-04T17:50:36.962360] [INFO ] [REMOTE ] BWMX: Full fetch complete (10208 records). [2025-11-04T17:50:36.973928] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-03. [2025-11-04T17:50:36.974069] [DEBUG] [VALIDATOR ] BWMX: Validating primary data. [2025-11-04T17:50:37.202910] [INFO ] [REMOTE ] BX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:50:37.446135] [DEBUG] [VALIDATOR ] BWMX: All expected trading days are present. [2025-11-04T17:50:37.446207] [DEBUG] [VALIDATOR ] BWMX: Primary data validation passed. [2025-11-04T17:50:37.446452] [INFO ] [MAIN ] BWMX: Fetched primary data is valid. [2025-11-04T17:50:37.446489] [DEBUG] [LOCAL ] BWMX: Saving primary data. [2025-11-04T17:50:37.446989] [DEBUG] [LOCAL ] BWMX: Trimmed 10208 records to 8184 (keeping 200 days of data) [2025-11-04T17:50:37.496483] [DEBUG] [LOCAL ] BWMX: Primary data saved (8184 records). [2025-11-04T17:50:37.496608] [INFO ] [MAIN ] BWMX: Full history saved. [2025-11-04T17:50:37.496708] [INFO ] [MAIN ] Update Progress: 371/2367 symbols completed. BWMX took 37.7s. Estimated remaining: 4h 32m 4s. Rate: 7.3 symbols/min [2025-11-04T17:50:37.671124] [INFO ] [MAIN ] BXC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:50:37.685605] [DEBUG] [LOCAL ] BXC: Loaded 13055 existing records from primary storage. [2025-11-04T17:50:37.685846] [INFO ] [MAIN ] BXC: Loaded 13055 existing records. [2025-11-04T17:50:37.772203] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-01. [2025-11-04T17:50:37.868983] [WARN ] [MAIN ] BXC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:50:37.869192] [INFO ] [MAIN ] BXC: Wiping existing primary data before full refresh. [2025-11-04T17:50:37.869283] [INFO ] [LOCAL ] BXC: Starting deletion of primary data. [2025-11-04T17:50:37.869511] [DEBUG] [LOCAL ] BXC: Deleted primary data file. [2025-11-04T17:50:37.869566] [INFO ] [MAIN ] BXC: Starting full primary refresh. [2025-11-04T17:50:38.227066] [INFO ] [REMOTE ] BXC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:50:38.254286] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T17:50:38.567095] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-01. [2025-11-04T17:50:39.097127] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T17:50:39.229829] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-11-04T17:50:39.367040] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-02. [2025-11-04T17:50:39.942430] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-11-04T17:50:40.167087] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-04. [2025-11-04T17:50:40.967065] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-03. [2025-11-04T17:50:41.536939] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T17:50:41.627958] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-11-04T17:50:41.767093] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-05. [2025-11-04T17:50:42.567061] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-07. [2025-11-04T17:50:43.086387] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T17:50:43.367032] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-04. [2025-11-04T17:50:43.739599] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T17:50:44.081200] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-11-04T17:50:44.167033] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-05. [2025-11-04T17:50:44.727382] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-11-04T17:50:44.967102] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-02. [2025-11-04T17:50:45.767038] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-06. [2025-11-04T17:50:46.297402] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T17:50:46.567070] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-08. [2025-11-04T17:50:47.197766] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-11-04T17:50:47.367051] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-07. [2025-11-04T17:50:47.602589] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T17:50:48.167090] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-06. [2025-11-04T17:50:48.750007] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-11-04T17:50:48.749633] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-11-04T17:50:48.967007] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-08. [2025-11-04T17:50:49.767046] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-09. [2025-11-04T17:50:50.342884] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T17:50:50.543891] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-11-04T17:50:50.567042] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-09. [2025-11-04T17:50:51.367026] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-07. [2025-11-04T17:50:52.084149] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-11-04T17:50:52.117595] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-11-04T17:50:52.167085] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-08. [2025-11-04T17:50:52.746886] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-11-04T17:50:52.967154] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-03. [2025-11-04T17:50:53.767069] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-09. [2025-11-04T17:50:54.358147] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T17:50:54.378234] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-11-04T17:50:54.567034] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-04. [2025-11-04T17:50:55.367064] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-10. [2025-11-04T17:50:55.992001] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T17:50:56.167035] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-05. [2025-11-04T17:50:56.967002] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-10. [2025-11-04T17:50:57.053259] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T17:50:57.681141] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-11-04T17:50:57.767055] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-11. [2025-11-04T17:50:58.059336] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-11-04T17:50:58.567100] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-11. [2025-11-04T17:50:59.238537] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-11 (1.47 seconds). [2025-11-04T17:50:59.240006] [INFO ] [REMOTE ] BWXT: Full fetch complete (67896 records). [2025-11-04T17:50:59.240092] [DEBUG] [VALIDATOR ] BWXT: Validating primary data. [2025-11-04T17:50:59.240979] [FATAL] [VALIDATOR ] BWXT: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:50:59.241255] [FATAL] [MAIN ] BWXT: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:50:59.241324] [WARN ] [MAIN ] BWXT: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:50:59.241371] [INFO ] [LOCAL ] BWXT: Starting deletion of primary data. [2025-11-04T17:50:59.241439] [DEBUG] [LOCAL ] BWXT: No primary data file found to delete. [2025-11-04T17:50:59.241494] [INFO ] [LOCAL ] BWXT: Starting deletion of derivative data. [2025-11-04T17:50:59.241962] [DEBUG] [LOCAL ] BWXT: Deleted derivative data file. [2025-11-04T17:50:59.242073] [INFO ] [MAIN ] Update Progress: 372/2367 symbols completed. BWXT took 32.5s. Estimated remaining: 4h 33m 9s. Rate: 7.3 symbols/min [2025-11-04T17:50:59.367175] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-10. [2025-11-04T17:50:59.410108] [INFO ] [MAIN ] BXMT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:50:59.414411] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-11 (0.85 seconds). [2025-11-04T17:50:59.416105] [INFO ] [REMOTE ] BWLP: Full fetch complete (47745 records). [2025-11-04T17:50:59.416185] [DEBUG] [VALIDATOR ] BWLP: Validating primary data. [2025-11-04T17:50:59.730491] [DEBUG] [LOCAL ] BXMT: Loaded 47088 existing records from primary storage. [2025-11-04T17:50:59.746414] [INFO ] [MAIN ] BXMT: Loaded 47088 existing records. [2025-11-04T17:50:59.930125] [DEBUG] [VALIDATOR ] BWLP: All expected trading days are present. [2025-11-04T17:50:59.946035] [DEBUG] [VALIDATOR ] BWLP: Primary data validation passed. [2025-11-04T17:51:00.001302] [WARN ] [MAIN ] BXMT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:50:59.971606] [INFO ] [MAIN ] BWLP: Fetched primary data is valid. [2025-11-04T17:51:00.001673] [DEBUG] [LOCAL ] BWLP: Saving primary data. [2025-11-04T17:51:00.002026] [INFO ] [MAIN ] BXMT: Wiping existing primary data before full refresh. [2025-11-04T17:51:00.002124] [INFO ] [LOCAL ] BXMT: Starting deletion of primary data. [2025-11-04T17:51:00.002624] [DEBUG] [LOCAL ] BXMT: Deleted primary data file. [2025-11-04T17:51:00.002784] [INFO ] [MAIN ] BXMT: Starting full primary refresh. [2025-11-04T17:51:00.039170] [DEBUG] [LOCAL ] BWLP: Trimmed 47745 records to 31898 (keeping 200 days of data) [2025-11-04T17:51:00.170817] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-06. [2025-11-04T17:51:00.209266] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-11-04T17:51:00.483197] [INFO ] [REMOTE ] BXMT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:51:00.572679] [DEBUG] [LOCAL ] BWLP: Primary data saved (31898 records). [2025-11-04T17:51:00.572806] [INFO ] [MAIN ] BWLP: Full history saved. [2025-11-04T17:51:00.573470] [INFO ] [MAIN ] Update Progress: 373/2367 symbols completed. BWLP took 1m 4s. Estimated remaining: 4h 32m 24s. Rate: 7.3 symbols/min [2025-11-04T17:51:00.739327] [INFO ] [MAIN ] BXMX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:51:00.759562] [DEBUG] [LOCAL ] BXMX: Loaded 18657 existing records from primary storage. [2025-11-04T17:51:00.759826] [INFO ] [MAIN ] BXMX: Loaded 18657 existing records. [2025-11-04T17:51:00.934109] [WARN ] [MAIN ] BXMX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:51:00.934329] [INFO ] [MAIN ] BXMX: Wiping existing primary data before full refresh. [2025-11-04T17:51:00.934375] [INFO ] [LOCAL ] BXMX: Starting deletion of primary data. [2025-11-04T17:51:00.934531] [DEBUG] [LOCAL ] BXMX: Deleted primary data file. [2025-11-04T17:51:00.934568] [INFO ] [MAIN ] BXMX: Starting full primary refresh. [2025-11-04T17:51:00.975774] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-01. [2025-11-04T17:51:01.284543] [INFO ] [REMOTE ] BXMX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:51:01.767124] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-11. [2025-11-04T17:51:02.363517] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-11 (0.60 seconds). [2025-11-04T17:51:02.364531] [INFO ] [REMOTE ] BXC: Full fetch complete (18375 records). [2025-11-04T17:51:02.364605] [DEBUG] [VALIDATOR ] BXC: Validating primary data. [2025-11-04T17:51:02.573236] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-01. [2025-11-04T17:51:02.758181] [DEBUG] [VALIDATOR ] BXC: All expected trading days are present. [2025-11-04T17:51:02.758259] [DEBUG] [VALIDATOR ] BXC: Primary data validation passed. [2025-11-04T17:51:02.758608] [INFO ] [MAIN ] BXC: Fetched primary data is valid. [2025-11-04T17:51:02.758666] [DEBUG] [LOCAL ] BXC: Saving primary data. [2025-11-04T17:51:02.759215] [DEBUG] [LOCAL ] BXC: Trimmed 18375 records to 13090 (keeping 200 days of data) [2025-11-04T17:51:02.834895] [DEBUG] [LOCAL ] BXC: Primary data saved (13090 records). [2025-11-04T17:51:02.835038] [INFO ] [MAIN ] BXC: Full history saved. [2025-11-04T17:51:02.835122] [INFO ] [MAIN ] Update Progress: 374/2367 symbols completed. BXC took 25.3s. Estimated remaining: 4h 31m 44s. Rate: 7.3 symbols/min [2025-11-04T17:51:03.003723] [INFO ] [MAIN ] BXP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:51:03.049753] [DEBUG] [LOCAL ] BXP: Loaded 49729 existing records from primary storage. [2025-11-04T17:51:03.050085] [INFO ] [MAIN ] BXP: Loaded 49729 existing records. [2025-11-04T17:51:03.225138] [WARN ] [MAIN ] BXP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:51:03.225323] [INFO ] [MAIN ] BXP: Wiping existing primary data before full refresh. [2025-11-04T17:51:03.225388] [INFO ] [LOCAL ] BXP: Starting deletion of primary data. [2025-11-04T17:51:03.225559] [DEBUG] [LOCAL ] BXP: Deleted primary data file. [2025-11-04T17:51:03.225599] [INFO ] [MAIN ] BXP: Starting full primary refresh. [2025-11-04T17:51:03.606342] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-11-04T17:51:03.610871] [INFO ] [REMOTE ] BXP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:51:03.611202] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-01. [2025-11-04T17:51:04.167075] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-02. [2025-11-04T17:51:04.716993] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-06 (4.55 seconds). [2025-11-04T17:51:04.851821] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-11-04T17:51:04.881821] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T17:51:04.966993] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-02. [2025-11-04T17:51:05.267428] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-01 (4.29 seconds). [2025-11-04T17:51:05.767171] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-07. [2025-11-04T17:51:06.181753] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T17:51:06.567090] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-02. [2025-11-04T17:51:07.367002] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-03. [2025-11-04T17:51:07.409018] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-11-04T17:51:07.823676] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T17:51:08.167041] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-08. [2025-11-04T17:51:08.576561] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-11-04T17:51:08.967044] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-03. [2025-11-04T17:51:09.743122] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-11-04T17:51:09.766996] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-09. [2025-11-04T17:51:10.368625] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T17:51:10.567080] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-04. [2025-11-04T17:51:11.340691] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-11-04T17:51:11.367050] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-10. [2025-11-04T17:51:11.842264] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T17:51:12.167091] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-03. [2025-11-04T17:51:12.962603] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-11-04T17:51:12.967325] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-05. [2025-11-04T17:51:13.058706] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T17:51:13.767092] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-11. [2025-11-04T17:51:14.125572] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T17:51:14.127671] [INFO ] [REMOTE ] BX: Full fetch complete (77613 records). [2025-11-04T17:51:14.127782] [DEBUG] [VALIDATOR ] BX: Validating primary data. [2025-11-04T17:51:14.476072] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-11-04T17:51:14.581155] [DEBUG] [VALIDATOR ] BX: All expected trading days are present. [2025-11-04T17:51:14.577160] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-04. [2025-11-04T17:51:14.581304] [DEBUG] [VALIDATOR ] BX: Primary data validation passed. [2025-11-04T17:51:14.583128] [INFO ] [MAIN ] BX: Fetched primary data is valid. [2025-11-04T17:51:14.583355] [DEBUG] [LOCAL ] BX: Saving primary data. [2025-11-04T17:51:14.584917] [DEBUG] [LOCAL ] BX: Trimmed 77613 records to 53482 (keeping 200 days of data) [2025-11-04T17:51:14.916341] [DEBUG] [LOCAL ] BX: Primary data saved (53482 records). [2025-11-04T17:51:14.916464] [INFO ] [MAIN ] BX: Full history saved. [2025-11-04T17:51:14.917559] [INFO ] [MAIN ] Update Progress: 375/2367 symbols completed. BX took 38.6s. Estimated remaining: 4h 31m 56s. Rate: 7.3 symbols/min [2025-11-04T17:51:15.085671] [INFO ] [MAIN ] BXSL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:51:15.129562] [DEBUG] [LOCAL ] BXSL: Loaded 48213 existing records from primary storage. [2025-11-04T17:51:15.129873] [INFO ] [MAIN ] BXSL: Loaded 48213 existing records. [2025-11-04T17:51:15.301764] [WARN ] [MAIN ] BXSL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:51:15.301953] [INFO ] [MAIN ] BXSL: Wiping existing primary data before full refresh. [2025-11-04T17:51:15.301998] [INFO ] [LOCAL ] BXSL: Starting deletion of primary data. [2025-11-04T17:51:15.302200] [DEBUG] [LOCAL ] BXSL: Deleted primary data file. [2025-11-04T17:51:15.302255] [INFO ] [MAIN ] BXSL: Starting full primary refresh. [2025-11-04T17:51:15.374474] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-04. [2025-11-04T17:51:15.656002] [INFO ] [REMOTE ] BXSL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:51:15.675973] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T17:51:16.167050] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-06. [2025-11-04T17:51:16.881552] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T17:51:16.967084] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-05. [2025-11-04T17:51:17.596410] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-11-04T17:51:17.767022] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-01. [2025-11-04T17:51:17.957455] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T17:51:18.567057] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-05. [2025-11-04T17:51:19.112555] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T17:51:19.214750] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-11-04T17:51:19.367024] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-06. [2025-11-04T17:51:20.128445] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-11-04T17:51:20.167052] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-06. [2025-11-04T17:51:20.967059] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-07. [2025-11-04T17:51:21.269038] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T17:51:21.733824] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T17:51:21.767018] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-08. [2025-11-04T17:51:22.367898] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-11-04T17:51:22.567056] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-09. [2025-11-04T17:51:23.116241] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-11-04T17:51:23.367026] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-02. [2025-11-04T17:51:24.167049] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-10. [2025-11-04T17:51:24.542250] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T17:51:24.819913] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-11-04T17:51:24.967056] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-07. [2025-11-04T17:51:25.767049] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-07. [2025-11-04T17:51:26.231720] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T17:51:26.567079] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-08. [2025-11-04T17:51:27.110313] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T17:51:27.367038] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-08. [2025-11-04T17:51:27.942302] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T17:51:28.167020] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-09. [2025-11-04T17:51:28.668739] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T17:51:28.967049] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-09. [2025-11-04T17:51:29.479365] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T17:51:29.767050] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-10. [2025-11-04T17:51:30.347401] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T17:51:30.567040] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-03. [2025-11-04T17:51:31.175938] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T17:51:31.367049] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-10. [2025-11-04T17:51:31.808237] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T17:51:32.167042] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-04. [2025-11-04T17:51:32.805518] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T17:51:32.967021] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-11. [2025-11-04T17:51:33.522489] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T17:51:33.767038] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-05. [2025-11-04T17:51:34.145521] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T17:51:34.147531] [INFO ] [REMOTE ] BXP: Full fetch complete (72298 records). [2025-11-04T17:51:34.147624] [DEBUG] [VALIDATOR ] BXP: Validating primary data. [2025-11-04T17:51:34.559853] [DEBUG] [VALIDATOR ] BXP: All expected trading days are present. [2025-11-04T17:51:34.559936] [DEBUG] [VALIDATOR ] BXP: Primary data validation passed. [2025-11-04T17:51:34.560688] [INFO ] [MAIN ] BXP: Fetched primary data is valid. [2025-11-04T17:51:34.560751] [DEBUG] [LOCAL ] BXP: Saving primary data. [2025-11-04T17:51:34.562218] [DEBUG] [LOCAL ] BXP: Trimmed 72298 records to 49702 (keeping 200 days of data) [2025-11-04T17:51:34.615286] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-11. [2025-11-04T17:51:34.881416] [DEBUG] [LOCAL ] BXP: Primary data saved (49702 records). [2025-11-04T17:51:34.881618] [INFO ] [MAIN ] BXP: Full history saved. [2025-11-04T17:51:34.881844] [INFO ] [MAIN ] Update Progress: 376/2367 symbols completed. BXP took 32.0s. Estimated remaining: 4h 32m 51s. Rate: 7.3 symbols/min [2025-11-04T17:51:35.079023] [INFO ] [MAIN ] BY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:51:35.081806] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T17:51:35.110906] [DEBUG] [LOCAL ] BY: Loaded 21260 existing records from primary storage. [2025-11-04T17:51:35.111155] [INFO ] [MAIN ] BY: Loaded 21260 existing records. [2025-11-04T17:51:35.282469] [WARN ] [MAIN ] BY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:51:35.282701] [INFO ] [MAIN ] BY: Wiping existing primary data before full refresh. [2025-11-04T17:51:35.282778] [INFO ] [LOCAL ] BY: Starting deletion of primary data. [2025-11-04T17:51:35.282945] [DEBUG] [LOCAL ] BY: Deleted primary data file. [2025-11-04T17:51:35.282982] [INFO ] [MAIN ] BY: Starting full primary refresh. [2025-11-04T17:51:35.377192] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-06. [2025-11-04T17:51:35.642082] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T17:51:35.657963] [INFO ] [REMOTE ] BY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:51:35.659723] [INFO ] [REMOTE ] BXMX: Full fetch complete (28662 records). [2025-11-04T17:51:35.659816] [DEBUG] [VALIDATOR ] BXMX: Validating primary data. [2025-11-04T17:51:36.044679] [DEBUG] [VALIDATOR ] BXMX: All expected trading days are present. [2025-11-04T17:51:36.044772] [DEBUG] [VALIDATOR ] BXMX: Primary data validation passed. [2025-11-04T17:51:36.045180] [INFO ] [MAIN ] BXMX: Fetched primary data is valid. [2025-11-04T17:51:36.045219] [DEBUG] [LOCAL ] BXMX: Saving primary data. [2025-11-04T17:51:36.046045] [DEBUG] [LOCAL ] BXMX: Trimmed 28662 records to 18619 (keeping 200 days of data) [2025-11-04T17:51:36.154086] [DEBUG] [LOCAL ] BXMX: Primary data saved (18619 records). [2025-11-04T17:51:36.154229] [INFO ] [MAIN ] BXMX: Full history saved. [2025-11-04T17:51:36.154309] [INFO ] [MAIN ] Update Progress: 377/2367 symbols completed. BXMX took 35.6s. Estimated remaining: 4h 32m 6s. Rate: 7.3 symbols/min [2025-11-04T17:51:36.174824] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-01. [2025-11-04T17:51:36.322105] [INFO ] [MAIN ] BYD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:51:36.365913] [DEBUG] [LOCAL ] BYD: Loaded 45108 existing records from primary storage. [2025-11-04T17:51:36.366190] [INFO ] [MAIN ] BYD: Loaded 45108 existing records. [2025-11-04T17:51:36.543066] [WARN ] [MAIN ] BYD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:51:36.543318] [INFO ] [MAIN ] BYD: Wiping existing primary data before full refresh. [2025-11-04T17:51:36.543425] [INFO ] [LOCAL ] BYD: Starting deletion of primary data. [2025-11-04T17:51:36.543631] [DEBUG] [LOCAL ] BYD: Deleted primary data file. [2025-11-04T17:51:36.543721] [INFO ] [MAIN ] BYD: Starting full primary refresh. [2025-11-04T17:51:36.674422] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T17:51:36.931008] [INFO ] [REMOTE ] BYD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:51:36.966990] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-01. [2025-11-04T17:51:37.053396] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T17:51:37.767090] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-07. [2025-11-04T17:51:38.324759] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T17:51:38.567095] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-02. [2025-11-04T17:51:39.136590] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T17:51:39.367108] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-08. [2025-11-04T17:51:39.768710] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T17:51:40.167091] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-11. [2025-11-04T17:51:40.589630] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T17:51:40.967095] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-03. [2025-11-04T17:51:41.464915] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T17:51:41.467037] [INFO ] [REMOTE ] BXMT: Full fetch complete (69841 records). [2025-11-04T17:51:41.467168] [DEBUG] [VALIDATOR ] BXMT: Validating primary data. [2025-11-04T17:51:41.774750] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-09. [2025-11-04T17:51:41.899907] [DEBUG] [VALIDATOR ] BXMT: All expected trading days are present. [2025-11-04T17:51:41.899987] [DEBUG] [VALIDATOR ] BXMT: Primary data validation passed. [2025-11-04T17:51:41.900760] [INFO ] [MAIN ] BXMT: Fetched primary data is valid. [2025-11-04T17:51:41.900821] [DEBUG] [LOCAL ] BXMT: Saving primary data. [2025-11-04T17:51:41.902104] [DEBUG] [LOCAL ] BXMT: Trimmed 69841 records to 47073 (keeping 200 days of data) [2025-11-04T17:51:42.184165] [DEBUG] [LOCAL ] BXMT: Primary data saved (47073 records). [2025-11-04T17:51:42.184465] [INFO ] [MAIN ] BXMT: Full history saved. [2025-11-04T17:51:42.184632] [INFO ] [MAIN ] Update Progress: 378/2367 symbols completed. BXMT took 42.9s. Estimated remaining: 4h 31m 46s. Rate: 7.3 symbols/min [2025-11-04T17:51:42.356345] [INFO ] [MAIN ] BYM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:51:42.389185] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T17:51:42.393586] [DEBUG] [LOCAL ] BYM: Loaded 11300 existing records from primary storage. [2025-11-04T17:51:42.403188] [INFO ] [MAIN ] BYM: Loaded 11300 existing records. [2025-11-04T17:51:42.573259] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-04. [2025-11-04T17:51:42.577965] [WARN ] [MAIN ] BYM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:51:42.578977] [INFO ] [MAIN ] BYM: Wiping existing primary data before full refresh. [2025-11-04T17:51:42.579034] [INFO ] [LOCAL ] BYM: Starting deletion of primary data. [2025-11-04T17:51:42.579186] [DEBUG] [LOCAL ] BYM: Deleted primary data file. [2025-11-04T17:51:42.579223] [INFO ] [MAIN ] BYM: Starting full primary refresh. [2025-11-04T17:51:42.961988] [INFO ] [REMOTE ] BYM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:51:43.173291] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T17:51:43.367028] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-01. [2025-11-04T17:51:43.889875] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T17:51:44.167104] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-05. [2025-11-04T17:51:44.327628] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T17:51:44.967050] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-02. [2025-11-04T17:51:45.256906] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-11-04T17:51:45.520157] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T17:51:45.767084] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-06. [2025-11-04T17:51:46.567062] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-10. [2025-11-04T17:51:46.925022] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T17:51:47.367087] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-03. [2025-11-04T17:51:47.914719] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T17:51:48.101235] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-11-04T17:51:48.167004] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-04. [2025-11-04T17:51:48.967020] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-07. [2025-11-04T17:51:48.983536] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T17:51:49.767050] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-05. [2025-11-04T17:51:50.353364] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T17:51:50.403374] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T17:51:50.567034] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-06. [2025-11-04T17:51:51.328481] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-11-04T17:51:51.367051] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-08. [2025-11-04T17:51:52.167054] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-11. [2025-11-04T17:51:52.624381] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T17:51:52.967047] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-07. [2025-11-04T17:51:53.419813] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T17:51:53.421907] [INFO ] [REMOTE ] BXSL: Full fetch complete (70380 records). [2025-11-04T17:51:53.422003] [DEBUG] [VALIDATOR ] BXSL: Validating primary data. [2025-11-04T17:51:53.767879] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-02. [2025-11-04T17:51:53.826896] [DEBUG] [VALIDATOR ] BXSL: All expected trading days are present. [2025-11-04T17:51:53.827064] [DEBUG] [VALIDATOR ] BXSL: Primary data validation passed. [2025-11-04T17:51:53.827857] [INFO ] [MAIN ] BXSL: Fetched primary data is valid. [2025-11-04T17:51:53.827934] [DEBUG] [LOCAL ] BXSL: Saving primary data. [2025-11-04T17:51:53.843557] [DEBUG] [LOCAL ] BXSL: Trimmed 70380 records to 48227 (keeping 200 days of data) [2025-11-04T17:51:53.845374] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-11-04T17:51:54.136186] [DEBUG] [LOCAL ] BXSL: Primary data saved (48227 records). [2025-11-04T17:51:54.136316] [INFO ] [MAIN ] BXSL: Full history saved. [2025-11-04T17:51:54.137025] [INFO ] [MAIN ] Update Progress: 379/2367 symbols completed. BXSL took 39.2s. Estimated remaining: 4h 31m 58s. Rate: 7.3 symbols/min [2025-11-04T17:51:54.312684] [INFO ] [MAIN ] BZH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:51:54.347170] [DEBUG] [LOCAL ] BZH: Loaded 31369 existing records from primary storage. [2025-11-04T17:51:54.347422] [INFO ] [MAIN ] BZH: Loaded 31369 existing records. [2025-11-04T17:51:54.476527] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T17:51:54.538541] [WARN ] [MAIN ] BZH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:51:54.538767] [INFO ] [MAIN ] BZH: Wiping existing primary data before full refresh. [2025-11-04T17:51:54.538818] [INFO ] [LOCAL ] BZH: Starting deletion of primary data. [2025-11-04T17:51:54.538990] [DEBUG] [LOCAL ] BZH: Deleted primary data file. [2025-11-04T17:51:54.539030] [INFO ] [MAIN ] BZH: Starting full primary refresh. [2025-11-04T17:51:54.572210] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-09. [2025-11-04T17:51:54.910195] [INFO ] [REMOTE ] BZH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:51:55.367054] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-03. [2025-11-04T17:51:55.794728] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T17:51:56.012738] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-11-04T17:51:56.167073] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-01. [2025-11-04T17:51:56.967040] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-04. [2025-11-04T17:51:57.143445] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T17:51:57.566606] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-11-04T17:51:57.767069] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-05. [2025-11-04T17:51:58.422517] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T17:51:58.567087] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-06. [2025-11-04T17:51:59.129703] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-11-04T17:51:59.367063] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-10. [2025-11-04T17:52:00.167066] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-02. [2025-11-04T17:52:00.933880] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-11-04T17:52:00.967011] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-07. [2025-11-04T17:52:01.444835] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T17:52:01.576386] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-11-04T17:52:01.767086] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-08. [2025-11-04T17:52:02.328164] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-11-04T17:52:02.567071] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-09. [2025-11-04T17:52:03.138692] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-11-04T17:52:03.367042] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-11. [2025-11-04T17:52:04.167046] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-08. [2025-11-04T17:52:04.477795] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T17:52:04.479627] [INFO ] [REMOTE ] BYD: Full fetch complete (65664 records). [2025-11-04T17:52:04.479778] [DEBUG] [VALIDATOR ] BYD: Validating primary data. [2025-11-04T17:52:04.870990] [DEBUG] [VALIDATOR ] BYD: All expected trading days are present. [2025-11-04T17:52:04.871079] [DEBUG] [VALIDATOR ] BYD: Primary data validation passed. [2025-11-04T17:52:04.871798] [INFO ] [MAIN ] BYD: Fetched primary data is valid. [2025-11-04T17:52:04.871860] [DEBUG] [LOCAL ] BYD: Saving primary data. [2025-11-04T17:52:04.873304] [DEBUG] [LOCAL ] BYD: Trimmed 65664 records to 45124 (keeping 200 days of data) [2025-11-04T17:52:04.970940] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-03. [2025-11-04T17:52:04.978564] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-11-04T17:52:05.190258] [DEBUG] [LOCAL ] BYD: Primary data saved (45124 records). [2025-11-04T17:52:05.190480] [INFO ] [MAIN ] BYD: Full history saved. [2025-11-04T17:52:05.191591] [INFO ] [MAIN ] Update Progress: 380/2367 symbols completed. BYD took 29.0s. Estimated remaining: 4h 32m 4s. Rate: 7.3 symbols/min [2025-11-04T17:52:05.389800] [INFO ] [MAIN ] C: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:52:05.389970] [DEBUG] [LOCAL ] C: No existing data file found in primary storage. [2025-11-04T17:52:05.390035] [WARN ] [MAIN ] C: No existing data found, performing full refresh instead. [2025-11-04T17:52:05.390156] [INFO ] [MAIN ] C: Wiping existing primary data before full refresh. [2025-11-04T17:52:05.390200] [INFO ] [LOCAL ] C: Starting deletion of primary data. [2025-11-04T17:52:05.390242] [DEBUG] [LOCAL ] C: No primary data file found to delete. [2025-11-04T17:52:05.390278] [INFO ] [MAIN ] C: Starting full primary refresh. [2025-11-04T17:52:05.771080] [INFO ] [REMOTE ] C: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:52:05.767779] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-09. [2025-11-04T17:52:06.033239] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-11-04T17:52:06.567015] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-04. [2025-11-04T17:52:06.582692] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-11-04T17:52:07.367063] [INFO ] [REMOTE ] C: Fetching primary data for 2025-01. [2025-11-04T17:52:07.763707] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T17:52:08.167042] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-05. [2025-11-04T17:52:08.967036] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-10. [2025-11-04T17:52:09.210470] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-11-04T17:52:09.336521] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-11-04T17:52:09.600959] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-11-04T17:52:09.767054] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-11. [2025-11-04T17:52:10.482069] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T17:52:10.483246] [INFO ] [REMOTE ] BY: Full fetch complete (29971 records). [2025-11-04T17:52:10.483313] [DEBUG] [VALIDATOR ] BY: Validating primary data. [2025-11-04T17:52:10.572225] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-10. [2025-11-04T17:52:10.873067] [DEBUG] [VALIDATOR ] BY: All expected trading days are present. [2025-11-04T17:52:10.873141] [DEBUG] [VALIDATOR ] BY: Primary data validation passed. [2025-11-04T17:52:10.873547] [INFO ] [MAIN ] BY: Fetched primary data is valid. [2025-11-04T17:52:10.873584] [DEBUG] [LOCAL ] BY: Saving primary data. [2025-11-04T17:52:10.874287] [DEBUG] [LOCAL ] BY: Trimmed 29971 records to 21192 (keeping 200 days of data) [2025-11-04T17:52:10.998069] [DEBUG] [LOCAL ] BY: Primary data saved (21192 records). [2025-11-04T17:52:10.998189] [INFO ] [MAIN ] BY: Full history saved. [2025-11-04T17:52:10.998261] [INFO ] [MAIN ] Update Progress: 381/2367 symbols completed. BY took 36.1s. Estimated remaining: 4h 31m 43s. Rate: 7.3 symbols/min [2025-11-04T17:52:11.179430] [INFO ] [MAIN ] CAAP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:52:11.199707] [DEBUG] [LOCAL ] CAAP: Loaded 19900 existing records from primary storage. [2025-11-04T17:52:11.199944] [INFO ] [MAIN ] CAAP: Loaded 19900 existing records. [2025-11-04T17:52:11.372943] [INFO ] [REMOTE ] C: Fetching primary data for 2025-02. [2025-11-04T17:52:11.395259] [WARN ] [MAIN ] CAAP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:52:11.397471] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-11-04T17:52:11.396886] [INFO ] [MAIN ] CAAP: Wiping existing primary data before full refresh. [2025-11-04T17:52:11.400589] [INFO ] [LOCAL ] CAAP: Starting deletion of primary data. [2025-11-04T17:52:11.401482] [DEBUG] [LOCAL ] CAAP: Deleted primary data file. [2025-11-04T17:52:11.401553] [INFO ] [MAIN ] CAAP: Starting full primary refresh. [2025-11-04T17:52:11.758396] [INFO ] [REMOTE ] CAAP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:52:12.167042] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-11. [2025-11-04T17:52:12.777664] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T17:52:12.778534] [INFO ] [REMOTE ] BYM: Full fetch complete (17339 records). [2025-11-04T17:52:12.778601] [DEBUG] [VALIDATOR ] BYM: Validating primary data. [2025-11-04T17:52:12.970721] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-06. [2025-11-04T17:52:13.165672] [DEBUG] [VALIDATOR ] BYM: All expected trading days are present. [2025-11-04T17:52:13.165777] [DEBUG] [VALIDATOR ] BYM: Primary data validation passed. [2025-11-04T17:52:13.166150] [INFO ] [MAIN ] BYM: Fetched primary data is valid. [2025-11-04T17:52:13.166280] [DEBUG] [LOCAL ] BYM: Saving primary data. [2025-11-04T17:52:13.167232] [DEBUG] [LOCAL ] BYM: Trimmed 17339 records to 11313 (keeping 200 days of data) [2025-11-04T17:52:13.261004] [DEBUG] [LOCAL ] BYM: Primary data saved (11313 records). [2025-11-04T17:52:13.261347] [INFO ] [MAIN ] BYM: Full history saved. [2025-11-04T17:52:13.262913] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-11-04T17:52:13.261849] [INFO ] [MAIN ] Update Progress: 382/2367 symbols completed. BYM took 31.1s. Estimated remaining: 4h 31m 4s. Rate: 7.3 symbols/min [2025-11-04T17:52:13.443289] [INFO ] [MAIN ] CABO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:52:13.461613] [DEBUG] [LOCAL ] CABO: Loaded 16526 existing records from primary storage. [2025-11-04T17:52:13.461885] [INFO ] [MAIN ] CABO: Loaded 16526 existing records. [2025-11-04T17:52:13.636003] [WARN ] [MAIN ] CABO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:52:13.636215] [INFO ] [MAIN ] CABO: Wiping existing primary data before full refresh. [2025-11-04T17:52:13.636260] [INFO ] [LOCAL ] CABO: Starting deletion of primary data. [2025-11-04T17:52:13.636411] [DEBUG] [LOCAL ] CABO: Deleted primary data file. [2025-11-04T17:52:13.636448] [INFO ] [MAIN ] CABO: Starting full primary refresh. [2025-11-04T17:52:13.773966] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-01. [2025-11-04T17:52:13.999789] [INFO ] [REMOTE ] CABO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:52:14.016449] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T17:52:14.480522] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T17:52:14.567009] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-02. [2025-11-04T17:52:15.087369] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-11-04T17:52:15.367054] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-03. [2025-11-04T17:52:16.109166] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T17:52:16.167071] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-07. [2025-11-04T17:52:16.967027] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-01. [2025-11-04T17:52:17.164114] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T17:52:17.496441] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T17:52:17.767055] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-08. [2025-11-04T17:52:18.567087] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-04. [2025-11-04T17:52:18.724870] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-11-04T17:52:19.309131] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-11-04T17:52:19.367008] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-05. [2025-11-04T17:52:20.167013] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-09. [2025-11-04T17:52:20.247160] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T17:52:20.967098] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-02. [2025-11-04T17:52:21.270952] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T17:52:21.605583] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T17:52:21.767050] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-06. [2025-11-04T17:52:22.566683] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-11-04T17:52:22.570702] [INFO ] [REMOTE ] C: Fetching primary data for 2025-03. [2025-11-04T17:52:23.367043] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-07. [2025-11-04T17:52:24.009210] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-11-04T17:52:24.167061] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-03. [2025-11-04T17:52:24.387726] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-11-04T17:52:24.960313] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T17:52:24.966988] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-04. [2025-11-04T17:52:25.624933] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-11-04T17:52:25.767056] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-10. [2025-11-04T17:52:26.567103] [INFO ] [REMOTE ] C: Fetching primary data for 2025-04. [2025-11-04T17:52:26.727427] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-11-04T17:52:27.367042] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-11. [2025-11-04T17:52:28.107325] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T17:52:28.108708] [INFO ] [REMOTE ] BZH: Full fetch complete (47111 records). [2025-11-04T17:52:28.108781] [DEBUG] [VALIDATOR ] BZH: Validating primary data. [2025-11-04T17:52:28.173590] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-08. [2025-11-04T17:52:28.494193] [DEBUG] [VALIDATOR ] BZH: All expected trading days are present. [2025-11-04T17:52:28.494288] [DEBUG] [VALIDATOR ] BZH: Primary data validation passed. [2025-11-04T17:52:28.494961] [INFO ] [MAIN ] BZH: Fetched primary data is valid. [2025-11-04T17:52:28.495035] [DEBUG] [LOCAL ] BZH: Saving primary data. [2025-11-04T17:52:28.496177] [DEBUG] [LOCAL ] BZH: Trimmed 47111 records to 31381 (keeping 200 days of data) [2025-11-04T17:52:28.610057] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-11-04T17:52:28.707676] [DEBUG] [LOCAL ] BZH: Primary data saved (31381 records). [2025-11-04T17:52:28.707803] [INFO ] [MAIN ] BZH: Full history saved. [2025-11-04T17:52:28.707879] [INFO ] [MAIN ] Update Progress: 383/2367 symbols completed. BZH took 34.6s. Estimated remaining: 4h 31m 34s. Rate: 7.3 symbols/min [2025-11-04T17:52:28.877072] [INFO ] [MAIN ] CACI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:52:28.897330] [DEBUG] [LOCAL ] CACI: Loaded 21711 existing records from primary storage. [2025-11-04T17:52:28.897562] [INFO ] [MAIN ] CACI: Loaded 21711 existing records. [2025-11-04T17:52:28.971830] [INFO ] [REMOTE ] C: Fetching primary data for 2025-05. [2025-11-04T17:52:29.061979] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T17:52:29.103695] [WARN ] [MAIN ] CACI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:52:29.104787] [INFO ] [MAIN ] CACI: Wiping existing primary data before full refresh. [2025-11-04T17:52:29.104858] [INFO ] [LOCAL ] CACI: Starting deletion of primary data. [2025-11-04T17:52:29.105069] [DEBUG] [LOCAL ] CACI: Deleted primary data file. [2025-11-04T17:52:29.105135] [INFO ] [MAIN ] CACI: Starting full primary refresh. [2025-11-04T17:52:29.498087] [INFO ] [REMOTE ] CACI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:52:29.767062] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-05. [2025-11-04T17:52:30.567040] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-09. [2025-11-04T17:52:30.606045] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-11-04T17:52:30.662928] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T17:52:31.367037] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-01. [2025-11-04T17:52:31.442035] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-11-04T17:52:32.167035] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-10. [2025-11-04T17:52:32.505584] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T17:52:32.967032] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-02. [2025-11-04T17:52:33.251133] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-11-04T17:52:33.767046] [INFO ] [REMOTE ] C: Fetching primary data for 2025-06. [2025-11-04T17:52:33.990882] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T17:52:34.567120] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-06. [2025-11-04T17:52:35.368152] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-03. [2025-11-04T17:52:35.374666] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-11-04T17:52:35.376540] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T17:52:36.167082] [INFO ] [REMOTE ] C: Fetching primary data for 2025-07. [2025-11-04T17:52:36.375708] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T17:52:36.967051] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-04. [2025-11-04T17:52:37.767129] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-07. [2025-11-04T17:52:38.105208] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-11-04T17:52:38.203846] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T17:52:38.502902] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T17:52:38.566994] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-08. [2025-11-04T17:52:39.252127] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-11-04T17:52:39.367007] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-05. [2025-11-04T17:52:40.167022] [INFO ] [REMOTE ] C: Fetching primary data for 2025-08. [2025-11-04T17:52:40.366987] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-11-04T17:52:40.967072] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-11. [2025-11-04T17:52:41.647576] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T17:52:41.663392] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-11 (0.70 seconds). [2025-11-04T17:52:41.664325] [INFO ] [REMOTE ] CAAP: Full fetch complete (28238 records). [2025-11-04T17:52:41.664395] [DEBUG] [VALIDATOR ] CAAP: Validating primary data. [2025-11-04T17:52:41.664979] [FATAL] [VALIDATOR ] CAAP: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:52:41.665059] [FATAL] [MAIN ] CAAP: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:52:41.665101] [WARN ] [MAIN ] CAAP: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:52:41.665137] [INFO ] [LOCAL ] CAAP: Starting deletion of primary data. [2025-11-04T17:52:41.665189] [DEBUG] [LOCAL ] CAAP: No primary data file found to delete. [2025-11-04T17:52:41.665222] [INFO ] [LOCAL ] CAAP: Starting deletion of derivative data. [2025-11-04T17:52:41.665349] [DEBUG] [LOCAL ] CAAP: Deleted derivative data file. [2025-11-04T17:52:41.665413] [INFO ] [MAIN ] Update Progress: 384/2367 symbols completed. CAAP took 30.7s. Estimated remaining: 4h 31m 50s. Rate: 7.3 symbols/min [2025-11-04T17:52:41.768849] [INFO ] [REMOTE ] C: Fetching primary data for 2025-09. [2025-11-04T17:52:41.841994] [INFO ] [MAIN ] CADE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:52:41.886076] [DEBUG] [LOCAL ] CADE: Loaded 49844 existing records from primary storage. [2025-11-04T17:52:41.886368] [INFO ] [MAIN ] CADE: Loaded 49844 existing records. [2025-11-04T17:52:42.095851] [WARN ] [MAIN ] CADE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:52:42.096089] [INFO ] [MAIN ] CADE: Wiping existing primary data before full refresh. [2025-11-04T17:52:42.096158] [INFO ] [LOCAL ] CADE: Starting deletion of primary data. [2025-11-04T17:52:42.096349] [DEBUG] [LOCAL ] CADE: Deleted primary data file. [2025-11-04T17:52:42.096388] [INFO ] [MAIN ] CADE: Starting full primary refresh. [2025-11-04T17:52:42.461849] [INFO ] [REMOTE ] CADE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:52:42.567113] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-06. [2025-11-04T17:52:43.367031] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-09. [2025-11-04T17:52:43.426489] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-11-04T17:52:43.452944] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T17:52:43.885863] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-11-04T17:52:44.167069] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-10. [2025-11-04T17:52:44.747587] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-11-04T17:52:44.967056] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-07. [2025-11-04T17:52:45.680638] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-11-04T17:52:45.767013] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-08. [2025-11-04T17:52:46.509099] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-11-04T17:52:46.567097] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-01. [2025-11-04T17:52:47.367048] [INFO ] [REMOTE ] C: Fetching primary data for 2025-10. [2025-11-04T17:52:47.990104] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-11-04T17:52:48.167069] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-11. [2025-11-04T17:52:48.872673] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T17:52:48.873583] [INFO ] [REMOTE ] CABO: Full fetch complete (23119 records). [2025-11-04T17:52:48.873701] [DEBUG] [VALIDATOR ] CABO: Validating primary data. [2025-11-04T17:52:48.975354] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-02. [2025-11-04T17:52:49.259545] [DEBUG] [VALIDATOR ] CABO: All expected trading days are present. [2025-11-04T17:52:49.259708] [DEBUG] [VALIDATOR ] CABO: Primary data validation passed. [2025-11-04T17:52:49.260201] [INFO ] [MAIN ] CABO: Fetched primary data is valid. [2025-11-04T17:52:49.260281] [DEBUG] [LOCAL ] CABO: Saving primary data. [2025-11-04T17:52:49.261006] [DEBUG] [LOCAL ] CABO: Trimmed 23119 records to 16505 (keeping 200 days of data) [2025-11-04T17:52:49.360136] [DEBUG] [LOCAL ] CABO: Primary data saved (16505 records). [2025-11-04T17:52:49.360298] [INFO ] [MAIN ] CABO: Full history saved. [2025-11-04T17:52:49.360437] [INFO ] [MAIN ] Update Progress: 385/2367 symbols completed. CABO took 36.1s. Estimated remaining: 4h 31m 39s. Rate: 7.3 symbols/min [2025-11-04T17:52:49.547453] [INFO ] [MAIN ] CAE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:52:49.547842] [DEBUG] [LOCAL ] CAE: No existing data file found in primary storage. [2025-11-04T17:52:49.547939] [WARN ] [MAIN ] CAE: No existing data found, performing full refresh instead. [2025-11-04T17:52:49.548229] [INFO ] [MAIN ] CAE: Wiping existing primary data before full refresh. [2025-11-04T17:52:49.548321] [INFO ] [LOCAL ] CAE: Starting deletion of primary data. [2025-11-04T17:52:49.548388] [DEBUG] [LOCAL ] CAE: No primary data file found to delete. [2025-11-04T17:52:49.548511] [INFO ] [MAIN ] CAE: Starting full primary refresh. [2025-11-04T17:52:49.707843] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-10 (2.34 seconds). [2025-11-04T17:52:49.768126] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-09. [2025-11-04T17:52:49.954674] [INFO ] [REMOTE ] CAE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:52:50.201000] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T17:52:50.567009] [INFO ] [REMOTE ] C: Fetching primary data for 2025-11. [2025-11-04T17:52:50.682112] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-11-04T17:52:50.993339] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T17:52:50.995444] [INFO ] [REMOTE ] C: Full fetch complete (77919 records). [2025-11-04T17:52:50.995545] [DEBUG] [VALIDATOR ] C: Validating primary data. [2025-11-04T17:52:51.379513] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-03. [2025-11-04T17:52:51.404359] [DEBUG] [VALIDATOR ] C: All expected trading days are present. [2025-11-04T17:52:51.404418] [DEBUG] [VALIDATOR ] C: Primary data validation passed. [2025-11-04T17:52:51.405240] [INFO ] [MAIN ] C: Fetched primary data is valid. [2025-11-04T17:52:51.405391] [DEBUG] [LOCAL ] C: Saving primary data. [2025-11-04T17:52:51.406878] [DEBUG] [LOCAL ] C: Trimmed 77919 records to 53741 (keeping 200 days of data) [2025-11-04T17:52:51.731308] [DEBUG] [LOCAL ] C: Primary data saved (53741 records). [2025-11-04T17:52:51.731436] [INFO ] [MAIN ] C: Full history saved. [2025-11-04T17:52:51.731504] [INFO ] [MAIN ] Update Progress: 386/2367 symbols completed. C took 46.5s. Estimated remaining: 4h 31m 1s. Rate: 7.3 symbols/min [2025-11-04T17:52:51.897228] [INFO ] [MAIN ] CAF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:52:51.902886] [DEBUG] [LOCAL ] CAF: Loaded 3303 existing records from primary storage. [2025-11-04T17:52:51.903041] [INFO ] [MAIN ] CAF: Loaded 3303 existing records. [2025-11-04T17:52:52.076692] [WARN ] [MAIN ] CAF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:52:52.076903] [INFO ] [MAIN ] CAF: Wiping existing primary data before full refresh. [2025-11-04T17:52:52.076953] [INFO ] [LOCAL ] CAF: Starting deletion of primary data. [2025-11-04T17:52:52.077077] [DEBUG] [LOCAL ] CAF: Deleted primary data file. [2025-11-04T17:52:52.077114] [INFO ] [MAIN ] CAF: Starting full primary refresh. [2025-11-04T17:52:52.174826] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-01. [2025-11-04T17:52:52.431618] [INFO ] [REMOTE ] CAF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:52:52.719988] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T17:52:52.967066] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-01. [2025-11-04T17:52:53.351217] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T17:52:53.490917] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-11-04T17:52:53.767082] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-02. [2025-11-04T17:52:54.567035] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-10. [2025-11-04T17:52:54.794017] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T17:52:55.364585] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T17:52:55.368873] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-02. [2025-11-04T17:52:55.795445] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-11-04T17:52:56.167078] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-11. [2025-11-04T17:52:56.891558] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T17:52:56.893496] [INFO ] [REMOTE ] CACI: Full fetch complete (38360 records). [2025-11-04T17:52:56.893570] [DEBUG] [VALIDATOR ] CACI: Validating primary data. [2025-11-04T17:52:56.972232] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-04. [2025-11-04T17:52:57.289933] [DEBUG] [VALIDATOR ] CACI: All expected trading days are present. [2025-11-04T17:52:57.290013] [DEBUG] [VALIDATOR ] CACI: Primary data validation passed. [2025-11-04T17:52:57.290528] [INFO ] [MAIN ] CACI: Fetched primary data is valid. [2025-11-04T17:52:57.290568] [DEBUG] [LOCAL ] CACI: Saving primary data. [2025-11-04T17:52:57.291440] [DEBUG] [LOCAL ] CACI: Trimmed 38360 records to 21797 (keeping 200 days of data) [2025-11-04T17:52:57.419307] [DEBUG] [LOCAL ] CACI: Primary data saved (21797 records). [2025-11-04T17:52:57.419453] [INFO ] [MAIN ] CACI: Full history saved. [2025-11-04T17:52:57.419557] [INFO ] [MAIN ] Update Progress: 387/2367 symbols completed. CACI took 28.7s. Estimated remaining: 4h 30m 40s. Rate: 7.3 symbols/min [2025-11-04T17:52:57.599820] [INFO ] [MAIN ] CAG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:52:57.599987] [DEBUG] [LOCAL ] CAG: No existing data file found in primary storage. [2025-11-04T17:52:57.600023] [WARN ] [MAIN ] CAG: No existing data found, performing full refresh instead. [2025-11-04T17:52:57.600153] [INFO ] [MAIN ] CAG: Wiping existing primary data before full refresh. [2025-11-04T17:52:57.600194] [INFO ] [LOCAL ] CAG: Starting deletion of primary data. [2025-11-04T17:52:57.600233] [DEBUG] [LOCAL ] CAG: No primary data file found to delete. [2025-11-04T17:52:57.600265] [INFO ] [MAIN ] CAG: Starting full primary refresh. [2025-11-04T17:52:57.774856] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-03. [2025-11-04T17:52:57.965595] [INFO ] [REMOTE ] CAG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:52:58.221780] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-11-04T17:52:58.456829] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-11-04T17:52:58.567022] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-04. [2025-11-04T17:52:59.025543] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-11-04T17:52:59.367066] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-05. [2025-11-04T17:52:59.761956] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-11-04T17:53:00.167039] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-03. [2025-11-04T17:53:00.967085] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-01. [2025-11-04T17:53:01.301314] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-11-04T17:53:01.767042] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-06. [2025-11-04T17:53:02.149588] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-11-04T17:53:02.567101] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-05. [2025-11-04T17:53:02.604395] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-11-04T17:53:03.367074] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-07. [2025-11-04T17:53:03.791237] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-11-04T17:53:03.914519] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-11-04T17:53:04.167075] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-02. [2025-11-04T17:53:04.967034] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-06. [2025-11-04T17:53:05.419663] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-11-04T17:53:05.767033] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-08. [2025-11-04T17:53:06.082940] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-11-04T17:53:06.222412] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-11-04T17:53:06.567088] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-07. [2025-11-04T17:53:07.367026] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-04. [2025-11-04T17:53:07.867961] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:53:08.167032] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-08. [2025-11-04T17:53:08.573323] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T17:53:08.967022] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-05. [2025-11-04T17:53:09.693062] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T17:53:09.767001] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-09. [2025-11-04T17:53:10.071490] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-11-04T17:53:10.567056] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-09. [2025-11-04T17:53:10.892205] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-09 (0.33 seconds). [2025-11-04T17:53:11.053574] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T17:53:11.367063] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-06. [2025-11-04T17:53:12.167094] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-10. [2025-11-04T17:53:12.450368] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T17:53:12.571600] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-11-04T17:53:12.967040] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-03. [2025-11-04T17:53:13.767041] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-10. [2025-11-04T17:53:14.567029] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-07. [2025-11-04T17:53:14.655729] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-11-04T17:53:15.149819] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T17:53:15.367026] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-11. [2025-11-04T17:53:15.590322] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T17:53:16.167040] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-08. [2025-11-04T17:53:16.486094] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T17:53:16.488098] [INFO ] [REMOTE ] CADE: Full fetch complete (71034 records). [2025-11-04T17:53:16.488200] [DEBUG] [VALIDATOR ] CADE: Validating primary data. [2025-11-04T17:53:16.872959] [DEBUG] [VALIDATOR ] CADE: All expected trading days are present. [2025-11-04T17:53:16.873038] [DEBUG] [VALIDATOR ] CADE: Primary data validation passed. [2025-11-04T17:53:16.873785] [INFO ] [MAIN ] CADE: Fetched primary data is valid. [2025-11-04T17:53:16.873848] [DEBUG] [LOCAL ] CADE: Saving primary data. [2025-11-04T17:53:16.875181] [DEBUG] [LOCAL ] CADE: Trimmed 71034 records to 49887 (keeping 200 days of data) [2025-11-04T17:53:16.973954] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-11. [2025-11-04T17:53:17.181900] [DEBUG] [LOCAL ] CADE: Primary data saved (49887 records). [2025-11-04T17:53:17.182466] [INFO ] [MAIN ] CADE: Full history saved. [2025-11-04T17:53:17.182577] [INFO ] [MAIN ] Update Progress: 388/2367 symbols completed. CADE took 35.5s. Estimated remaining: 4h 31m 30s. Rate: 7.3 symbols/min [2025-11-04T17:53:17.377527] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T17:53:17.379907] [INFO ] [MAIN ] CAH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:53:17.434715] [DEBUG] [LOCAL ] CAH: Loaded 51368 existing records from primary storage. [2025-11-04T17:53:17.435018] [INFO ] [MAIN ] CAH: Loaded 51368 existing records. [2025-11-04T17:53:17.536092] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-11 (0.56 seconds). [2025-11-04T17:53:17.547605] [INFO ] [REMOTE ] CAF: Full fetch complete (5549 records). [2025-11-04T17:53:17.557962] [DEBUG] [VALIDATOR ] CAF: Validating primary data. [2025-11-04T17:53:17.711625] [WARN ] [MAIN ] CAH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:53:17.722069] [INFO ] [MAIN ] CAH: Wiping existing primary data before full refresh. [2025-11-04T17:53:17.722443] [INFO ] [LOCAL ] CAH: Starting deletion of primary data. [2025-11-04T17:53:17.733850] [DEBUG] [LOCAL ] CAH: Deleted primary data file. [2025-11-04T17:53:17.749456] [INFO ] [MAIN ] CAH: Starting full primary refresh. [2025-11-04T17:53:17.809274] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-04. [2025-11-04T17:53:18.243359] [DEBUG] [VALIDATOR ] CAF: All expected trading days are present. [2025-11-04T17:53:18.260010] [DEBUG] [VALIDATOR ] CAF: Primary data validation passed. [2025-11-04T17:53:18.281918] [INFO ] [MAIN ] CAF: Fetched primary data is valid. [2025-11-04T17:53:18.284322] [DEBUG] [LOCAL ] CAF: Saving primary data. [2025-11-04T17:53:18.290457] [DEBUG] [LOCAL ] CAF: Trimmed 5549 records to 3306 (keeping 200 days of data) [2025-11-04T17:53:18.379213] [INFO ] [REMOTE ] CAH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:53:18.383517] [DEBUG] [LOCAL ] CAF: Primary data saved (3306 records). [2025-11-04T17:53:18.383581] [INFO ] [MAIN ] CAF: Full history saved. [2025-11-04T17:53:18.383668] [INFO ] [MAIN ] Update Progress: 389/2367 symbols completed. CAF took 26.7s. Estimated remaining: 4h 30m 46s. Rate: 7.3 symbols/min [2025-11-04T17:53:18.553047] [INFO ] [MAIN ] CAL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:53:18.575013] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-09. [2025-11-04T17:53:18.593958] [DEBUG] [LOCAL ] CAL: Loaded 42490 existing records from primary storage. [2025-11-04T17:53:18.594240] [INFO ] [MAIN ] CAL: Loaded 42490 existing records. [2025-11-04T17:53:18.781761] [WARN ] [MAIN ] CAL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:53:18.781996] [INFO ] [MAIN ] CAL: Wiping existing primary data before full refresh. [2025-11-04T17:53:18.782044] [INFO ] [LOCAL ] CAL: Starting deletion of primary data. [2025-11-04T17:53:18.782217] [DEBUG] [LOCAL ] CAL: Deleted primary data file. [2025-11-04T17:53:18.782255] [INFO ] [MAIN ] CAL: Starting full primary refresh. [2025-11-04T17:53:19.131744] [INFO ] [REMOTE ] CAL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:53:19.367013] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-01. [2025-11-04T17:53:19.480628] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-11-04T17:53:19.716681] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T17:53:20.167102] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-10. [2025-11-04T17:53:20.549778] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T17:53:20.967095] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-01. [2025-11-04T17:53:21.186223] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T17:53:21.767092] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-11. [2025-11-04T17:53:22.495541] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-11-04T17:53:22.567004] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-02. [2025-11-04T17:53:22.733037] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T17:53:22.734582] [INFO ] [REMOTE ] CAE: Full fetch complete (56640 records). [2025-11-04T17:53:22.734703] [DEBUG] [VALIDATOR ] CAE: Validating primary data. [2025-11-04T17:53:23.134457] [DEBUG] [VALIDATOR ] CAE: All expected trading days are present. [2025-11-04T17:53:23.134533] [DEBUG] [VALIDATOR ] CAE: Primary data validation passed. [2025-11-04T17:53:23.135162] [INFO ] [MAIN ] CAE: Fetched primary data is valid. [2025-11-04T17:53:23.135224] [DEBUG] [LOCAL ] CAE: Saving primary data. [2025-11-04T17:53:23.136264] [DEBUG] [LOCAL ] CAE: Trimmed 56640 records to 38678 (keeping 200 days of data) [2025-11-04T17:53:23.354584] [DEBUG] [LOCAL ] CAE: Primary data saved (38678 records). [2025-11-04T17:53:23.354793] [INFO ] [MAIN ] CAE: Full history saved. [2025-11-04T17:53:23.354882] [INFO ] [MAIN ] Update Progress: 390/2367 symbols completed. CAE took 34.0s. Estimated remaining: 4h 30m 22s. Rate: 7.3 symbols/min [2025-11-04T17:53:23.376253] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-02. [2025-11-04T17:53:23.528375] [INFO ] [MAIN ] CALX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:53:23.561987] [DEBUG] [LOCAL ] CALX: Loaded 37093 existing records from primary storage. [2025-11-04T17:53:23.562257] [INFO ] [MAIN ] CALX: Loaded 37093 existing records. [2025-11-04T17:53:23.745140] [WARN ] [MAIN ] CALX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:53:23.745348] [INFO ] [MAIN ] CALX: Wiping existing primary data before full refresh. [2025-11-04T17:53:23.745402] [INFO ] [LOCAL ] CALX: Starting deletion of primary data. [2025-11-04T17:53:23.745720] [DEBUG] [LOCAL ] CALX: Deleted primary data file. [2025-11-04T17:53:23.745777] [INFO ] [MAIN ] CALX: Starting full primary refresh. [2025-11-04T17:53:24.126101] [INFO ] [REMOTE ] CALX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:53:24.152953] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-11-04T17:53:24.166997] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-01. [2025-11-04T17:53:24.419433] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-11-04T17:53:24.967038] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-03. [2025-11-04T17:53:25.113068] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T17:53:25.767162] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-05. [2025-11-04T17:53:26.344979] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T17:53:26.567061] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-02. [2025-11-04T17:53:27.264210] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-11-04T17:53:27.367013] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-06. [2025-11-04T17:53:27.551117] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-11-04T17:53:28.167046] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-03. [2025-11-04T17:53:28.875182] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-11-04T17:53:28.967005] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-07. [2025-11-04T17:53:29.473304] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T17:53:29.767033] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-03. [2025-11-04T17:53:30.477719] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T17:53:30.567027] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-04. [2025-11-04T17:53:30.953816] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T17:53:31.367028] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-04. [2025-11-04T17:53:31.914571] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T17:53:32.167041] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-08. [2025-11-04T17:53:32.937815] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-11-04T17:53:32.967009] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-05. [2025-11-04T17:53:33.767021] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-04. [2025-11-04T17:53:33.881825] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-11-04T17:53:34.567117] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-05. [2025-11-04T17:53:34.875121] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-11-04T17:53:35.367059] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-05. [2025-11-04T17:53:35.588980] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T17:53:36.110885] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-05 (3.14 seconds). [2025-11-04T17:53:36.167034] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-06. [2025-11-04T17:53:36.464131] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-11-04T17:53:36.967090] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-09. [2025-11-04T17:53:37.476683] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T17:53:37.767109] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-06. [2025-11-04T17:53:38.550701] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-11-04T17:53:38.572101] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-10. [2025-11-04T17:53:38.583066] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-11-04T17:53:39.367027] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-07. [2025-11-04T17:53:40.167052] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-07. [2025-11-04T17:53:40.215159] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-11-04T17:53:40.663202] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:53:40.967107] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-11. [2025-11-04T17:53:41.332851] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T17:53:41.506711] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T17:53:41.509396] [INFO ] [REMOTE ] CAG: Full fetch complete (77829 records). [2025-11-04T17:53:41.509554] [DEBUG] [VALIDATOR ] CAG: Validating primary data. [2025-11-04T17:53:41.772260] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-08. [2025-11-04T17:53:41.965968] [DEBUG] [VALIDATOR ] CAG: All expected trading days are present. [2025-11-04T17:53:41.966067] [DEBUG] [VALIDATOR ] CAG: Primary data validation passed. [2025-11-04T17:53:41.967082] [INFO ] [MAIN ] CAG: Fetched primary data is valid. [2025-11-04T17:53:41.967153] [DEBUG] [LOCAL ] CAG: Saving primary data. [2025-11-04T17:53:41.968812] [DEBUG] [LOCAL ] CAG: Trimmed 77829 records to 53668 (keeping 200 days of data) [2025-11-04T17:53:42.327373] [DEBUG] [LOCAL ] CAG: Primary data saved (53668 records). [2025-11-04T17:53:42.327501] [INFO ] [MAIN ] CAG: Full history saved. [2025-11-04T17:53:42.327570] [INFO ] [MAIN ] Update Progress: 391/2367 symbols completed. CAG took 44.9s. Estimated remaining: 4h 31m 8s. Rate: 7.3 symbols/min [2025-11-04T17:53:42.511305] [INFO ] [MAIN ] CANG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:53:42.537295] [DEBUG] [LOCAL ] CANG: Loaded 27445 existing records from primary storage. [2025-11-04T17:53:42.537583] [INFO ] [MAIN ] CANG: Loaded 27445 existing records. [2025-11-04T17:53:42.572275] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-06. [2025-11-04T17:53:42.719760] [WARN ] [MAIN ] CANG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:53:42.719987] [INFO ] [MAIN ] CANG: Wiping existing primary data before full refresh. [2025-11-04T17:53:42.720034] [INFO ] [LOCAL ] CANG: Starting deletion of primary data. [2025-11-04T17:53:42.720197] [DEBUG] [LOCAL ] CANG: Deleted primary data file. [2025-11-04T17:53:42.720293] [INFO ] [MAIN ] CANG: Starting full primary refresh. [2025-11-04T17:53:43.082578] [INFO ] [REMOTE ] CANG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:53:43.106748] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-11-04T17:53:43.367050] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-01. [2025-11-04T17:53:43.961762] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T17:53:44.167032] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-07. [2025-11-04T17:53:44.338139] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-11-04T17:53:44.967055] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-09. [2025-11-04T17:53:45.243261] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T17:53:45.767018] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-08. [2025-11-04T17:53:45.908462] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-11-04T17:53:46.567007] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-02. [2025-11-04T17:53:46.849475] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-11-04T17:53:47.367079] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-08. [2025-11-04T17:53:47.436247] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T17:53:48.167117] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-09. [2025-11-04T17:53:48.915101] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-11-04T17:53:48.967019] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-09. [2025-11-04T17:53:49.696356] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-11-04T17:53:49.767075] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-10. [2025-11-04T17:53:50.207396] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-11-04T17:53:50.567045] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-10. [2025-11-04T17:53:51.005237] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-11-04T17:53:51.367032] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-11. [2025-11-04T17:53:51.812693] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T17:53:52.167053] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-10. [2025-11-04T17:53:52.389779] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T17:53:52.391541] [INFO ] [REMOTE ] CALX: Full fetch complete (54697 records). [2025-11-04T17:53:52.391618] [DEBUG] [VALIDATOR ] CALX: Validating primary data. [2025-11-04T17:53:52.783892] [DEBUG] [VALIDATOR ] CALX: All expected trading days are present. [2025-11-04T17:53:52.783974] [DEBUG] [VALIDATOR ] CALX: Primary data validation passed. [2025-11-04T17:53:52.784616] [INFO ] [MAIN ] CALX: Fetched primary data is valid. [2025-11-04T17:53:52.784686] [DEBUG] [LOCAL ] CALX: Saving primary data. [2025-11-04T17:53:52.785755] [DEBUG] [LOCAL ] CALX: Trimmed 54697 records to 37142 (keeping 200 days of data) [2025-11-04T17:53:52.969829] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-11. [2025-11-04T17:53:52.999948] [DEBUG] [LOCAL ] CALX: Primary data saved (37142 records). [2025-11-04T17:53:53.000084] [INFO ] [MAIN ] CALX: Full history saved. [2025-11-04T17:53:53.000164] [INFO ] [MAIN ] Update Progress: 392/2367 symbols completed. CALX took 29.6s. Estimated remaining: 4h 31m 12s. Rate: 7.3 symbols/min [2025-11-04T17:53:53.186128] [INFO ] [MAIN ] CAPL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:53:53.193639] [DEBUG] [LOCAL ] CAPL: Loaded 5344 existing records from primary storage. [2025-11-04T17:53:53.193871] [INFO ] [MAIN ] CAPL: Loaded 5344 existing records. [2025-11-04T17:53:53.382819] [WARN ] [MAIN ] CAPL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:53:53.383009] [INFO ] [MAIN ] CAPL: Wiping existing primary data before full refresh. [2025-11-04T17:53:53.383056] [INFO ] [LOCAL ] CAPL: Starting deletion of primary data. [2025-11-04T17:53:53.383193] [DEBUG] [LOCAL ] CAPL: Deleted primary data file. [2025-11-04T17:53:53.383232] [INFO ] [MAIN ] CAPL: Starting full primary refresh. [2025-11-04T17:53:53.747872] [INFO ] [REMOTE ] CAPL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:53:53.773262] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-01. [2025-11-04T17:53:53.787631] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-11-04T17:53:54.166590] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T17:53:54.168728] [INFO ] [REMOTE ] CAL: Full fetch complete (62203 records). [2025-11-04T17:53:54.168881] [DEBUG] [VALIDATOR ] CAL: Validating primary data. [2025-11-04T17:53:54.458821] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T17:53:54.576197] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-02. [2025-11-04T17:53:54.579542] [DEBUG] [VALIDATOR ] CAL: All expected trading days are present. [2025-11-04T17:53:54.579805] [DEBUG] [VALIDATOR ] CAL: Primary data validation passed. [2025-11-04T17:53:54.581592] [INFO ] [MAIN ] CAL: Fetched primary data is valid. [2025-11-04T17:53:54.581690] [DEBUG] [LOCAL ] CAL: Saving primary data. [2025-11-04T17:53:54.583041] [DEBUG] [LOCAL ] CAL: Trimmed 62203 records to 42493 (keeping 200 days of data) [2025-11-04T17:53:54.831248] [DEBUG] [LOCAL ] CAL: Primary data saved (42493 records). [2025-11-04T17:53:54.831370] [INFO ] [MAIN ] CAL: Full history saved. [2025-11-04T17:53:54.831916] [INFO ] [MAIN ] Update Progress: 393/2367 symbols completed. CAL took 36.4s. Estimated remaining: 4h 30m 32s. Rate: 7.3 symbols/min [2025-11-04T17:53:54.998889] [INFO ] [MAIN ] CARR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:53:55.006580] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-11-04T17:53:55.056169] [DEBUG] [LOCAL ] CARR: Loaded 53604 existing records from primary storage. [2025-11-04T17:53:55.056491] [INFO ] [MAIN ] CARR: Loaded 53604 existing records. [2025-11-04T17:53:55.240092] [WARN ] [MAIN ] CARR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:53:55.240336] [INFO ] [MAIN ] CARR: Wiping existing primary data before full refresh. [2025-11-04T17:53:55.240386] [INFO ] [LOCAL ] CARR: Starting deletion of primary data. [2025-11-04T17:53:55.240590] [DEBUG] [LOCAL ] CARR: Deleted primary data file. [2025-11-04T17:53:55.240688] [INFO ] [MAIN ] CARR: Starting full primary refresh. [2025-11-04T17:53:55.380886] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-03. [2025-11-04T17:53:55.607858] [INFO ] [REMOTE ] CARR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:53:56.102784] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-11-04T17:53:56.167009] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-04. [2025-11-04T17:53:56.643849] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-11-04T17:53:56.967114] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-03. [2025-11-04T17:53:57.767047] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-01. [2025-11-04T17:53:57.835144] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-11-04T17:53:58.567066] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-05. [2025-11-04T17:53:58.999061] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T17:53:59.293683] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-11-04T17:53:59.367016] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-11. [2025-11-04T17:53:59.699871] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-11 (0.33 seconds). [2025-11-04T17:53:59.701855] [INFO ] [REMOTE ] CAH: Full fetch complete (74923 records). [2025-11-04T17:53:59.701955] [DEBUG] [VALIDATOR ] CAH: Validating primary data. [2025-11-04T17:54:00.115819] [DEBUG] [VALIDATOR ] CAH: All expected trading days are present. [2025-11-04T17:54:00.115899] [DEBUG] [VALIDATOR ] CAH: Primary data validation passed. [2025-11-04T17:54:00.116670] [INFO ] [MAIN ] CAH: Fetched primary data is valid. [2025-11-04T17:54:00.116730] [DEBUG] [LOCAL ] CAH: Saving primary data. [2025-11-04T17:54:00.118071] [DEBUG] [LOCAL ] CAH: Trimmed 74923 records to 51380 (keeping 200 days of data) [2025-11-04T17:54:00.178108] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-06. [2025-11-04T17:54:00.474984] [DEBUG] [LOCAL ] CAH: Primary data saved (51380 records). [2025-11-04T17:54:00.475142] [INFO ] [MAIN ] CAH: Full history saved. [2025-11-04T17:54:00.475249] [INFO ] [MAIN ] Update Progress: 394/2367 symbols completed. CAH took 43.3s. Estimated remaining: 4h 30m 11s. Rate: 7.3 symbols/min [2025-11-04T17:54:00.648072] [INFO ] [MAIN ] CARS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:54:00.684165] [DEBUG] [LOCAL ] CARS: Loaded 39327 existing records from primary storage. [2025-11-04T17:54:00.684447] [INFO ] [MAIN ] CARS: Loaded 39327 existing records. [2025-11-04T17:54:00.771610] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-11-04T17:54:00.873949] [WARN ] [MAIN ] CARS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:54:00.874173] [INFO ] [MAIN ] CARS: Wiping existing primary data before full refresh. [2025-11-04T17:54:00.874221] [INFO ] [LOCAL ] CARS: Starting deletion of primary data. [2025-11-04T17:54:00.874402] [DEBUG] [LOCAL ] CARS: Deleted primary data file. [2025-11-04T17:54:00.874439] [INFO ] [MAIN ] CARS: Starting full primary refresh. [2025-11-04T17:54:00.972258] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-07. [2025-11-04T17:54:01.235007] [INFO ] [REMOTE ] CARS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:54:01.620462] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-11-04T17:54:01.767103] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-02. [2025-11-04T17:54:02.567058] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-08. [2025-11-04T17:54:03.027723] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T17:54:03.079196] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-11-04T17:54:03.367034] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-03. [2025-11-04T17:54:04.167065] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-09. [2025-11-04T17:54:04.649743] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-11-04T17:54:04.849984] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-11-04T17:54:04.967096] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-10. [2025-11-04T17:54:05.446227] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-11-04T17:54:05.767046] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-11. [2025-11-04T17:54:06.195600] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T17:54:06.196604] [INFO ] [REMOTE ] CAPL: Full fetch complete (8742 records). [2025-11-04T17:54:06.196703] [DEBUG] [VALIDATOR ] CAPL: Validating primary data. [2025-11-04T17:54:06.578576] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-01. [2025-11-04T17:54:06.587588] [DEBUG] [VALIDATOR ] CAPL: All expected trading days are present. [2025-11-04T17:54:06.587709] [DEBUG] [VALIDATOR ] CAPL: Primary data validation passed. [2025-11-04T17:54:06.587980] [INFO ] [MAIN ] CAPL: Fetched primary data is valid. [2025-11-04T17:54:06.588025] [DEBUG] [LOCAL ] CAPL: Saving primary data. [2025-11-04T17:54:06.588526] [DEBUG] [LOCAL ] CAPL: Trimmed 8742 records to 5332 (keeping 200 days of data) [2025-11-04T17:54:06.620807] [DEBUG] [LOCAL ] CAPL: Primary data saved (5332 records). [2025-11-04T17:54:06.620881] [INFO ] [MAIN ] CAPL: Full history saved. [2025-11-04T17:54:06.620949] [INFO ] [MAIN ] Update Progress: 395/2367 symbols completed. CAPL took 13.6s. Estimated remaining: 4h 29m 52s. Rate: 7.3 symbols/min [2025-11-04T17:54:06.790238] [INFO ] [MAIN ] CAT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:54:06.838889] [DEBUG] [LOCAL ] CAT: Loaded 52904 existing records from primary storage. [2025-11-04T17:54:06.839200] [INFO ] [MAIN ] CAT: Loaded 52904 existing records. [2025-11-04T17:54:07.043067] [WARN ] [MAIN ] CAT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:54:07.043308] [INFO ] [MAIN ] CAT: Wiping existing primary data before full refresh. [2025-11-04T17:54:07.043357] [INFO ] [LOCAL ] CAT: Starting deletion of primary data. [2025-11-04T17:54:07.043543] [DEBUG] [LOCAL ] CAT: Deleted primary data file. [2025-11-04T17:54:07.043581] [INFO ] [MAIN ] CAT: Starting full primary refresh. [2025-11-04T17:54:07.379835] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-04. [2025-11-04T17:54:07.423750] [INFO ] [REMOTE ] CAT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:54:07.656423] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T17:54:08.167015] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-04. [2025-11-04T17:54:08.237324] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T17:54:08.967041] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-05. [2025-11-04T17:54:09.767069] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-02. [2025-11-04T17:54:09.891920] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T17:54:09.926093] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-11-04T17:54:10.567012] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-05. [2025-11-04T17:54:10.704849] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-11-04T17:54:11.367037] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-03. [2025-11-04T17:54:11.847807] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T17:54:12.167031] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-06. [2025-11-04T17:54:12.353291] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T17:54:12.967008] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-06. [2025-11-04T17:54:13.095725] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T17:54:13.767018] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-07. [2025-11-04T17:54:14.504550] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-11-04T17:54:14.567027] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-04. [2025-11-04T17:54:14.835479] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T17:54:15.367041] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-07. [2025-11-04T17:54:15.769852] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T17:54:16.167079] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-08. [2025-11-04T17:54:16.641781] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T17:54:16.967012] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-08. [2025-11-04T17:54:17.008540] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-11-04T17:54:17.767083] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-05. [2025-11-04T17:54:18.513986] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-11-04T17:54:18.567018] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-01. [2025-11-04T17:54:18.991924] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T17:54:19.367126] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-09. [2025-11-04T17:54:20.100064] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-11-04T17:54:20.167005] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-02. [2025-11-04T17:54:20.436311] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-11-04T17:54:20.967079] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-06. [2025-11-04T17:54:21.767034] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-10. [2025-11-04T17:54:21.875674] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-11-04T17:54:21.990007] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T17:54:22.567058] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-09. [2025-11-04T17:54:22.841909] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T17:54:23.367073] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-03. [2025-11-04T17:54:24.167063] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-11. [2025-11-04T17:54:24.329059] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-11-04T17:54:24.923596] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T17:54:24.967012] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-04. [2025-11-04T17:54:25.057429] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T17:54:25.058900] [INFO ] [REMOTE ] CANG: Full fetch complete (39278 records). [2025-11-04T17:54:25.058972] [DEBUG] [VALIDATOR ] CANG: Validating primary data. [2025-11-04T17:54:25.430622] [DEBUG] [VALIDATOR ] CANG: All expected trading days are present. [2025-11-04T17:54:25.430724] [DEBUG] [VALIDATOR ] CANG: Primary data validation passed. [2025-11-04T17:54:25.431322] [INFO ] [MAIN ] CANG: Fetched primary data is valid. [2025-11-04T17:54:25.431385] [DEBUG] [LOCAL ] CANG: Saving primary data. [2025-11-04T17:54:25.432218] [DEBUG] [LOCAL ] CANG: Trimmed 39278 records to 27593 (keeping 200 days of data) [2025-11-04T17:54:25.595346] [DEBUG] [LOCAL ] CANG: Primary data saved (27593 records). [2025-11-04T17:54:25.595507] [INFO ] [MAIN ] CANG: Full history saved. [2025-11-04T17:54:25.595606] [INFO ] [MAIN ] Update Progress: 396/2367 symbols completed. CANG took 43.3s. Estimated remaining: 4h 30m 37s. Rate: 7.3 symbols/min [2025-11-04T17:54:25.762833] [INFO ] [MAIN ] CATO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:54:25.767494] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-07. [2025-11-04T17:54:25.772122] [DEBUG] [LOCAL ] CATO: Loaded 6554 existing records from primary storage. [2025-11-04T17:54:25.772274] [INFO ] [MAIN ] CATO: Loaded 6554 existing records. [2025-11-04T17:54:25.946458] [WARN ] [MAIN ] CATO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:54:25.946689] [INFO ] [MAIN ] CATO: Wiping existing primary data before full refresh. [2025-11-04T17:54:25.946746] [INFO ] [LOCAL ] CATO: Starting deletion of primary data. [2025-11-04T17:54:25.946893] [DEBUG] [LOCAL ] CATO: Deleted primary data file. [2025-11-04T17:54:25.946947] [INFO ] [MAIN ] CATO: Starting full primary refresh. [2025-11-04T17:54:26.331739] [INFO ] [REMOTE ] CATO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:54:26.567064] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-01. [2025-11-04T17:54:26.954585] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-11-04T17:54:27.010385] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-11-04T17:54:27.070627] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-11-04T17:54:27.367057] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-02. [2025-11-04T17:54:27.874318] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T17:54:28.167048] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-10. [2025-11-04T17:54:28.967063] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-08. [2025-11-04T17:54:29.644530] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-11-04T17:54:29.767048] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-11. [2025-11-04T17:54:30.174377] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T17:54:30.567023] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-09. [2025-11-04T17:54:31.206495] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-11 (1.44 seconds). [2025-11-04T17:54:31.208545] [INFO ] [REMOTE ] CARR: Full fetch complete (77771 records). [2025-11-04T17:54:31.208680] [DEBUG] [VALIDATOR ] CARR: Validating primary data. [2025-11-04T17:54:31.382559] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-05. [2025-11-04T17:54:31.601725] [DEBUG] [VALIDATOR ] CARR: All expected trading days are present. [2025-11-04T17:54:31.601827] [DEBUG] [VALIDATOR ] CARR: Primary data validation passed. [2025-11-04T17:54:31.602747] [INFO ] [MAIN ] CARR: Fetched primary data is valid. [2025-11-04T17:54:31.602857] [DEBUG] [LOCAL ] CARR: Saving primary data. [2025-11-04T17:54:31.623326] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T17:54:31.630979] [DEBUG] [LOCAL ] CARR: Trimmed 77771 records to 53608 (keeping 200 days of data) [2025-11-04T17:54:31.953332] [DEBUG] [LOCAL ] CARR: Primary data saved (53608 records). [2025-11-04T17:54:31.953454] [INFO ] [MAIN ] CARR: Full history saved. [2025-11-04T17:54:31.954040] [INFO ] [MAIN ] Update Progress: 397/2367 symbols completed. CARR took 37.1s. Estimated remaining: 4h 30m 20s. Rate: 7.3 symbols/min [2025-11-04T17:54:32.167062] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-03. [2025-11-04T17:54:32.665772] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-11-04T17:54:32.973141] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-10. [2025-11-04T17:54:32.986497] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-11-04T17:54:33.767109] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-04. [2025-11-04T17:54:34.125072] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-11-04T17:54:34.219494] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-11-04T17:54:34.567032] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-05. [2025-11-04T17:54:34.997497] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T17:54:35.367078] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-06. [2025-11-04T17:54:35.814541] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-11-04T17:54:36.167112] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-11. [2025-11-04T17:54:36.967047] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-07. [2025-11-04T17:54:37.340288] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-11 (1.17 seconds). [2025-11-04T17:54:37.342021] [INFO ] [REMOTE ] CARS: Full fetch complete (57065 records). [2025-11-04T17:54:37.342114] [DEBUG] [VALIDATOR ] CARS: Validating primary data. [2025-11-04T17:54:37.621678] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-11-04T17:54:37.741597] [DEBUG] [VALIDATOR ] CARS: All expected trading days are present. [2025-11-04T17:54:37.741711] [DEBUG] [VALIDATOR ] CARS: Primary data validation passed. [2025-11-04T17:54:37.742427] [INFO ] [MAIN ] CARS: Fetched primary data is valid. [2025-11-04T17:54:37.742469] [DEBUG] [LOCAL ] CARS: Saving primary data. [2025-11-04T17:54:37.743622] [DEBUG] [LOCAL ] CARS: Trimmed 57065 records to 39412 (keeping 200 days of data) [2025-11-04T17:54:37.790023] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-06. [2025-11-04T17:54:37.966341] [DEBUG] [LOCAL ] CARS: Primary data saved (39412 records). [2025-11-04T17:54:37.966480] [INFO ] [MAIN ] CARS: Full history saved. [2025-11-04T17:54:37.966561] [INFO ] [MAIN ] Update Progress: 398/2367 symbols completed. CARS took 37.5s. Estimated remaining: 4h 30m 1s. Rate: 7.3 symbols/min [2025-11-04T17:54:38.567035] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-08. [2025-11-04T17:54:39.055694] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-11-04T17:54:39.331297] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-11-04T17:54:39.367000] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-07. [2025-11-04T17:54:40.167039] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-09. [2025-11-04T17:54:40.653250] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-11-04T17:54:40.967045] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-10. [2025-11-04T17:54:41.114263] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-11-04T17:54:41.461992] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-11-04T17:54:41.767089] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-11. [2025-11-04T17:54:42.197107] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T17:54:42.197900] [INFO ] [REMOTE ] CATO: Full fetch complete (10578 records). [2025-11-04T17:54:42.197975] [DEBUG] [VALIDATOR ] CATO: Validating primary data. [2025-11-04T17:54:42.553130] [DEBUG] [VALIDATOR ] CATO: All expected trading days are present. [2025-11-04T17:54:42.553205] [DEBUG] [VALIDATOR ] CATO: Primary data validation passed. [2025-11-04T17:54:42.553443] [INFO ] [MAIN ] CATO: Fetched primary data is valid. [2025-11-04T17:54:42.553481] [DEBUG] [LOCAL ] CATO: Saving primary data. [2025-11-04T17:54:42.553918] [DEBUG] [LOCAL ] CATO: Trimmed 10578 records to 6534 (keeping 200 days of data) [2025-11-04T17:54:42.567793] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-08. [2025-11-04T17:54:42.591265] [DEBUG] [LOCAL ] CATO: Primary data saved (6534 records). [2025-11-04T17:54:42.591358] [INFO ] [MAIN ] CATO: Full history saved. [2025-11-04T17:54:42.591443] [INFO ] [MAIN ] Update Progress: 399/2367 symbols completed. CATO took 17.0s. Estimated remaining: 4h 29m 35s. Rate: 7.3 symbols/min [2025-11-04T17:54:44.256406] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-11-04T17:54:44.269320] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-09. [2025-11-04T17:54:45.764709] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T17:54:45.776020] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-10. [2025-11-04T17:54:47.763994] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-11-04T17:54:47.782122] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-11. [2025-11-04T17:54:48.067761] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-11 (0.29 seconds). [2025-11-04T17:54:48.069219] [INFO ] [REMOTE ] CAT: Full fetch complete (76488 records). [2025-11-04T17:54:48.069316] [DEBUG] [VALIDATOR ] CAT: Validating primary data. [2025-11-04T17:54:48.070202] [FATAL] [VALIDATOR ] CAT: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T17:54:48.070287] [FATAL] [MAIN ] CAT: Fetched primary data is invalid. Refusing to save it. [2025-11-04T17:54:48.070330] [WARN ] [MAIN ] CAT: Deleting existing primary and derivative data due to validation failure. [2025-11-04T17:54:48.070392] [INFO ] [LOCAL ] CAT: Starting deletion of primary data. [2025-11-04T17:54:48.070454] [DEBUG] [LOCAL ] CAT: No primary data file found to delete. [2025-11-04T17:54:48.070489] [INFO ] [LOCAL ] CAT: Starting deletion of derivative data. [2025-11-04T17:54:48.070637] [DEBUG] [LOCAL ] CAT: Deleted derivative data file. [2025-11-04T17:54:48.070751] [INFO ] [MAIN ] Update Progress: 400/2367 symbols completed. CAT took 41.4s. Estimated remaining: 4h 29m 13s. Rate: 7.3 symbols/min [2025-11-04T17:54:48.751830] [INFO ] [MAIN ] CBAN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:54:48.755516] [INFO ] [MAIN ] CBL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:54:48.750900] [INFO ] [MAIN ] CAVA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:54:48.782861] [INFO ] [MAIN ] CB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:54:48.783290] [DEBUG] [LOCAL ] CB: No existing data file found in primary storage. [2025-11-04T17:54:48.783341] [WARN ] [MAIN ] CB: No existing data found, performing full refresh instead. [2025-11-04T17:54:48.783427] [INFO ] [MAIN ] CB: Wiping existing primary data before full refresh. [2025-11-04T17:54:48.783463] [INFO ] [LOCAL ] CB: Starting deletion of primary data. [2025-11-04T17:54:48.783502] [DEBUG] [LOCAL ] CB: No primary data file found to delete. [2025-11-04T17:54:48.783532] [INFO ] [MAIN ] CB: Starting full primary refresh. [2025-11-04T17:54:48.902592] [DEBUG] [LOCAL ] CBAN: Loaded 9885 existing records from primary storage. [2025-11-04T17:54:48.903187] [DEBUG] [LOCAL ] CBL: Loaded 15410 existing records from primary storage. [2025-11-04T17:54:48.903676] [INFO ] [MAIN ] CBAN: Loaded 9885 existing records. [2025-11-04T17:54:48.914559] [INFO ] [MAIN ] CBL: Loaded 15410 existing records. [2025-11-04T17:54:49.375111] [DEBUG] [LOCAL ] CAVA: Loaded 53544 existing records from primary storage. [2025-11-04T17:54:49.424924] [INFO ] [MAIN ] CAVA: Loaded 53544 existing records. [2025-11-04T17:54:49.609136] [WARN ] [MAIN ] CBL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:54:49.606194] [WARN ] [MAIN ] CBAN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:54:49.643211] [INFO ] [MAIN ] CBL: Wiping existing primary data before full refresh. [2025-11-04T17:54:49.643325] [INFO ] [LOCAL ] CBL: Starting deletion of primary data. [2025-11-04T17:54:49.643463] [INFO ] [MAIN ] CBAN: Wiping existing primary data before full refresh. [2025-11-04T17:54:49.669447] [INFO ] [REMOTE ] CB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:54:49.665355] [INFO ] [LOCAL ] CBAN: Starting deletion of primary data. [2025-11-04T17:54:49.685422] [DEBUG] [LOCAL ] CBAN: Deleted primary data file. [2025-11-04T17:54:49.685693] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-01. [2025-11-04T17:54:49.696018] [DEBUG] [LOCAL ] CBL: Deleted primary data file. [2025-11-04T17:54:49.707602] [INFO ] [MAIN ] CBL: Starting full primary refresh. [2025-11-04T17:54:49.702427] [INFO ] [MAIN ] CBAN: Starting full primary refresh. [2025-11-04T17:54:50.035152] [WARN ] [MAIN ] CAVA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:54:50.059031] [INFO ] [MAIN ] CAVA: Wiping existing primary data before full refresh. [2025-11-04T17:54:50.064595] [INFO ] [LOCAL ] CAVA: Starting deletion of primary data. [2025-11-04T17:54:50.116339] [DEBUG] [LOCAL ] CAVA: Deleted primary data file. [2025-11-04T17:54:50.179700] [INFO ] [MAIN ] CAVA: Starting full primary refresh. [2025-11-04T17:54:50.763532] [INFO ] [REMOTE ] CBL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:54:50.777626] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-01. [2025-11-04T17:54:50.781199] [INFO ] [REMOTE ] CBAN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:54:50.792258] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-01. [2025-11-04T17:54:50.993324] [INFO ] [REMOTE ] CAVA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:54:50.993686] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-01. [2025-11-04T17:54:51.471411] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-11-04T17:54:51.481635] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-02. [2025-11-04T17:54:51.506043] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-11-04T17:54:51.510623] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-02. [2025-11-04T17:54:51.649161] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T17:54:51.652315] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-02. [2025-11-04T17:54:52.117101] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-11-04T17:54:52.121327] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-03. [2025-11-04T17:54:52.206713] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T17:54:52.209914] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-03. [2025-11-04T17:54:52.546955] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-11-04T17:54:52.870284] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-11-04T17:54:52.967020] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-02. [2025-11-04T17:54:52.990413] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-11-04T17:54:53.019618] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-11-04T17:54:53.767057] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-04. [2025-11-04T17:54:54.434337] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-11-04T17:54:54.470128] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-11-04T17:54:54.567082] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-03. [2025-11-04T17:54:55.367045] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-04. [2025-11-04T17:54:56.020888] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-11-04T17:54:56.167019] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-05. [2025-11-04T17:54:56.336131] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-11-04T17:54:56.807866] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T17:54:56.967071] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-03. [2025-11-04T17:54:57.767104] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-05. [2025-11-04T17:54:58.399897] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-11-04T17:54:58.454483] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-11-04T17:54:58.567051] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-04. [2025-11-04T17:54:59.367053] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-04. [2025-11-04T17:55:00.115035] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-11-04T17:55:00.167038] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-05. [2025-11-04T17:55:00.967033] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-06. [2025-11-04T17:55:01.064744] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-11-04T17:55:01.767122] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-05. [2025-11-04T17:55:01.997893] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T17:55:02.059331] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-11-04T17:55:02.567062] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-06. [2025-11-04T17:55:03.367060] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-06. [2025-11-04T17:55:03.750803] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-11-04T17:55:04.031943] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T17:55:04.167020] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-07. [2025-11-04T17:55:04.195765] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T17:55:04.967040] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-06. [2025-11-04T17:55:05.535208] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T17:55:05.767093] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-08. [2025-11-04T17:55:06.567013] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-07. [2025-11-04T17:55:06.646885] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-11-04T17:55:07.064707] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-11-04T17:55:07.145479] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T17:55:07.367065] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-07. [2025-11-04T17:55:08.065566] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T17:55:08.167028] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-07. [2025-11-04T17:55:08.967045] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-08. [2025-11-04T17:55:09.677301] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T17:55:09.767022] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-08. [2025-11-04T17:55:09.809570] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-11-04T17:55:10.374013] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-11-04T17:55:10.567050] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-09. [2025-11-04T17:55:11.367051] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-08. [2025-11-04T17:55:12.047729] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-11-04T17:55:12.167008] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-10. [2025-11-04T17:55:12.967035] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-09. [2025-11-04T17:55:13.767137] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-09. [2025-11-04T17:55:14.018241] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T17:55:14.317495] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-08 (2.95 seconds). [2025-11-04T17:55:14.567069] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-10. [2025-11-04T17:55:15.160368] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-10 (2.99 seconds). [2025-11-04T17:55:15.172360] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-11-04T17:55:15.186148] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T17:55:15.367046] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-11. [2025-11-04T17:55:16.167040] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-11. [2025-11-04T17:55:16.287440] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T17:55:16.288325] [INFO ] [REMOTE ] CBAN: Full fetch complete (16306 records). [2025-11-04T17:55:16.288395] [DEBUG] [VALIDATOR ] CBAN: Validating primary data. [2025-11-04T17:55:16.643845] [DEBUG] [VALIDATOR ] CBAN: All expected trading days are present. [2025-11-04T17:55:16.643922] [DEBUG] [VALIDATOR ] CBAN: Primary data validation passed. [2025-11-04T17:55:16.644250] [INFO ] [MAIN ] CBAN: Fetched primary data is valid. [2025-11-04T17:55:16.644288] [DEBUG] [LOCAL ] CBAN: Saving primary data. [2025-11-04T17:55:16.644807] [DEBUG] [LOCAL ] CBAN: Trimmed 16306 records to 9896 (keeping 200 days of data) [2025-11-04T17:55:16.703946] [DEBUG] [LOCAL ] CBAN: Primary data saved (9896 records). [2025-11-04T17:55:16.704088] [INFO ] [MAIN ] CBAN: Full history saved. [2025-11-04T17:55:16.704172] [INFO ] [MAIN ] Update Progress: 401/2367 symbols completed. CBAN took 28.6s. Estimated remaining: 4h 30m 45s. Rate: 7.3 symbols/min [2025-11-04T17:55:16.825214] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T17:55:16.860953] [INFO ] [REMOTE ] CB: Full fetch complete (72846 records). [2025-11-04T17:55:16.871820] [DEBUG] [VALIDATOR ] CB: Validating primary data. [2025-11-04T17:55:16.911728] [INFO ] [MAIN ] CBNA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:55:16.972310] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-10. [2025-11-04T17:55:17.024800] [DEBUG] [LOCAL ] CBNA: Loaded 2696 existing records from primary storage. [2025-11-04T17:55:17.025581] [INFO ] [MAIN ] CBNA: Loaded 2696 existing records. [2025-11-04T17:55:17.434765] [WARN ] [MAIN ] CBNA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:55:17.458921] [INFO ] [MAIN ] CBNA: Wiping existing primary data before full refresh. [2025-11-04T17:55:17.459308] [INFO ] [LOCAL ] CBNA: Starting deletion of primary data. [2025-11-04T17:55:17.475234] [DEBUG] [LOCAL ] CBNA: Deleted primary data file. [2025-11-04T17:55:17.505254] [INFO ] [MAIN ] CBNA: Starting full primary refresh. [2025-11-04T17:55:17.613755] [DEBUG] [VALIDATOR ] CB: All expected trading days are present. [2025-11-04T17:55:17.670281] [DEBUG] [VALIDATOR ] CB: Primary data validation passed. [2025-11-04T17:55:17.695680] [INFO ] [MAIN ] CB: Fetched primary data is valid. [2025-11-04T17:55:17.708845] [DEBUG] [LOCAL ] CB: Saving primary data. [2025-11-04T17:55:17.743062] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-11-04T17:55:17.769986] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-09. [2025-11-04T17:55:17.818995] [DEBUG] [LOCAL ] CB: Trimmed 72846 records to 49917 (keeping 200 days of data) [2025-11-04T17:55:18.135101] [INFO ] [REMOTE ] CBNA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:55:18.320888] [DEBUG] [LOCAL ] CB: Primary data saved (49917 records). [2025-11-04T17:55:18.321014] [INFO ] [MAIN ] CB: Full history saved. [2025-11-04T17:55:18.321081] [INFO ] [MAIN ] Update Progress: 402/2367 symbols completed. CB took 30.2s. Estimated remaining: 4h 30m 4s. Rate: 7.3 symbols/min [2025-11-04T17:55:18.489101] [INFO ] [MAIN ] CBO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:55:18.489285] [DEBUG] [LOCAL ] CBO: No existing data file found in primary storage. [2025-11-04T17:55:18.489322] [WARN ] [MAIN ] CBO: No existing data found, performing full refresh instead. [2025-11-04T17:55:18.489458] [INFO ] [MAIN ] CBO: Wiping existing primary data before full refresh. [2025-11-04T17:55:18.489498] [INFO ] [LOCAL ] CBO: Starting deletion of primary data. [2025-11-04T17:55:18.489538] [DEBUG] [LOCAL ] CBO: No primary data file found to delete. [2025-11-04T17:55:18.489568] [INFO ] [MAIN ] CBO: Starting full primary refresh. [2025-11-04T17:55:18.572205] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-01. [2025-11-04T17:55:18.839270] [INFO ] [REMOTE ] CBO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:55:19.212748] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-11-04T17:55:19.367103] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-01. [2025-11-04T17:55:19.547176] [DEBUG] [REMOTE ] CBO: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:55:19.547508] [WARN ] [REMOTE ] CBO: No data returned for month 2025-01 [2025-11-04T17:55:19.547606] [FATAL] [REMOTE ] CBO: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:55:19.547702] [FATAL] [MAIN ] CBO: Failed to fetch primary data. [2025-11-04T17:55:19.547761] [WARN ] [MAIN ] CBO: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:55:19.547829] [INFO ] [LOCAL ] CBO: Starting deletion of primary data. [2025-11-04T17:55:19.547892] [DEBUG] [LOCAL ] CBO: No primary data file found to delete. [2025-11-04T17:55:19.547927] [INFO ] [LOCAL ] CBO: Starting deletion of derivative data. [2025-11-04T17:55:19.547966] [DEBUG] [LOCAL ] CBO: No derivative data file found to delete. [2025-11-04T17:55:19.548010] [INFO ] [MAIN ] Update Progress: 403/2367 symbols completed. CBO took 1.2s. Estimated remaining: 4h 29m 22s. Rate: 7.3 symbols/min [2025-11-04T17:55:19.724979] [INFO ] [MAIN ] CBRE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:55:19.772788] [DEBUG] [LOCAL ] CBRE: Loaded 50209 existing records from primary storage. [2025-11-04T17:55:19.773081] [INFO ] [MAIN ] CBRE: Loaded 50209 existing records. [2025-11-04T17:55:19.990750] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-11-04T17:55:19.989076] [WARN ] [MAIN ] CBRE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:55:20.002059] [INFO ] [MAIN ] CBRE: Wiping existing primary data before full refresh. [2025-11-04T17:55:20.002143] [INFO ] [LOCAL ] CBRE: Starting deletion of primary data. [2025-11-04T17:55:20.002418] [DEBUG] [LOCAL ] CBRE: Deleted primary data file. [2025-11-04T17:55:20.002478] [INFO ] [MAIN ] CBRE: Starting full primary refresh. [2025-11-04T17:55:20.168137] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-10. [2025-11-04T17:55:20.367700] [INFO ] [REMOTE ] CBRE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:55:20.967049] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-02. [2025-11-04T17:55:21.271554] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-02 (0.30 seconds). [2025-11-04T17:55:21.767074] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-11. [2025-11-04T17:55:22.163163] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-11-04T17:55:22.317225] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T17:55:22.318511] [INFO ] [REMOTE ] CBL: Full fetch complete (24099 records). [2025-11-04T17:55:22.318585] [DEBUG] [VALIDATOR ] CBL: Validating primary data. [2025-11-04T17:55:22.572230] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-01. [2025-11-04T17:55:22.710051] [DEBUG] [VALIDATOR ] CBL: All expected trading days are present. [2025-11-04T17:55:22.710170] [DEBUG] [VALIDATOR ] CBL: Primary data validation passed. [2025-11-04T17:55:22.710775] [INFO ] [MAIN ] CBL: Fetched primary data is valid. [2025-11-04T17:55:22.710860] [DEBUG] [LOCAL ] CBL: Saving primary data. [2025-11-04T17:55:22.711986] [DEBUG] [LOCAL ] CBL: Trimmed 24099 records to 15442 (keeping 200 days of data) [2025-11-04T17:55:22.812298] [DEBUG] [LOCAL ] CBL: Primary data saved (15442 records). [2025-11-04T17:55:22.812451] [INFO ] [MAIN ] CBL: Full history saved. [2025-11-04T17:55:22.812539] [INFO ] [MAIN ] Update Progress: 404/2367 symbols completed. CBL took 34.7s. Estimated remaining: 4h 28m 49s. Rate: 7.3 symbols/min [2025-11-04T17:55:23.024577] [INFO ] [MAIN ] CBT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:55:23.024798] [DEBUG] [LOCAL ] CBT: No existing data file found in primary storage. [2025-11-04T17:55:23.024842] [WARN ] [MAIN ] CBT: No existing data found, performing full refresh instead. [2025-11-04T17:55:23.024957] [INFO ] [MAIN ] CBT: Wiping existing primary data before full refresh. [2025-11-04T17:55:23.024998] [INFO ] [LOCAL ] CBT: Starting deletion of primary data. [2025-11-04T17:55:23.025043] [DEBUG] [LOCAL ] CBT: No primary data file found to delete. [2025-11-04T17:55:23.025103] [INFO ] [MAIN ] CBT: Starting full primary refresh. [2025-11-04T17:55:23.372486] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-11. [2025-11-04T17:55:23.413174] [INFO ] [REMOTE ] CBT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:55:24.014948] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T17:55:24.167037] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-02. [2025-11-04T17:55:24.805181] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-11 (1.43 seconds). [2025-11-04T17:55:24.807437] [INFO ] [REMOTE ] CAVA: Full fetch complete (77527 records). [2025-11-04T17:55:24.807592] [DEBUG] [VALIDATOR ] CAVA: Validating primary data. [2025-11-04T17:55:24.972183] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-03. [2025-11-04T17:55:25.206670] [DEBUG] [VALIDATOR ] CAVA: All expected trading days are present. [2025-11-04T17:55:25.206772] [DEBUG] [VALIDATOR ] CAVA: Primary data validation passed. [2025-11-04T17:55:25.207688] [INFO ] [MAIN ] CAVA: Fetched primary data is valid. [2025-11-04T17:55:25.207751] [DEBUG] [LOCAL ] CAVA: Saving primary data. [2025-11-04T17:55:25.209289] [DEBUG] [LOCAL ] CAVA: Trimmed 77527 records to 53548 (keeping 200 days of data) [2025-11-04T17:55:25.426209] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-11-04T17:55:25.548900] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-11-04T17:55:25.576391] [DEBUG] [LOCAL ] CAVA: Primary data saved (53548 records). [2025-11-04T17:55:25.576527] [INFO ] [MAIN ] CAVA: Full history saved. [2025-11-04T17:55:25.577349] [INFO ] [MAIN ] Update Progress: 405/2367 symbols completed. CAVA took 37.5s. Estimated remaining: 4h 28m 15s. Rate: 7.3 symbols/min [2025-11-04T17:55:25.753354] [INFO ] [MAIN ] CBU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:55:25.767041] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-03. [2025-11-04T17:55:25.777018] [DEBUG] [LOCAL ] CBU: Loaded 22239 existing records from primary storage. [2025-11-04T17:55:25.777267] [INFO ] [MAIN ] CBU: Loaded 22239 existing records. [2025-11-04T17:55:25.951965] [WARN ] [MAIN ] CBU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:55:25.952210] [INFO ] [MAIN ] CBU: Wiping existing primary data before full refresh. [2025-11-04T17:55:25.952256] [INFO ] [LOCAL ] CBU: Starting deletion of primary data. [2025-11-04T17:55:25.952820] [DEBUG] [LOCAL ] CBU: Deleted primary data file. [2025-11-04T17:55:25.952877] [INFO ] [MAIN ] CBU: Starting full primary refresh. [2025-11-04T17:55:26.301996] [INFO ] [REMOTE ] CBU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:55:26.567064] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-04. [2025-11-04T17:55:26.966532] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-11-04T17:55:27.146534] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T17:55:27.367033] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-04. [2025-11-04T17:55:28.167047] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-01. [2025-11-04T17:55:28.802109] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T17:55:28.967033] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-05. [2025-11-04T17:55:29.195946] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-11-04T17:55:29.767032] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-05. [2025-11-04T17:55:30.241459] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-11-04T17:55:30.299839] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T17:55:30.567028] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-01. [2025-11-04T17:55:31.367017] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-06. [2025-11-04T17:55:31.385353] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-11-04T17:55:31.705142] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-11-04T17:55:32.167068] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-06. [2025-11-04T17:55:32.967095] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-02. [2025-11-04T17:55:33.370748] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T17:55:33.767076] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-07. [2025-11-04T17:55:33.931891] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T17:55:34.567050] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-02. [2025-11-04T17:55:35.280789] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T17:55:35.340043] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-11-04T17:55:35.367005] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-08. [2025-11-04T17:55:36.167057] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-03. [2025-11-04T17:55:36.671531] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T17:55:36.967012] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-09. [2025-11-04T17:55:37.239940] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T17:55:37.767049] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-04. [2025-11-04T17:55:38.144636] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T17:55:38.567016] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-07. [2025-11-04T17:55:38.780160] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-11-04T17:55:38.852110] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-07 (0.29 seconds). [2025-11-04T17:55:39.367054] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-10. [2025-11-04T17:55:40.167078] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-05. [2025-11-04T17:55:40.759222] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T17:55:40.967047] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-08. [2025-11-04T17:55:41.261509] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-11-04T17:55:41.406911] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T17:55:41.767066] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-11. [2025-11-04T17:55:42.567088] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-06. [2025-11-04T17:55:42.791261] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T17:55:42.793167] [INFO ] [REMOTE ] CBRE: Full fetch complete (73661 records). [2025-11-04T17:55:42.793272] [DEBUG] [VALIDATOR ] CBRE: Validating primary data. [2025-11-04T17:55:43.205384] [DEBUG] [VALIDATOR ] CBRE: All expected trading days are present. [2025-11-04T17:55:43.205477] [DEBUG] [VALIDATOR ] CBRE: Primary data validation passed. [2025-11-04T17:55:43.206277] [INFO ] [MAIN ] CBRE: Fetched primary data is valid. [2025-11-04T17:55:43.206339] [DEBUG] [LOCAL ] CBRE: Saving primary data. [2025-11-04T17:55:43.207733] [DEBUG] [LOCAL ] CBRE: Trimmed 73661 records to 50202 (keeping 200 days of data) [2025-11-04T17:55:43.374240] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-03. [2025-11-04T17:55:43.508900] [DEBUG] [LOCAL ] CBRE: Primary data saved (50202 records). [2025-11-04T17:55:43.509120] [INFO ] [MAIN ] CBRE: Full history saved. [2025-11-04T17:55:43.509230] [INFO ] [MAIN ] Update Progress: 406/2367 symbols completed. CBRE took 24.0s. Estimated remaining: 4h 28m 53s. Rate: 7.3 symbols/min [2025-11-04T17:55:43.626821] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-11-04T17:55:43.700754] [INFO ] [MAIN ] CBX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:55:43.700929] [DEBUG] [LOCAL ] CBX: No existing data file found in primary storage. [2025-11-04T17:55:43.700966] [WARN ] [MAIN ] CBX: No existing data found, performing full refresh instead. [2025-11-04T17:55:43.701085] [INFO ] [MAIN ] CBX: Wiping existing primary data before full refresh. [2025-11-04T17:55:43.701149] [INFO ] [LOCAL ] CBX: Starting deletion of primary data. [2025-11-04T17:55:43.701211] [DEBUG] [LOCAL ] CBX: No primary data file found to delete. [2025-11-04T17:55:43.701254] [INFO ] [MAIN ] CBX: Starting full primary refresh. [2025-11-04T17:55:44.050009] [INFO ] [REMOTE ] CBX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:55:44.167011] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-01. [2025-11-04T17:55:44.324178] [DEBUG] [REMOTE ] CBX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:55:44.324809] [WARN ] [REMOTE ] CBX: No data returned for month 2025-01 [2025-11-04T17:55:44.324938] [FATAL] [REMOTE ] CBX: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:55:44.325275] [FATAL] [MAIN ] CBX: Failed to fetch primary data. [2025-11-04T17:55:44.325462] [WARN ] [MAIN ] CBX: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:55:44.330774] [INFO ] [LOCAL ] CBX: Starting deletion of primary data. [2025-11-04T17:55:44.339437] [DEBUG] [LOCAL ] CBX: No primary data file found to delete. [2025-11-04T17:55:44.339722] [INFO ] [LOCAL ] CBX: Starting deletion of derivative data. [2025-11-04T17:55:44.340144] [DEBUG] [LOCAL ] CBX: No derivative data file found to delete. [2025-11-04T17:55:44.340610] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-11-04T17:55:44.340804] [INFO ] [MAIN ] Update Progress: 407/2367 symbols completed. CBX took 0.8s. Estimated remaining: 4h 28m 10s. Rate: 7.3 symbols/min [2025-11-04T17:55:44.519431] [INFO ] [MAIN ] CBZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:55:44.550562] [DEBUG] [LOCAL ] CBZ: Loaded 34394 existing records from primary storage. [2025-11-04T17:55:44.550853] [INFO ] [MAIN ] CBZ: Loaded 34394 existing records. [2025-11-04T17:55:44.723546] [WARN ] [MAIN ] CBZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:55:44.723791] [INFO ] [MAIN ] CBZ: Wiping existing primary data before full refresh. [2025-11-04T17:55:44.723839] [INFO ] [LOCAL ] CBZ: Starting deletion of primary data. [2025-11-04T17:55:44.724009] [DEBUG] [LOCAL ] CBZ: Deleted primary data file. [2025-11-04T17:55:44.724045] [INFO ] [MAIN ] CBZ: Starting full primary refresh. [2025-11-04T17:55:44.972150] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-04. [2025-11-04T17:55:45.078363] [INFO ] [REMOTE ] CBZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:55:45.767052] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-09. [2025-11-04T17:55:45.785964] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-11-04T17:55:46.105914] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-09 (0.34 seconds). [2025-11-04T17:55:46.567089] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-05. [2025-11-04T17:55:47.256661] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-11-04T17:55:47.367054] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-06. [2025-11-04T17:55:48.022332] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-11-04T17:55:48.167011] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-01. [2025-11-04T17:55:48.945718] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-11-04T17:55:48.966997] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-02. [2025-11-04T17:55:49.767022] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-07. [2025-11-04T17:55:49.795143] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T17:55:50.567015] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-10. [2025-11-04T17:55:50.688776] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T17:55:50.845971] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-10 (0.28 seconds). [2025-11-04T17:55:51.367037] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-08. [2025-11-04T17:55:52.167035] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-11. [2025-11-04T17:55:52.200263] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-11-04T17:55:52.886629] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T17:55:52.887348] [INFO ] [REMOTE ] CBNA: Full fetch complete (4056 records). [2025-11-04T17:55:52.887417] [DEBUG] [VALIDATOR ] CBNA: Validating primary data. [2025-11-04T17:55:52.972172] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-03. [2025-11-04T17:55:53.242435] [DEBUG] [VALIDATOR ] CBNA: All expected trading days are present. [2025-11-04T17:55:53.242510] [DEBUG] [VALIDATOR ] CBNA: Primary data validation passed. [2025-11-04T17:55:53.242716] [INFO ] [MAIN ] CBNA: Fetched primary data is valid. [2025-11-04T17:55:53.242776] [DEBUG] [LOCAL ] CBNA: Saving primary data. [2025-11-04T17:55:53.243124] [DEBUG] [LOCAL ] CBNA: Trimmed 4056 records to 2644 (keeping 200 days of data) [2025-11-04T17:55:53.258403] [DEBUG] [LOCAL ] CBNA: Primary data saved (2644 records). [2025-11-04T17:55:53.258484] [INFO ] [MAIN ] CBNA: Full history saved. [2025-11-04T17:55:53.258553] [INFO ] [MAIN ] Update Progress: 408/2367 symbols completed. CBNA took 36.6s. Estimated remaining: 4h 28m 5s. Rate: 7.3 symbols/min [2025-11-04T17:55:53.425969] [INFO ] [MAIN ] CC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:55:53.477626] [DEBUG] [LOCAL ] CC: Loaded 52977 existing records from primary storage. [2025-11-04T17:55:53.477992] [INFO ] [MAIN ] CC: Loaded 52977 existing records. [2025-11-04T17:55:53.687059] [WARN ] [MAIN ] CC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:55:53.687327] [INFO ] [MAIN ] CC: Wiping existing primary data before full refresh. [2025-11-04T17:55:53.687379] [INFO ] [LOCAL ] CC: Starting deletion of primary data. [2025-11-04T17:55:53.687596] [DEBUG] [LOCAL ] CC: Deleted primary data file. [2025-11-04T17:55:53.687689] [INFO ] [MAIN ] CC: Starting full primary refresh. [2025-11-04T17:55:53.772238] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-09. [2025-11-04T17:55:54.052046] [INFO ] [REMOTE ] CC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:55:54.071231] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T17:55:54.567052] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-04. [2025-11-04T17:55:54.765784] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T17:55:55.367093] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-10. [2025-11-04T17:55:55.764096] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T17:55:55.983218] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-11-04T17:55:56.167044] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-05. [2025-11-04T17:55:56.967019] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-07. [2025-11-04T17:55:57.018742] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-11-04T17:55:57.767004] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-06. [2025-11-04T17:55:57.944897] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-11-04T17:55:58.567055] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-08. [2025-11-04T17:55:58.760517] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-11-04T17:55:59.367042] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-11. [2025-11-04T17:55:59.483254] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-11-04T17:56:00.085959] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T17:56:00.087308] [INFO ] [REMOTE ] CBU: Full fetch complete (32726 records). [2025-11-04T17:56:00.087398] [DEBUG] [VALIDATOR ] CBU: Validating primary data. [2025-11-04T17:56:00.172312] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-09. [2025-11-04T17:56:00.461736] [DEBUG] [VALIDATOR ] CBU: All expected trading days are present. [2025-11-04T17:56:00.461819] [DEBUG] [VALIDATOR ] CBU: Primary data validation passed. [2025-11-04T17:56:00.462260] [INFO ] [MAIN ] CBU: Fetched primary data is valid. [2025-11-04T17:56:00.462301] [DEBUG] [LOCAL ] CBU: Saving primary data. [2025-11-04T17:56:00.463119] [DEBUG] [LOCAL ] CBU: Trimmed 32726 records to 22268 (keeping 200 days of data) [2025-11-04T17:56:00.597171] [DEBUG] [LOCAL ] CBU: Primary data saved (22268 records). [2025-11-04T17:56:00.597299] [INFO ] [MAIN ] CBU: Full history saved. [2025-11-04T17:56:00.597378] [INFO ] [MAIN ] Update Progress: 409/2367 symbols completed. CBU took 35.0s. Estimated remaining: 4h 27m 52s. Rate: 7.3 symbols/min [2025-11-04T17:56:00.768783] [INFO ] [MAIN ] CCI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:56:00.819211] [DEBUG] [LOCAL ] CCI: Loaded 53094 existing records from primary storage. [2025-11-04T17:56:00.819533] [INFO ] [MAIN ] CCI: Loaded 53094 existing records. [2025-11-04T17:56:00.973088] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-07. [2025-11-04T17:56:00.996711] [WARN ] [MAIN ] CCI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:56:00.997526] [INFO ] [MAIN ] CCI: Wiping existing primary data before full refresh. [2025-11-04T17:56:00.997583] [INFO ] [LOCAL ] CCI: Starting deletion of primary data. [2025-11-04T17:56:00.997809] [DEBUG] [LOCAL ] CCI: Deleted primary data file. [2025-11-04T17:56:00.997865] [INFO ] [MAIN ] CCI: Starting full primary refresh. [2025-11-04T17:56:01.399793] [INFO ] [REMOTE ] CCI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:56:01.420477] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T17:56:01.767068] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-01. [2025-11-04T17:56:02.269367] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T17:56:02.567057] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-01. [2025-11-04T17:56:03.218791] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T17:56:03.367097] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-02. [2025-11-04T17:56:04.066115] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T17:56:04.167053] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-10. [2025-11-04T17:56:04.615970] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-11-04T17:56:04.967098] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-02. [2025-11-04T17:56:05.250274] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-11-04T17:56:05.767080] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-03. [2025-11-04T17:56:06.262367] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T17:56:06.567039] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-03. [2025-11-04T17:56:07.066160] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T17:56:07.367061] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-04. [2025-11-04T17:56:08.167029] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-08. [2025-11-04T17:56:08.227277] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-11-04T17:56:08.693450] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T17:56:08.967049] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-04. [2025-11-04T17:56:09.306688] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T17:56:09.767114] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-05. [2025-11-04T17:56:10.433792] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-11-04T17:56:10.567004] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-05. [2025-11-04T17:56:11.119423] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-11-04T17:56:11.367032] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-09. [2025-11-04T17:56:11.904930] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T17:56:12.167043] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-06. [2025-11-04T17:56:12.556131] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T17:56:12.967047] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-06. [2025-11-04T17:56:13.463258] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T17:56:13.767051] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-11. [2025-11-04T17:56:14.322534] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T17:56:14.567059] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-07. [2025-11-04T17:56:15.030281] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-11 (1.26 seconds). [2025-11-04T17:56:15.031973] [INFO ] [REMOTE ] CBT: Full fetch complete (46049 records). [2025-11-04T17:56:15.032054] [DEBUG] [VALIDATOR ] CBT: Validating primary data. [2025-11-04T17:56:15.369686] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-07. [2025-11-04T17:56:15.406875] [DEBUG] [VALIDATOR ] CBT: All expected trading days are present. [2025-11-04T17:56:15.406967] [DEBUG] [VALIDATOR ] CBT: Primary data validation passed. [2025-11-04T17:56:15.407588] [INFO ] [MAIN ] CBT: Fetched primary data is valid. [2025-11-04T17:56:15.407678] [DEBUG] [LOCAL ] CBT: Saving primary data. [2025-11-04T17:56:15.408728] [DEBUG] [LOCAL ] CBT: Trimmed 46049 records to 30720 (keeping 200 days of data) [2025-11-04T17:56:15.593736] [DEBUG] [LOCAL ] CBT: Primary data saved (30720 records). [2025-11-04T17:56:15.593855] [INFO ] [MAIN ] CBT: Full history saved. [2025-11-04T17:56:15.593927] [INFO ] [MAIN ] Update Progress: 410/2367 symbols completed. CBT took 52.8s. Estimated remaining: 4h 28m 17s. Rate: 7.3 symbols/min [2025-11-04T17:56:15.774682] [INFO ] [MAIN ] CCID: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:56:15.774897] [DEBUG] [LOCAL ] CCID: No existing data file found in primary storage. [2025-11-04T17:56:15.774945] [WARN ] [MAIN ] CCID: No existing data found, performing full refresh instead. [2025-11-04T17:56:15.775055] [INFO ] [MAIN ] CCID: Wiping existing primary data before full refresh. [2025-11-04T17:56:15.775096] [INFO ] [LOCAL ] CCID: Starting deletion of primary data. [2025-11-04T17:56:15.775137] [DEBUG] [LOCAL ] CCID: No primary data file found to delete. [2025-11-04T17:56:15.775198] [INFO ] [MAIN ] CCID: Starting full primary refresh. [2025-11-04T17:56:16.138616] [INFO ] [REMOTE ] CCID: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:56:16.167864] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-10. [2025-11-04T17:56:16.173284] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-11-04T17:56:16.951578] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-11-04T17:56:16.966994] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-08. [2025-11-04T17:56:17.359495] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-11-04T17:56:17.767083] [INFO ] [REMOTE ] CCID: Fetching primary data for 2025-01. [2025-11-04T17:56:17.963953] [DEBUG] [REMOTE ] CCID: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:56:17.964273] [WARN ] [REMOTE ] CCID: No data returned for month 2025-01 [2025-11-04T17:56:17.964405] [FATAL] [REMOTE ] CCID: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:56:17.964493] [FATAL] [MAIN ] CCID: Failed to fetch primary data. [2025-11-04T17:56:17.964571] [WARN ] [MAIN ] CCID: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:56:17.964689] [INFO ] [LOCAL ] CCID: Starting deletion of primary data. [2025-11-04T17:56:17.964784] [DEBUG] [LOCAL ] CCID: No primary data file found to delete. [2025-11-04T17:56:17.964822] [INFO ] [LOCAL ] CCID: Starting deletion of derivative data. [2025-11-04T17:56:17.964879] [DEBUG] [LOCAL ] CCID: No derivative data file found to delete. [2025-11-04T17:56:17.964926] [INFO ] [MAIN ] Update Progress: 411/2367 symbols completed. CCID took 2.4s. Estimated remaining: 4h 27m 40s. Rate: 7.3 symbols/min [2025-11-04T17:56:18.141512] [INFO ] [MAIN ] CCIF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:56:18.157283] [DEBUG] [LOCAL ] CCIF: Loaded 14981 existing records from primary storage. [2025-11-04T17:56:18.157522] [INFO ] [MAIN ] CCIF: Loaded 14981 existing records. [2025-11-04T17:56:18.349873] [WARN ] [MAIN ] CCIF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:56:18.350209] [INFO ] [MAIN ] CCIF: Wiping existing primary data before full refresh. [2025-11-04T17:56:18.350348] [INFO ] [LOCAL ] CCIF: Starting deletion of primary data. [2025-11-04T17:56:18.350582] [DEBUG] [LOCAL ] CCIF: Deleted primary data file. [2025-11-04T17:56:18.350704] [INFO ] [MAIN ] CCIF: Starting full primary refresh. [2025-11-04T17:56:18.568050] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-08. [2025-11-04T17:56:18.584547] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-11-04T17:56:18.798602] [INFO ] [REMOTE ] CCIF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:56:19.367038] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-11. [2025-11-04T17:56:19.941282] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-11-04T17:56:20.167051] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-01. [2025-11-04T17:56:20.705561] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-11 (1.34 seconds). [2025-11-04T17:56:20.707337] [INFO ] [REMOTE ] CBZ: Full fetch complete (47438 records). [2025-11-04T17:56:20.707480] [DEBUG] [VALIDATOR ] CBZ: Validating primary data. [2025-11-04T17:56:20.972165] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-09. [2025-11-04T17:56:21.051810] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T17:56:21.148774] [DEBUG] [VALIDATOR ] CBZ: All expected trading days are present. [2025-11-04T17:56:21.148863] [DEBUG] [VALIDATOR ] CBZ: Primary data validation passed. [2025-11-04T17:56:21.149504] [INFO ] [MAIN ] CBZ: Fetched primary data is valid. [2025-11-04T17:56:21.149548] [DEBUG] [LOCAL ] CBZ: Saving primary data. [2025-11-04T17:56:21.150739] [DEBUG] [LOCAL ] CBZ: Trimmed 47438 records to 34380 (keeping 200 days of data) [2025-11-04T17:56:21.351233] [DEBUG] [LOCAL ] CBZ: Primary data saved (34380 records). [2025-11-04T17:56:21.351396] [INFO ] [MAIN ] CBZ: Full history saved. [2025-11-04T17:56:21.351497] [INFO ] [MAIN ] Update Progress: 412/2367 symbols completed. CBZ took 37.0s. Estimated remaining: 4h 27m 9s. Rate: 7.3 symbols/min [2025-11-04T17:56:21.528580] [INFO ] [MAIN ] CCJ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:56:21.577775] [DEBUG] [LOCAL ] CCJ: Loaded 53694 existing records from primary storage. [2025-11-04T17:56:21.578082] [INFO ] [MAIN ] CCJ: Loaded 53694 existing records. [2025-11-04T17:56:21.770536] [WARN ] [MAIN ] CCJ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:56:21.767458] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-02. [2025-11-04T17:56:21.771605] [INFO ] [MAIN ] CCJ: Wiping existing primary data before full refresh. [2025-11-04T17:56:21.771701] [INFO ] [LOCAL ] CCJ: Starting deletion of primary data. [2025-11-04T17:56:21.771930] [DEBUG] [LOCAL ] CCJ: Deleted primary data file. [2025-11-04T17:56:21.772006] [INFO ] [MAIN ] CCJ: Starting full primary refresh. [2025-11-04T17:56:22.204166] [INFO ] [REMOTE ] CCJ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:56:22.463062] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T17:56:22.566266] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T17:56:22.572547] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-01. [2025-11-04T17:56:23.367052] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-03. [2025-11-04T17:56:24.007071] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T17:56:24.167006] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-10. [2025-11-04T17:56:24.395444] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-11-04T17:56:24.967056] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-04. [2025-11-04T17:56:25.576993] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-11-04T17:56:25.767028] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-09. [2025-11-04T17:56:26.062621] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-11-04T17:56:26.567041] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-02. [2025-11-04T17:56:27.280516] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-11-04T17:56:27.366997] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-10. [2025-11-04T17:56:27.906017] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-11-04T17:56:28.167041] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-03. [2025-11-04T17:56:28.764857] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T17:56:28.967074] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-05. [2025-11-04T17:56:29.605031] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T17:56:29.687794] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-11-04T17:56:29.767051] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-11. [2025-11-04T17:56:29.824519] [DEBUG] [REMOTE ] CC: Error fetching primary data for 2025-11: 'API call returned an empty response.' [2025-11-04T17:56:29.825044] [WARN ] [REMOTE ] CC: No data returned for month 2025-11 [2025-11-04T17:56:29.825177] [FATAL] [REMOTE ] CC: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:56:29.825308] [FATAL] [MAIN ] CC: Failed to fetch primary data. [2025-11-04T17:56:29.825384] [WARN ] [MAIN ] CC: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:56:29.825465] [INFO ] [LOCAL ] CC: Starting deletion of primary data. [2025-11-04T17:56:29.825544] [DEBUG] [LOCAL ] CC: No primary data file found to delete. [2025-11-04T17:56:29.825585] [INFO ] [LOCAL ] CC: Starting deletion of derivative data. [2025-11-04T17:56:29.825771] [DEBUG] [LOCAL ] CC: Deleted derivative data file. [2025-11-04T17:56:29.825871] [INFO ] [MAIN ] Update Progress: 413/2367 symbols completed. CC took 36.6s. Estimated remaining: 4h 27m 2s. Rate: 7.3 symbols/min [2025-11-04T17:56:30.020443] [INFO ] [MAIN ] CCK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:56:30.069899] [DEBUG] [LOCAL ] CCK: Loaded 46541 existing records from primary storage. [2025-11-04T17:56:30.070352] [INFO ] [MAIN ] CCK: Loaded 46541 existing records. [2025-11-04T17:56:30.266847] [WARN ] [MAIN ] CCK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:56:30.267247] [INFO ] [MAIN ] CCK: Wiping existing primary data before full refresh. [2025-11-04T17:56:30.267370] [INFO ] [LOCAL ] CCK: Starting deletion of primary data. [2025-11-04T17:56:30.267699] [DEBUG] [LOCAL ] CCK: Deleted primary data file. [2025-11-04T17:56:30.267760] [INFO ] [MAIN ] CCK: Starting full primary refresh. [2025-11-04T17:56:30.572184] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-11. [2025-11-04T17:56:30.648509] [INFO ] [REMOTE ] CCK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:56:30.940866] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-11 (0.37 seconds). [2025-11-04T17:56:30.942866] [INFO ] [REMOTE ] CCI: Full fetch complete (77095 records). [2025-11-04T17:56:30.943085] [DEBUG] [VALIDATOR ] CCI: Validating primary data. [2025-11-04T17:56:31.327959] [DEBUG] [VALIDATOR ] CCI: All expected trading days are present. [2025-11-04T17:56:31.328037] [DEBUG] [VALIDATOR ] CCI: Primary data validation passed. [2025-11-04T17:56:31.328840] [INFO ] [MAIN ] CCI: Fetched primary data is valid. [2025-11-04T17:56:31.328903] [DEBUG] [LOCAL ] CCI: Saving primary data. [2025-11-04T17:56:31.330284] [DEBUG] [LOCAL ] CCI: Trimmed 77095 records to 53104 (keeping 200 days of data) [2025-11-04T17:56:31.384139] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-01. [2025-11-04T17:56:31.657800] [DEBUG] [LOCAL ] CCI: Primary data saved (53104 records). [2025-11-04T17:56:31.658017] [INFO ] [MAIN ] CCI: Full history saved. [2025-11-04T17:56:31.658698] [INFO ] [MAIN ] Update Progress: 414/2367 symbols completed. CCI took 31.1s. Estimated remaining: 4h 26m 24s. Rate: 7.3 symbols/min [2025-11-04T17:56:31.829546] [INFO ] [MAIN ] CCL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:56:31.886246] [DEBUG] [LOCAL ] CCL: Loaded 53800 existing records from primary storage. [2025-11-04T17:56:31.886569] [INFO ] [MAIN ] CCL: Loaded 53800 existing records. [2025-11-04T17:56:32.068014] [WARN ] [MAIN ] CCL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:56:32.068238] [INFO ] [MAIN ] CCL: Wiping existing primary data before full refresh. [2025-11-04T17:56:32.068318] [INFO ] [LOCAL ] CCL: Starting deletion of primary data. [2025-11-04T17:56:32.068570] [DEBUG] [LOCAL ] CCL: Deleted primary data file. [2025-11-04T17:56:32.068637] [INFO ] [MAIN ] CCL: Starting full primary refresh. [2025-11-04T17:56:32.170717] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-04. [2025-11-04T17:56:32.428418] [INFO ] [REMOTE ] CCL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:56:32.744609] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T17:56:32.967142] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-01. [2025-11-04T17:56:33.560021] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T17:56:33.767046] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-06. [2025-11-04T17:56:34.567089] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-02. [2025-11-04T17:56:34.661387] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-11-04T17:56:34.661835] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T17:56:35.367043] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-02. [2025-11-04T17:56:35.721572] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T17:56:36.167043] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-05. [2025-11-04T17:56:36.944847] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-11-04T17:56:36.967017] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-03. [2025-11-04T17:56:37.767071] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-07. [2025-11-04T17:56:37.825558] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-11-04T17:56:38.395268] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-11-04T17:56:38.567058] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-06. [2025-11-04T17:56:38.670870] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-11-04T17:56:39.367064] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-03. [2025-11-04T17:56:40.167063] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-08. [2025-11-04T17:56:40.236619] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-11-04T17:56:40.561811] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T17:56:40.813632] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-11-04T17:56:40.967033] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-09. [2025-11-04T17:56:41.698945] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-11-04T17:56:41.767010] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-07. [2025-11-04T17:56:42.567026] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-04. [2025-11-04T17:56:43.375729] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-04. [2025-11-04T17:56:43.396759] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-11-04T17:56:44.167059] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-08. [2025-11-04T17:56:44.432603] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-11-04T17:56:44.908469] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-11-04T17:56:44.967006] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-05. [2025-11-04T17:56:45.701544] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T17:56:45.767014] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-10. [2025-11-04T17:56:46.047283] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-11-04T17:56:46.409789] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-11-04T17:56:46.567049] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-05. [2025-11-04T17:56:47.367085] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-09. [2025-11-04T17:56:48.167089] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-11. [2025-11-04T17:56:48.714089] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-11-04T17:56:48.767353] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-11 (0.60 seconds). [2025-11-04T17:56:48.768386] [INFO ] [REMOTE ] CCIF: Full fetch complete (21758 records). [2025-11-04T17:56:48.768458] [DEBUG] [VALIDATOR ] CCIF: Validating primary data. [2025-11-04T17:56:48.977328] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-06. [2025-11-04T17:56:49.144591] [DEBUG] [VALIDATOR ] CCIF: All expected trading days are present. [2025-11-04T17:56:49.144726] [DEBUG] [VALIDATOR ] CCIF: Primary data validation passed. [2025-11-04T17:56:49.145288] [INFO ] [MAIN ] CCIF: Fetched primary data is valid. [2025-11-04T17:56:49.145371] [DEBUG] [LOCAL ] CCIF: Saving primary data. [2025-11-04T17:56:49.146295] [DEBUG] [LOCAL ] CCIF: Trimmed 21758 records to 14999 (keeping 200 days of data) [2025-11-04T17:56:49.232971] [DEBUG] [LOCAL ] CCIF: Primary data saved (14999 records). [2025-11-04T17:56:49.233176] [INFO ] [MAIN ] CCIF: Full history saved. [2025-11-04T17:56:49.233402] [INFO ] [MAIN ] Update Progress: 415/2367 symbols completed. CCIF took 31.3s. Estimated remaining: 4h 27m 0s. Rate: 7.3 symbols/min [2025-11-04T17:56:49.418402] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-11-04T17:56:49.460706] [INFO ] [MAIN ] CCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:56:49.504986] [DEBUG] [LOCAL ] CCO: Loaded 48738 existing records from primary storage. [2025-11-04T17:56:49.505305] [INFO ] [MAIN ] CCO: Loaded 48738 existing records. [2025-11-04T17:56:49.678659] [WARN ] [MAIN ] CCO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:56:49.678896] [INFO ] [MAIN ] CCO: Wiping existing primary data before full refresh. [2025-11-04T17:56:49.678943] [INFO ] [LOCAL ] CCO: Starting deletion of primary data. [2025-11-04T17:56:49.679125] [DEBUG] [LOCAL ] CCO: Deleted primary data file. [2025-11-04T17:56:49.679163] [INFO ] [MAIN ] CCO: Starting full primary refresh. [2025-11-04T17:56:49.772289] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-06. [2025-11-04T17:56:50.109205] [INFO ] [REMOTE ] CCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:56:50.183939] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T17:56:50.567045] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-01. [2025-11-04T17:56:51.367051] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-07. [2025-11-04T17:56:51.461490] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-11-04T17:56:51.662624] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-11-04T17:56:52.167048] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-02. [2025-11-04T17:56:52.489889] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-11-04T17:56:52.967092] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-08. [2025-11-04T17:56:53.133809] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T17:56:53.767055] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-10. [2025-11-04T17:56:54.169495] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T17:56:54.567041] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-07. [2025-11-04T17:56:55.367057] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-09. [2025-11-04T17:56:55.646618] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-11-04T17:56:56.167055] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-11. [2025-11-04T17:56:56.479968] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-11-04T17:56:56.635510] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T17:56:56.637595] [INFO ] [REMOTE ] CCJ: Full fetch complete (77846 records). [2025-11-04T17:56:56.637727] [DEBUG] [VALIDATOR ] CCJ: Validating primary data. [2025-11-04T17:56:56.875318] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-11-04T17:56:56.972957] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-10. [2025-11-04T17:56:57.095600] [DEBUG] [VALIDATOR ] CCJ: All expected trading days are present. [2025-11-04T17:56:57.095710] [DEBUG] [VALIDATOR ] CCJ: Primary data validation passed. [2025-11-04T17:56:57.096668] [INFO ] [MAIN ] CCJ: Fetched primary data is valid. [2025-11-04T17:56:57.096733] [DEBUG] [LOCAL ] CCJ: Saving primary data. [2025-11-04T17:56:57.098262] [DEBUG] [LOCAL ] CCJ: Trimmed 77846 records to 53697 (keeping 200 days of data) [2025-11-04T17:56:57.449034] [DEBUG] [LOCAL ] CCJ: Primary data saved (53697 records). [2025-11-04T17:56:57.449206] [INFO ] [MAIN ] CCJ: Full history saved. [2025-11-04T17:56:57.449347] [INFO ] [MAIN ] Update Progress: 416/2367 symbols completed. CCJ took 36.1s. Estimated remaining: 4h 26m 52s. Rate: 7.3 symbols/min [2025-11-04T17:56:57.657184] [INFO ] [MAIN ] CCS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:56:57.688612] [DEBUG] [LOCAL ] CCS: Loaded 29710 existing records from primary storage. [2025-11-04T17:56:57.688930] [INFO ] [MAIN ] CCS: Loaded 29710 existing records. [2025-11-04T17:56:57.772239] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-08. [2025-11-04T17:56:57.890555] [WARN ] [MAIN ] CCS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:56:57.890810] [INFO ] [MAIN ] CCS: Wiping existing primary data before full refresh. [2025-11-04T17:56:57.890873] [INFO ] [LOCAL ] CCS: Starting deletion of primary data. [2025-11-04T17:56:57.891083] [DEBUG] [LOCAL ] CCS: Deleted primary data file. [2025-11-04T17:56:57.891138] [INFO ] [MAIN ] CCS: Starting full primary refresh. [2025-11-04T17:56:58.279969] [INFO ] [REMOTE ] CCS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:56:58.310526] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-11-04T17:56:58.567014] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-01. [2025-11-04T17:56:59.367122] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-11. [2025-11-04T17:56:59.559036] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T17:56:59.740087] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-11-04T17:57:00.167064] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-02. [2025-11-04T17:57:00.528126] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T17:57:00.530140] [INFO ] [REMOTE ] CCK: Full fetch complete (68167 records). [2025-11-04T17:57:00.530274] [DEBUG] [VALIDATOR ] CCK: Validating primary data. [2025-11-04T17:57:00.924185] [DEBUG] [VALIDATOR ] CCK: All expected trading days are present. [2025-11-04T17:57:00.924267] [DEBUG] [VALIDATOR ] CCK: Primary data validation passed. [2025-11-04T17:57:00.925015] [INFO ] [MAIN ] CCK: Fetched primary data is valid. [2025-11-04T17:57:00.925079] [DEBUG] [LOCAL ] CCK: Saving primary data. [2025-11-04T17:57:00.926363] [DEBUG] [LOCAL ] CCK: Trimmed 68167 records to 46592 (keeping 200 days of data) [2025-11-04T17:57:00.980289] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-09. [2025-11-04T17:57:01.206583] [DEBUG] [LOCAL ] CCK: Primary data saved (46592 records). [2025-11-04T17:57:01.206944] [INFO ] [MAIN ] CCK: Full history saved. [2025-11-04T17:57:01.207042] [INFO ] [MAIN ] Update Progress: 417/2367 symbols completed. CCK took 31.4s. Estimated remaining: 4h 26m 23s. Rate: 7.3 symbols/min [2025-11-04T17:57:01.377488] [INFO ] [MAIN ] CCU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:57:01.397160] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T17:57:01.424991] [DEBUG] [LOCAL ] CCU: Loaded 20715 existing records from primary storage. [2025-11-04T17:57:01.425276] [INFO ] [MAIN ] CCU: Loaded 20715 existing records. [2025-11-04T17:57:01.598036] [WARN ] [MAIN ] CCU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:57:01.598274] [INFO ] [MAIN ] CCU: Wiping existing primary data before full refresh. [2025-11-04T17:57:01.598331] [INFO ] [LOCAL ] CCU: Starting deletion of primary data. [2025-11-04T17:57:01.598508] [DEBUG] [LOCAL ] CCU: Deleted primary data file. [2025-11-04T17:57:01.598564] [INFO ] [MAIN ] CCU: Starting full primary refresh. [2025-11-04T17:57:01.769380] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-03. [2025-11-04T17:57:01.956939] [INFO ] [REMOTE ] CCU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:57:02.567160] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-03. [2025-11-04T17:57:02.689033] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-11-04T17:57:02.856771] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T17:57:03.367060] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-04. [2025-11-04T17:57:03.646175] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T17:57:04.167001] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-04. [2025-11-04T17:57:04.339495] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-11-04T17:57:04.967036] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-10. [2025-11-04T17:57:05.466971] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T17:57:05.767080] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-01. [2025-11-04T17:57:06.567042] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-05. [2025-11-04T17:57:06.663731] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T17:57:06.858698] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-11-04T17:57:07.367096] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-05. [2025-11-04T17:57:07.455339] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T17:57:08.167085] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-02. [2025-11-04T17:57:08.585586] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T17:57:08.892014] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T17:57:08.967037] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-03. [2025-11-04T17:57:09.767137] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-11. [2025-11-04T17:57:10.034919] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T17:57:10.094785] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-11 (0.33 seconds). [2025-11-04T17:57:10.097093] [INFO ] [REMOTE ] CCL: Full fetch complete (77980 records). [2025-11-04T17:57:10.097215] [DEBUG] [VALIDATOR ] CCL: Validating primary data. [2025-11-04T17:57:10.498853] [DEBUG] [VALIDATOR ] CCL: All expected trading days are present. [2025-11-04T17:57:10.498943] [DEBUG] [VALIDATOR ] CCL: Primary data validation passed. [2025-11-04T17:57:10.499891] [INFO ] [MAIN ] CCL: Fetched primary data is valid. [2025-11-04T17:57:10.499958] [DEBUG] [LOCAL ] CCL: Saving primary data. [2025-11-04T17:57:10.501381] [DEBUG] [LOCAL ] CCL: Trimmed 77980 records to 53800 (keeping 200 days of data) [2025-11-04T17:57:10.584623] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-06. [2025-11-04T17:57:10.822615] [DEBUG] [LOCAL ] CCL: Primary data saved (53800 records). [2025-11-04T17:57:10.822776] [INFO ] [MAIN ] CCL: Full history saved. [2025-11-04T17:57:10.823318] [INFO ] [MAIN ] Update Progress: 418/2367 symbols completed. CCL took 39.2s. Estimated remaining: 4h 26m 21s. Rate: 7.3 symbols/min [2025-11-04T17:57:10.988274] [INFO ] [MAIN ] CCZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:57:10.988445] [DEBUG] [LOCAL ] CCZ: No existing data file found in primary storage. [2025-11-04T17:57:10.988481] [WARN ] [MAIN ] CCZ: No existing data found, performing full refresh instead. [2025-11-04T17:57:10.988593] [INFO ] [MAIN ] CCZ: Wiping existing primary data before full refresh. [2025-11-04T17:57:10.988633] [INFO ] [LOCAL ] CCZ: Starting deletion of primary data. [2025-11-04T17:57:10.988716] [DEBUG] [LOCAL ] CCZ: No primary data file found to delete. [2025-11-04T17:57:10.988755] [INFO ] [MAIN ] CCZ: Starting full primary refresh. [2025-11-04T17:57:11.341087] [INFO ] [REMOTE ] CCZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:57:11.367003] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-04. [2025-11-04T17:57:11.756975] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T17:57:12.167013] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-06. [2025-11-04T17:57:12.325223] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-11-04T17:57:12.967008] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-07. [2025-11-04T17:57:13.136930] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-11-04T17:57:13.767036] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-07. [2025-11-04T17:57:14.200233] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T17:57:14.567021] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-08. [2025-11-04T17:57:14.806866] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T17:57:15.367081] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-08. [2025-11-04T17:57:15.695155] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T17:57:16.167050] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-09. [2025-11-04T17:57:16.303495] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-11-04T17:57:16.967097] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-01. [2025-11-04T17:57:17.469557] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-11-04T17:57:17.502452] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T17:57:17.767054] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-05. [2025-11-04T17:57:18.567017] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-09. [2025-11-04T17:57:18.730948] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T17:57:19.367006] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-02. [2025-11-04T17:57:19.519377] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-11-04T17:57:19.535931] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-02 (0.17 seconds). [2025-11-04T17:57:20.167072] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-10. [2025-11-04T17:57:20.967075] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-10. [2025-11-04T17:57:21.397837] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T17:57:21.767120] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-06. [2025-11-04T17:57:21.879344] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-11-04T17:57:22.495199] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-11-04T17:57:22.566998] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-07. [2025-11-04T17:57:23.335238] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T17:57:23.367047] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-08. [2025-11-04T17:57:24.084447] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-11-04T17:57:24.167025] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-11. [2025-11-04T17:57:24.967075] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-11. [2025-11-04T17:57:25.310378] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T17:57:25.312247] [INFO ] [REMOTE ] CCO: Full fetch complete (67789 records). [2025-11-04T17:57:25.312325] [DEBUG] [VALIDATOR ] CCO: Validating primary data. [2025-11-04T17:57:25.733065] [DEBUG] [VALIDATOR ] CCO: All expected trading days are present. [2025-11-04T17:57:25.733155] [DEBUG] [VALIDATOR ] CCO: Primary data validation passed. [2025-11-04T17:57:25.733955] [INFO ] [MAIN ] CCO: Fetched primary data is valid. [2025-11-04T17:57:25.734018] [DEBUG] [LOCAL ] CCO: Saving primary data. [2025-11-04T17:57:25.735235] [DEBUG] [LOCAL ] CCO: Trimmed 67789 records to 48720 (keeping 200 days of data) [2025-11-04T17:57:25.776945] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-03. [2025-11-04T17:57:25.988168] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T17:57:25.990671] [INFO ] [REMOTE ] CCS: Full fetch complete (44682 records). [2025-11-04T17:57:25.990997] [DEBUG] [VALIDATOR ] CCS: Validating primary data. [2025-11-04T17:57:26.065363] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-11-04T17:57:26.242049] [DEBUG] [LOCAL ] CCO: Primary data saved (48720 records). [2025-11-04T17:57:26.262747] [INFO ] [MAIN ] CCO: Full history saved. [2025-11-04T17:57:26.268320] [INFO ] [MAIN ] Update Progress: 419/2367 symbols completed. CCO took 37.0s. Estimated remaining: 4h 26m 47s. Rate: 7.3 symbols/min [2025-11-04T17:57:26.545829] [DEBUG] [VALIDATOR ] CCS: All expected trading days are present. [2025-11-04T17:57:26.561532] [DEBUG] [VALIDATOR ] CCS: Primary data validation passed. [2025-11-04T17:57:26.564703] [INFO ] [MAIN ] CCS: Fetched primary data is valid. [2025-11-04T17:57:26.564055] [INFO ] [MAIN ] CDE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:57:26.564816] [DEBUG] [LOCAL ] CCS: Saving primary data. [2025-11-04T17:57:26.565992] [DEBUG] [LOCAL ] CCS: Trimmed 44682 records to 29737 (keeping 200 days of data) [2025-11-04T17:57:26.567035] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-04. [2025-11-04T17:57:26.767078] [DEBUG] [LOCAL ] CDE: Loaded 53644 existing records from primary storage. [2025-11-04T17:57:26.776300] [DEBUG] [LOCAL ] CCS: Primary data saved (29737 records). [2025-11-04T17:57:26.776503] [INFO ] [MAIN ] CCS: Full history saved. [2025-11-04T17:57:26.776742] [INFO ] [MAIN ] Update Progress: 420/2367 symbols completed. CCS took 29.3s. Estimated remaining: 4h 26m 3s. Rate: 7.3 symbols/min [2025-11-04T17:57:26.776625] [INFO ] [MAIN ] CDE: Loaded 53644 existing records. [2025-11-04T17:57:26.878200] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-11-04T17:57:27.113364] [INFO ] [MAIN ] CDLR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:57:27.132128] [WARN ] [MAIN ] CDE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:57:27.132361] [INFO ] [MAIN ] CDE: Wiping existing primary data before full refresh. [2025-11-04T17:57:27.132412] [INFO ] [LOCAL ] CDE: Starting deletion of primary data. [2025-11-04T17:57:27.133377] [DEBUG] [LOCAL ] CDE: Deleted primary data file. [2025-11-04T17:57:27.133440] [INFO ] [MAIN ] CDE: Starting full primary refresh. [2025-11-04T17:57:27.266811] [DEBUG] [LOCAL ] CDLR: Loaded 10417 existing records from primary storage. [2025-11-04T17:57:27.298586] [INFO ] [MAIN ] CDLR: Loaded 10417 existing records. [2025-11-04T17:57:27.372124] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-09. [2025-11-04T17:57:27.657205] [INFO ] [REMOTE ] CDE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:57:27.676225] [WARN ] [MAIN ] CDLR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:57:27.676377] [INFO ] [MAIN ] CDLR: Wiping existing primary data before full refresh. [2025-11-04T17:57:27.676423] [INFO ] [LOCAL ] CDLR: Starting deletion of primary data. [2025-11-04T17:57:27.676598] [DEBUG] [LOCAL ] CDLR: Deleted primary data file. [2025-11-04T17:57:27.676686] [INFO ] [MAIN ] CDLR: Starting full primary refresh. [2025-11-04T17:57:28.023007] [INFO ] [REMOTE ] CDLR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:57:28.167004] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-05. [2025-11-04T17:57:28.206510] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-11-04T17:57:28.506750] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-05 (0.34 seconds). [2025-11-04T17:57:28.967037] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-10. [2025-11-04T17:57:29.767038] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-01. [2025-11-04T17:57:29.912728] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T17:57:30.264894] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-11-04T17:57:30.567094] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-02. [2025-11-04T17:57:30.955307] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-11-04T17:57:31.367034] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-11. [2025-11-04T17:57:31.991615] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T17:57:31.993133] [INFO ] [REMOTE ] CCU: Full fetch complete (32038 records). [2025-11-04T17:57:31.993232] [DEBUG] [VALIDATOR ] CCU: Validating primary data. [2025-11-04T17:57:32.172270] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-01. [2025-11-04T17:57:32.399997] [DEBUG] [VALIDATOR ] CCU: All expected trading days are present. [2025-11-04T17:57:32.400304] [DEBUG] [VALIDATOR ] CCU: Primary data validation passed. [2025-11-04T17:57:32.400799] [INFO ] [MAIN ] CCU: Fetched primary data is valid. [2025-11-04T17:57:32.400863] [DEBUG] [LOCAL ] CCU: Saving primary data. [2025-11-04T17:57:32.401564] [DEBUG] [LOCAL ] CCU: Trimmed 32038 records to 20602 (keeping 200 days of data) [2025-11-04T17:57:32.516474] [DEBUG] [LOCAL ] CCU: Primary data saved (20602 records). [2025-11-04T17:57:32.516597] [INFO ] [MAIN ] CCU: Full history saved. [2025-11-04T17:57:32.516693] [INFO ] [MAIN ] Update Progress: 421/2367 symbols completed. CCU took 31.3s. Estimated remaining: 4h 25m 43s. Rate: 7.3 symbols/min [2025-11-04T17:57:32.682245] [INFO ] [MAIN ] CDP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:57:32.719345] [DEBUG] [LOCAL ] CDP: Loaded 42196 existing records from primary storage. [2025-11-04T17:57:32.719617] [INFO ] [MAIN ] CDP: Loaded 42196 existing records. [2025-11-04T17:57:32.893792] [WARN ] [MAIN ] CDP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:57:32.894070] [INFO ] [MAIN ] CDP: Wiping existing primary data before full refresh. [2025-11-04T17:57:32.894117] [INFO ] [LOCAL ] CDP: Starting deletion of primary data. [2025-11-04T17:57:32.894587] [DEBUG] [LOCAL ] CDP: Deleted primary data file. [2025-11-04T17:57:32.894678] [INFO ] [MAIN ] CDP: Starting full primary refresh. [2025-11-04T17:57:32.972289] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-03. [2025-11-04T17:57:33.248948] [INFO ] [REMOTE ] CDP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:57:33.643947] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-11-04T17:57:33.752506] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-11-04T17:57:33.766992] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-02. [2025-11-04T17:57:34.567046] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-01. [2025-11-04T17:57:35.126906] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T17:57:35.367032] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-03. [2025-11-04T17:57:35.729364] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T17:57:36.167027] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-02. [2025-11-04T17:57:36.783742] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T17:57:36.967067] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-04. [2025-11-04T17:57:37.288737] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-11-04T17:57:37.370271] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-11-04T17:57:37.767119] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-05. [2025-11-04T17:57:38.417579] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-11-04T17:57:38.567033] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-06. [2025-11-04T17:57:39.220838] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-11-04T17:57:39.367088] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-03. [2025-11-04T17:57:40.167047] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-07. [2025-11-04T17:57:40.475195] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T17:57:40.820620] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-11-04T17:57:40.967070] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-08. [2025-11-04T17:57:41.557945] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-11-04T17:57:41.767057] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-04. [2025-11-04T17:57:42.567055] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-09. [2025-11-04T17:57:42.946928] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T17:57:43.175764] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-11-04T17:57:43.367090] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-05. [2025-11-04T17:57:44.167157] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-10. [2025-11-04T17:57:44.444819] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-11-04T17:57:44.789328] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-11-04T17:57:44.967046] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-04. [2025-11-04T17:57:45.767064] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-06. [2025-11-04T17:57:46.038828] [DEBUG] [REMOTE ] CCZ: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:57:46.039121] [WARN ] [REMOTE ] CCZ: No data returned for month 2025-06 [2025-11-04T17:57:46.039207] [FATAL] [REMOTE ] CCZ: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:57:46.039290] [FATAL] [MAIN ] CCZ: Failed to fetch primary data. [2025-11-04T17:57:46.039348] [WARN ] [MAIN ] CCZ: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:57:46.039388] [INFO ] [LOCAL ] CCZ: Starting deletion of primary data. [2025-11-04T17:57:46.039455] [DEBUG] [LOCAL ] CCZ: No primary data file found to delete. [2025-11-04T17:57:46.039489] [INFO ] [LOCAL ] CCZ: Starting deletion of derivative data. [2025-11-04T17:57:46.039528] [DEBUG] [LOCAL ] CCZ: No derivative data file found to delete. [2025-11-04T17:57:46.039572] [INFO ] [MAIN ] Update Progress: 422/2367 symbols completed. CCZ took 35.2s. Estimated remaining: 4h 25m 60s. Rate: 7.3 symbols/min [2025-11-04T17:57:46.204379] [INFO ] [MAIN ] CDRE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:57:46.227629] [DEBUG] [LOCAL ] CDRE: Loaded 24446 existing records from primary storage. [2025-11-04T17:57:46.227888] [INFO ] [MAIN ] CDRE: Loaded 24446 existing records. [2025-11-04T17:57:46.401598] [WARN ] [MAIN ] CDRE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:57:46.401789] [INFO ] [MAIN ] CDRE: Wiping existing primary data before full refresh. [2025-11-04T17:57:46.401835] [INFO ] [LOCAL ] CDRE: Starting deletion of primary data. [2025-11-04T17:57:46.402091] [DEBUG] [LOCAL ] CDRE: Deleted primary data file. [2025-11-04T17:57:46.402146] [INFO ] [MAIN ] CDRE: Starting full primary refresh. [2025-11-04T17:57:46.577185] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-06. [2025-11-04T17:57:46.759111] [INFO ] [REMOTE ] CDRE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:57:46.791815] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-11-04T17:57:47.367054] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-05. [2025-11-04T17:57:47.879226] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T17:57:48.167060] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-07. [2025-11-04T17:57:48.867846] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-11-04T17:57:48.967047] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-06. [2025-11-04T17:57:49.286899] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-11-04T17:57:49.767054] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-01. [2025-11-04T17:57:50.336278] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-11-04T17:57:50.567012] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-08. [2025-11-04T17:57:50.721896] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T17:57:51.367049] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-07. [2025-11-04T17:57:51.754614] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T17:57:52.167044] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-02. [2025-11-04T17:57:52.786893] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-11-04T17:57:52.967036] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-09. [2025-11-04T17:57:53.144707] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-11-04T17:57:53.767089] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-03. [2025-11-04T17:57:54.084388] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T17:57:54.567069] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-08. [2025-11-04T17:57:54.639731] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-11-04T17:57:55.367059] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-10. [2025-11-04T17:57:56.167018] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-04. [2025-11-04T17:57:56.246299] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-11-04T17:57:56.514717] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-11-04T17:57:56.967043] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-09. [2025-11-04T17:57:56.984973] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T17:57:57.767070] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-11. [2025-11-04T17:57:58.492674] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T17:57:58.493778] [INFO ] [REMOTE ] CDLR: Full fetch complete (14799 records). [2025-11-04T17:57:58.493847] [DEBUG] [VALIDATOR ] CDLR: Validating primary data. [2025-11-04T17:57:58.573768] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-05. [2025-11-04T17:57:58.863185] [DEBUG] [VALIDATOR ] CDLR: All expected trading days are present. [2025-11-04T17:57:58.863281] [DEBUG] [VALIDATOR ] CDLR: Primary data validation passed. [2025-11-04T17:57:58.863733] [INFO ] [MAIN ] CDLR: Fetched primary data is valid. [2025-11-04T17:57:58.873838] [DEBUG] [LOCAL ] CDLR: Saving primary data. [2025-11-04T17:57:58.891990] [DEBUG] [LOCAL ] CDLR: Trimmed 14799 records to 10541 (keeping 200 days of data) [2025-11-04T17:57:58.893963] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-11-04T17:57:58.967216] [DEBUG] [LOCAL ] CDLR: Primary data saved (10541 records). [2025-11-04T17:57:58.967353] [INFO ] [MAIN ] CDLR: Full history saved. [2025-11-04T17:57:58.967466] [INFO ] [MAIN ] Update Progress: 423/2367 symbols completed. CDLR took 32.2s. Estimated remaining: 4h 26m 13s. Rate: 7.3 symbols/min [2025-11-04T17:57:59.155368] [INFO ] [MAIN ] CE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:57:59.204286] [DEBUG] [LOCAL ] CE: Loaded 51118 existing records from primary storage. [2025-11-04T17:57:59.204586] [INFO ] [MAIN ] CE: Loaded 51118 existing records. [2025-11-04T17:57:59.379293] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-11. [2025-11-04T17:57:59.385512] [WARN ] [MAIN ] CE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:57:59.386083] [INFO ] [MAIN ] CE: Wiping existing primary data before full refresh. [2025-11-04T17:57:59.386281] [INFO ] [LOCAL ] CE: Starting deletion of primary data. [2025-11-04T17:57:59.386515] [DEBUG] [LOCAL ] CE: Deleted primary data file. [2025-11-04T17:57:59.386571] [INFO ] [MAIN ] CE: Starting full primary refresh. [2025-11-04T17:57:59.733612] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T17:57:59.810677] [INFO ] [REMOTE ] CE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:00.167104] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-06. [2025-11-04T17:58:00.600186] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T17:58:00.601999] [INFO ] [REMOTE ] CDP: Full fetch complete (62388 records). [2025-11-04T17:58:00.602083] [DEBUG] [VALIDATOR ] CDP: Validating primary data. [2025-11-04T17:58:00.969514] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-01. [2025-11-04T17:58:00.988331] [DEBUG] [VALIDATOR ] CDP: All expected trading days are present. [2025-11-04T17:58:00.988393] [DEBUG] [VALIDATOR ] CDP: Primary data validation passed. [2025-11-04T17:58:00.989100] [INFO ] [MAIN ] CDP: Fetched primary data is valid. [2025-11-04T17:58:00.989162] [DEBUG] [LOCAL ] CDP: Saving primary data. [2025-11-04T17:58:00.990399] [DEBUG] [LOCAL ] CDP: Trimmed 62388 records to 42209 (keeping 200 days of data) [2025-11-04T17:58:01.245445] [DEBUG] [LOCAL ] CDP: Primary data saved (42209 records). [2025-11-04T17:58:01.245594] [INFO ] [MAIN ] CDP: Full history saved. [2025-11-04T17:58:01.245697] [INFO ] [MAIN ] Update Progress: 424/2367 symbols completed. CDP took 28.7s. Estimated remaining: 4h 25m 38s. Rate: 7.3 symbols/min [2025-11-04T17:58:01.443138] [INFO ] [MAIN ] CEE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:58:01.472352] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T17:58:01.483568] [DEBUG] [LOCAL ] CEE: Loaded 4840 existing records from primary storage. [2025-11-04T17:58:01.484154] [INFO ] [MAIN ] CEE: Loaded 4840 existing records. [2025-11-04T17:58:01.694591] [WARN ] [MAIN ] CEE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:58:01.694883] [INFO ] [MAIN ] CEE: Wiping existing primary data before full refresh. [2025-11-04T17:58:01.694957] [INFO ] [LOCAL ] CEE: Starting deletion of primary data. [2025-11-04T17:58:01.695159] [DEBUG] [LOCAL ] CEE: Deleted primary data file. [2025-11-04T17:58:01.695232] [INFO ] [MAIN ] CEE: Starting full primary refresh. [2025-11-04T17:58:01.772192] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-07. [2025-11-04T17:58:02.260803] [INFO ] [REMOTE ] CEE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:02.504604] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-11-04T17:58:02.567083] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-01. [2025-11-04T17:58:02.636261] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-11-04T17:58:03.144429] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-11-04T17:58:03.367020] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-08. [2025-11-04T17:58:04.117143] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T17:58:04.167064] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-02. [2025-11-04T17:58:04.967125] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-09. [2025-11-04T17:58:05.478917] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T17:58:05.730044] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-11-04T17:58:05.767044] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-10. [2025-11-04T17:58:06.567024] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-03. [2025-11-04T17:58:06.707417] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T17:58:07.367039] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-10. [2025-11-04T17:58:07.900593] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T17:58:08.167045] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-04. [2025-11-04T17:58:08.967146] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-02. [2025-11-04T17:58:09.353369] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-11-04T17:58:09.422632] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T17:58:09.646537] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-11-04T17:58:09.767032] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-03. [2025-11-04T17:58:10.510290] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T17:58:10.567007] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-04. [2025-11-04T17:58:11.062069] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-11-04T17:58:11.367050] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-05. [2025-11-04T17:58:11.784768] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-11-04T17:58:12.167038] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-06. [2025-11-04T17:58:12.573669] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-11-04T17:58:12.967036] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-07. [2025-11-04T17:58:13.474279] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-11-04T17:58:13.767049] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-05. [2025-11-04T17:58:14.567054] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-11. [2025-11-04T17:58:15.273866] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-11-04T17:58:15.367002] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-06. [2025-11-04T17:58:16.167042] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-11. [2025-11-04T17:58:16.626668] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-11 (2.06 seconds). [2025-11-04T17:58:16.628724] [INFO ] [REMOTE ] CDE: Full fetch complete (77814 records). [2025-11-04T17:58:16.628851] [DEBUG] [VALIDATOR ] CDE: Validating primary data. [2025-11-04T17:58:16.927124] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-11-04T17:58:16.973742] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-08. [2025-11-04T17:58:17.086490] [DEBUG] [VALIDATOR ] CDE: All expected trading days are present. [2025-11-04T17:58:17.090482] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T17:58:17.090566] [DEBUG] [VALIDATOR ] CDE: Primary data validation passed. [2025-11-04T17:58:17.092852] [INFO ] [MAIN ] CDE: Fetched primary data is valid. [2025-11-04T17:58:17.093100] [DEBUG] [LOCAL ] CDE: Saving primary data. [2025-11-04T17:58:17.093310] [INFO ] [REMOTE ] CDRE: Full fetch complete (34812 records). [2025-11-04T17:58:17.093396] [DEBUG] [VALIDATOR ] CDRE: Validating primary data. [2025-11-04T17:58:17.202940] [DEBUG] [LOCAL ] CDE: Trimmed 77814 records to 53644 (keeping 200 days of data) [2025-11-04T17:58:17.605502] [DEBUG] [VALIDATOR ] CDRE: All expected trading days are present. [2025-11-04T17:58:17.608766] [DEBUG] [VALIDATOR ] CDRE: Primary data validation passed. [2025-11-04T17:58:17.609813] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-11-04T17:58:17.612745] [INFO ] [MAIN ] CDRE: Fetched primary data is valid. [2025-11-04T17:58:17.613364] [DEBUG] [LOCAL ] CDRE: Saving primary data. [2025-11-04T17:58:17.617433] [DEBUG] [LOCAL ] CDRE: Trimmed 34812 records to 24547 (keeping 200 days of data) [2025-11-04T17:58:17.806252] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-07. [2025-11-04T17:58:17.944085] [DEBUG] [LOCAL ] CDRE: Primary data saved (24547 records). [2025-11-04T17:58:17.962007] [INFO ] [MAIN ] CDRE: Full history saved. [2025-11-04T17:58:17.984470] [DEBUG] [LOCAL ] CDE: Primary data saved (53644 records). [2025-11-04T17:58:17.984608] [INFO ] [MAIN ] CDE: Full history saved. [2025-11-04T17:58:17.984761] [INFO ] [MAIN ] Update Progress: 425/2367 symbols completed. CDE took 51.7s. Estimated remaining: 4h 26m 9s. Rate: 7.3 symbols/min [2025-11-04T17:58:17.985048] [INFO ] [MAIN ] Update Progress: 426/2367 symbols completed. CDRE took 31.9s. Estimated remaining: 4h 25m 23s. Rate: 7.3 symbols/min [2025-11-04T17:58:18.322132] [INFO ] [MAIN ] CEPU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:58:18.339365] [INFO ] [MAIN ] CF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:58:18.366671] [DEBUG] [LOCAL ] CEPU: Loaded 27644 existing records from primary storage. [2025-11-04T17:58:18.378852] [INFO ] [MAIN ] CEPU: Loaded 27644 existing records. [2025-11-04T17:58:18.529278] [DEBUG] [LOCAL ] CF: Loaded 51971 existing records from primary storage. [2025-11-04T17:58:18.552101] [INFO ] [MAIN ] CF: Loaded 51971 existing records. [2025-11-04T17:58:18.572593] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-09. [2025-11-04T17:58:18.658045] [WARN ] [MAIN ] CEPU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:58:18.687054] [INFO ] [MAIN ] CEPU: Wiping existing primary data before full refresh. [2025-11-04T17:58:18.698205] [INFO ] [LOCAL ] CEPU: Starting deletion of primary data. [2025-11-04T17:58:18.698502] [DEBUG] [LOCAL ] CEPU: Deleted primary data file. [2025-11-04T17:58:18.709062] [INFO ] [MAIN ] CEPU: Starting full primary refresh. [2025-11-04T17:58:18.866427] [WARN ] [MAIN ] CF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:58:18.877743] [INFO ] [MAIN ] CF: Wiping existing primary data before full refresh. [2025-11-04T17:58:18.885358] [INFO ] [LOCAL ] CF: Starting deletion of primary data. [2025-11-04T17:58:18.915147] [DEBUG] [LOCAL ] CF: Deleted primary data file. [2025-11-04T17:58:18.937962] [INFO ] [MAIN ] CF: Starting full primary refresh. [2025-11-04T17:58:19.133425] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T17:58:19.293813] [INFO ] [REMOTE ] CEPU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:19.367015] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-10. [2025-11-04T17:58:19.534664] [INFO ] [REMOTE ] CF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:19.564764] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-11-04T17:58:19.822204] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-11-04T17:58:20.167037] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-08. [2025-11-04T17:58:20.967063] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-11. [2025-11-04T17:58:21.549196] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-11 (0.58 seconds). [2025-11-04T17:58:21.549941] [INFO ] [REMOTE ] CEE: Full fetch complete (9902 records). [2025-11-04T17:58:21.550008] [DEBUG] [VALIDATOR ] CEE: Validating primary data. [2025-11-04T17:58:21.770306] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-01. [2025-11-04T17:58:21.913401] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-11-04T17:58:21.994539] [DEBUG] [VALIDATOR ] CEE: All expected trading days are present. [2025-11-04T17:58:21.994620] [DEBUG] [VALIDATOR ] CEE: Primary data validation passed. [2025-11-04T17:58:21.994946] [INFO ] [MAIN ] CEE: Fetched primary data is valid. [2025-11-04T17:58:21.995009] [DEBUG] [LOCAL ] CEE: Saving primary data. [2025-11-04T17:58:21.995409] [DEBUG] [LOCAL ] CEE: Trimmed 9902 records to 4838 (keeping 200 days of data) [2025-11-04T17:58:22.023151] [DEBUG] [LOCAL ] CEE: Primary data saved (4838 records). [2025-11-04T17:58:22.023223] [INFO ] [MAIN ] CEE: Full history saved. [2025-11-04T17:58:22.023296] [INFO ] [MAIN ] Update Progress: 427/2367 symbols completed. CEE took 20.8s. Estimated remaining: 4h 24m 56s. Rate: 7.3 symbols/min [2025-11-04T17:58:22.190692] [INFO ] [MAIN ] CFG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:58:22.239768] [DEBUG] [LOCAL ] CFG: Loaded 53622 existing records from primary storage. [2025-11-04T17:58:22.240066] [INFO ] [MAIN ] CFG: Loaded 53622 existing records. [2025-11-04T17:58:22.421737] [WARN ] [MAIN ] CFG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:58:22.421989] [INFO ] [MAIN ] CFG: Wiping existing primary data before full refresh. [2025-11-04T17:58:22.422036] [INFO ] [LOCAL ] CFG: Starting deletion of primary data. [2025-11-04T17:58:22.422215] [DEBUG] [LOCAL ] CFG: Deleted primary data file. [2025-11-04T17:58:22.422253] [INFO ] [MAIN ] CFG: Starting full primary refresh. [2025-11-04T17:58:22.577835] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-01. [2025-11-04T17:58:22.781925] [INFO ] [REMOTE ] CFG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:23.367078] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-09. [2025-11-04T17:58:23.461886] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-11-04T17:58:23.961902] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T17:58:24.167058] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-02. [2025-11-04T17:58:24.474446] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T17:58:24.967055] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-01. [2025-11-04T17:58:25.026070] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-11-04T17:58:25.767077] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-03. [2025-11-04T17:58:26.461371] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-11-04T17:58:26.567022] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-02. [2025-11-04T17:58:26.625333] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-11-04T17:58:27.367050] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-04. [2025-11-04T17:58:27.801425] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T17:58:28.167031] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-03. [2025-11-04T17:58:28.200725] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-11-04T17:58:28.967048] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-10. [2025-11-04T17:58:29.533880] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T17:58:29.767043] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-02. [2025-11-04T17:58:30.374398] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T17:58:30.567009] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-04. [2025-11-04T17:58:31.051454] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T17:58:31.367042] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-05. [2025-11-04T17:58:32.072284] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T17:58:32.167049] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-03. [2025-11-04T17:58:32.323302] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T17:58:32.967111] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-06. [2025-11-04T17:58:33.587469] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T17:58:33.633749] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-11-04T17:58:33.767049] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-04. [2025-11-04T17:58:34.567063] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-07. [2025-11-04T17:58:35.182549] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T17:58:35.367047] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-11. [2025-11-04T17:58:35.429708] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-11-04T17:58:36.167041] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-05. [2025-11-04T17:58:36.690217] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-11 (1.32 seconds). [2025-11-04T17:58:36.692191] [INFO ] [REMOTE ] CE: Full fetch complete (74812 records). [2025-11-04T17:58:36.692301] [DEBUG] [VALIDATOR ] CE: Validating primary data. [2025-11-04T17:58:36.973934] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-08. [2025-11-04T17:58:37.084273] [DEBUG] [VALIDATOR ] CE: All expected trading days are present. [2025-11-04T17:58:37.084360] [DEBUG] [VALIDATOR ] CE: Primary data validation passed. [2025-11-04T17:58:37.085248] [INFO ] [MAIN ] CE: Fetched primary data is valid. [2025-11-04T17:58:37.085318] [DEBUG] [LOCAL ] CE: Saving primary data. [2025-11-04T17:58:37.086776] [DEBUG] [LOCAL ] CE: Trimmed 74812 records to 51119 (keeping 200 days of data) [2025-11-04T17:58:37.402565] [DEBUG] [LOCAL ] CE: Primary data saved (51119 records). [2025-11-04T17:58:37.402832] [INFO ] [MAIN ] CE: Full history saved. [2025-11-04T17:58:37.402932] [INFO ] [MAIN ] Update Progress: 428/2367 symbols completed. CE took 38.4s. Estimated remaining: 4h 25m 20s. Rate: 7.3 symbols/min [2025-11-04T17:58:37.577141] [INFO ] [MAIN ] CFND: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:58:37.577383] [DEBUG] [LOCAL ] CFND: No existing data file found in primary storage. [2025-11-04T17:58:37.577452] [WARN ] [MAIN ] CFND: No existing data found, performing full refresh instead. [2025-11-04T17:58:37.577666] [INFO ] [MAIN ] CFND: Wiping existing primary data before full refresh. [2025-11-04T17:58:37.577739] [INFO ] [LOCAL ] CFND: Starting deletion of primary data. [2025-11-04T17:58:37.577803] [DEBUG] [LOCAL ] CFND: No primary data file found to delete. [2025-11-04T17:58:37.578065] [INFO ] [MAIN ] CFND: Starting full primary refresh. [2025-11-04T17:58:37.773372] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-05. [2025-11-04T17:58:37.963578] [INFO ] [REMOTE ] CFND: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:37.996128] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-11-04T17:58:37.998378] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-11-04T17:58:38.567086] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-06. [2025-11-04T17:58:39.246764] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-11-04T17:58:39.367013] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-06. [2025-11-04T17:58:39.751854] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T17:58:40.167062] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-07. [2025-11-04T17:58:40.494907] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T17:58:40.967091] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-01. [2025-11-04T17:58:41.101057] [DEBUG] [REMOTE ] CFND: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T17:58:41.101365] [WARN ] [REMOTE ] CFND: No data returned for month 2025-01 [2025-11-04T17:58:41.101451] [FATAL] [REMOTE ] CFND: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T17:58:41.101496] [FATAL] [MAIN ] CFND: Failed to fetch primary data. [2025-11-04T17:58:41.101530] [WARN ] [MAIN ] CFND: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T17:58:41.101566] [INFO ] [LOCAL ] CFND: Starting deletion of primary data. [2025-11-04T17:58:41.101619] [DEBUG] [LOCAL ] CFND: No primary data file found to delete. [2025-11-04T17:58:41.101680] [INFO ] [LOCAL ] CFND: Starting deletion of derivative data. [2025-11-04T17:58:41.101744] [DEBUG] [LOCAL ] CFND: No derivative data file found to delete. [2025-11-04T17:58:41.101789] [INFO ] [MAIN ] Update Progress: 429/2367 symbols completed. CFND took 3.7s. Estimated remaining: 4h 24m 52s. Rate: 7.3 symbols/min [2025-11-04T17:58:41.276698] [INFO ] [MAIN ] CFR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:58:41.311058] [DEBUG] [LOCAL ] CFR: Loaded 32116 existing records from primary storage. [2025-11-04T17:58:41.311312] [INFO ] [MAIN ] CFR: Loaded 32116 existing records. [2025-11-04T17:58:41.493869] [WARN ] [MAIN ] CFR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:58:41.494098] [INFO ] [MAIN ] CFR: Wiping existing primary data before full refresh. [2025-11-04T17:58:41.494234] [INFO ] [LOCAL ] CFR: Starting deletion of primary data. [2025-11-04T17:58:41.494500] [DEBUG] [LOCAL ] CFR: Deleted primary data file. [2025-11-04T17:58:41.494550] [INFO ] [MAIN ] CFR: Starting full primary refresh. [2025-11-04T17:58:41.699675] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T17:58:41.777223] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-09. [2025-11-04T17:58:41.899104] [INFO ] [REMOTE ] CFR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:42.567055] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-01. [2025-11-04T17:58:42.773711] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T17:58:43.367038] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-10. [2025-11-04T17:58:43.511457] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-11-04T17:58:44.167040] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-07. [2025-11-04T17:58:44.385365] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T17:58:44.967085] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-02. [2025-11-04T17:58:45.493346] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T17:58:45.767017] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-08. [2025-11-04T17:58:45.905785] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-11-04T17:58:46.567049] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-08. [2025-11-04T17:58:46.998229] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T17:58:47.367020] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-09. [2025-11-04T17:58:47.909903] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T17:58:48.167025] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-03. [2025-11-04T17:58:48.719183] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T17:58:48.967097] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-09. [2025-11-04T17:58:49.197269] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T17:58:49.767093] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-11. [2025-11-04T17:58:50.227687] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T17:58:50.567091] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-04. [2025-11-04T17:58:50.885551] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T17:58:50.887085] [INFO ] [REMOTE ] CEPU: Full fetch complete (40026 records). [2025-11-04T17:58:50.887163] [DEBUG] [VALIDATOR ] CEPU: Validating primary data. [2025-11-04T17:58:51.252404] [DEBUG] [VALIDATOR ] CEPU: All expected trading days are present. [2025-11-04T17:58:51.252484] [DEBUG] [VALIDATOR ] CEPU: Primary data validation passed. [2025-11-04T17:58:51.253055] [INFO ] [MAIN ] CEPU: Fetched primary data is valid. [2025-11-04T17:58:51.253168] [DEBUG] [LOCAL ] CEPU: Saving primary data. [2025-11-04T17:58:51.254226] [DEBUG] [LOCAL ] CEPU: Trimmed 40026 records to 27804 (keeping 200 days of data) [2025-11-04T17:58:51.368156] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-10. [2025-11-04T17:58:51.416156] [DEBUG] [LOCAL ] CEPU: Primary data saved (27804 records). [2025-11-04T17:58:51.416401] [INFO ] [MAIN ] CEPU: Full history saved. [2025-11-04T17:58:51.416490] [INFO ] [MAIN ] Update Progress: 430/2367 symbols completed. CEPU took 33.4s. Estimated remaining: 4h 24m 53s. Rate: 7.3 symbols/min [2025-11-04T17:58:51.593420] [INFO ] [MAIN ] CGAU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:58:51.616779] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-11-04T17:58:51.660782] [DEBUG] [LOCAL ] CGAU: Loaded 44049 existing records from primary storage. [2025-11-04T17:58:51.661084] [INFO ] [MAIN ] CGAU: Loaded 44049 existing records. [2025-11-04T17:58:51.835356] [WARN ] [MAIN ] CGAU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:58:51.835558] [INFO ] [MAIN ] CGAU: Wiping existing primary data before full refresh. [2025-11-04T17:58:51.835605] [INFO ] [LOCAL ] CGAU: Starting deletion of primary data. [2025-11-04T17:58:51.835811] [DEBUG] [LOCAL ] CGAU: Deleted primary data file. [2025-11-04T17:58:51.835864] [INFO ] [MAIN ] CGAU: Starting full primary refresh. [2025-11-04T17:58:52.177949] [INFO ] [REMOTE ] CGAU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:52.173436] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-05. [2025-11-04T17:58:52.632490] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-11-04T17:58:52.967086] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-10. [2025-11-04T17:58:53.059094] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T17:58:53.767089] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-11. [2025-11-04T17:58:54.105868] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T17:58:54.108034] [INFO ] [REMOTE ] CF: Full fetch complete (75517 records). [2025-11-04T17:58:54.108184] [DEBUG] [VALIDATOR ] CF: Validating primary data. [2025-11-04T17:58:54.529195] [DEBUG] [VALIDATOR ] CF: All expected trading days are present. [2025-11-04T17:58:54.529338] [DEBUG] [VALIDATOR ] CF: Primary data validation passed. [2025-11-04T17:58:54.530198] [INFO ] [MAIN ] CF: Fetched primary data is valid. [2025-11-04T17:58:54.530308] [DEBUG] [LOCAL ] CF: Saving primary data. [2025-11-04T17:58:54.532203] [DEBUG] [LOCAL ] CF: Trimmed 75517 records to 51978 (keeping 200 days of data) [2025-11-04T17:58:54.614999] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-01. [2025-11-04T17:58:54.657768] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T17:58:54.894034] [DEBUG] [LOCAL ] CF: Primary data saved (51978 records). [2025-11-04T17:58:54.894196] [INFO ] [MAIN ] CF: Full history saved. [2025-11-04T17:58:54.895171] [INFO ] [MAIN ] Update Progress: 431/2367 symbols completed. CF took 36.9s. Estimated remaining: 4h 24m 24s. Rate: 7.3 symbols/min [2025-11-04T17:58:55.065292] [INFO ] [MAIN ] CGV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:58:55.069168] [DEBUG] [LOCAL ] CGV: Loaded 1352 existing records from primary storage. [2025-11-04T17:58:55.069304] [INFO ] [MAIN ] CGV: Loaded 1352 existing records. [2025-11-04T17:58:55.247425] [WARN ] [MAIN ] CGV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:58:55.247720] [INFO ] [MAIN ] CGV: Wiping existing primary data before full refresh. [2025-11-04T17:58:55.247776] [INFO ] [LOCAL ] CGV: Starting deletion of primary data. [2025-11-04T17:58:55.247916] [DEBUG] [LOCAL ] CGV: Deleted primary data file. [2025-11-04T17:58:55.247972] [INFO ] [MAIN ] CGV: Starting full primary refresh. [2025-11-04T17:58:55.372247] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-11. [2025-11-04T17:58:55.609329] [INFO ] [REMOTE ] CGV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:55.884283] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T17:58:56.167087] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-02. [2025-11-04T17:58:56.765146] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-11 (1.39 seconds). [2025-11-04T17:58:56.767166] [INFO ] [REMOTE ] CFG: Full fetch complete (77782 records). [2025-11-04T17:58:56.767264] [DEBUG] [VALIDATOR ] CFG: Validating primary data. [2025-11-04T17:58:56.981887] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-01. [2025-11-04T17:58:57.168609] [DEBUG] [VALIDATOR ] CFG: All expected trading days are present. [2025-11-04T17:58:57.168788] [DEBUG] [VALIDATOR ] CFG: Primary data validation passed. [2025-11-04T17:58:57.169686] [INFO ] [MAIN ] CFG: Fetched primary data is valid. [2025-11-04T17:58:57.169777] [DEBUG] [LOCAL ] CFG: Saving primary data. [2025-11-04T17:58:57.192013] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T17:58:57.199889] [DEBUG] [LOCAL ] CFG: Trimmed 77782 records to 53622 (keeping 200 days of data) [2025-11-04T17:58:57.525002] [DEBUG] [LOCAL ] CFG: Primary data saved (53622 records). [2025-11-04T17:58:57.525141] [INFO ] [MAIN ] CFG: Full history saved. [2025-11-04T17:58:57.525228] [INFO ] [MAIN ] Update Progress: 432/2367 symbols completed. CFG took 35.5s. Estimated remaining: 4h 23m 50s. Rate: 7.3 symbols/min [2025-11-04T17:58:57.635406] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-11-04T17:58:57.700435] [INFO ] [MAIN ] CHCT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:58:57.724481] [DEBUG] [LOCAL ] CHCT: Loaded 23907 existing records from primary storage. [2025-11-04T17:58:57.724772] [INFO ] [MAIN ] CHCT: Loaded 23907 existing records. [2025-11-04T17:58:57.776637] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-02. [2025-11-04T17:58:57.905603] [WARN ] [MAIN ] CHCT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:58:57.905837] [INFO ] [MAIN ] CHCT: Wiping existing primary data before full refresh. [2025-11-04T17:58:57.905884] [INFO ] [LOCAL ] CHCT: Starting deletion of primary data. [2025-11-04T17:58:57.906097] [DEBUG] [LOCAL ] CHCT: Deleted primary data file. [2025-11-04T17:58:57.906151] [INFO ] [MAIN ] CHCT: Starting full primary refresh. [2025-11-04T17:58:58.176835] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-11-04T17:58:58.264445] [INFO ] [REMOTE ] CHCT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:58:58.567091] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-01. [2025-11-04T17:58:59.367093] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-03. [2025-11-04T17:58:59.428126] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T17:58:59.704436] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-11-04T17:59:00.167057] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-06. [2025-11-04T17:59:00.967074] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-04. [2025-11-04T17:59:01.143526] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T17:59:01.396433] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-11-04T17:59:01.767071] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-05. [2025-11-04T17:59:02.108524] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-05 (0.34 seconds). [2025-11-04T17:59:02.567045] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-06. [2025-11-04T17:59:02.855627] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-06 (0.29 seconds). [2025-11-04T17:59:03.367076] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-03. [2025-11-04T17:59:04.167077] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-02. [2025-11-04T17:59:04.433536] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T17:59:04.955035] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T17:59:04.966991] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-03. [2025-11-04T17:59:05.767077] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-04. [2025-11-04T17:59:05.790431] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-11-04T17:59:06.567051] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-07. [2025-11-04T17:59:06.858490] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-07 (0.29 seconds). [2025-11-04T17:59:07.016333] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T17:59:07.367151] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-04. [2025-11-04T17:59:08.137202] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-11-04T17:59:08.167017] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-05. [2025-11-04T17:59:08.919057] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-11-04T17:59:08.967017] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-06. [2025-11-04T17:59:09.767018] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-07. [2025-11-04T17:59:09.926525] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-11-04T17:59:10.567487] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-08. [2025-11-04T17:59:10.777230] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T17:59:10.945061] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-11-04T17:59:11.367055] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-08. [2025-11-04T17:59:12.167045] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-05. [2025-11-04T17:59:12.475259] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T17:59:12.967044] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-09. [2025-11-04T17:59:13.210555] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-09 (0.24 seconds). [2025-11-04T17:59:13.393130] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T17:59:13.767030] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-10. [2025-11-04T17:59:14.079796] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-10 (0.31 seconds). [2025-11-04T17:59:14.567033] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-06. [2025-11-04T17:59:15.367068] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-09. [2025-11-04T17:59:15.638600] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T17:59:16.167017] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-11. [2025-11-04T17:59:16.298096] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-11-04T17:59:16.577089] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T17:59:16.577744] [INFO ] [REMOTE ] CGV: Full fetch complete (2185 records). [2025-11-04T17:59:16.577811] [DEBUG] [VALIDATOR ] CGV: Validating primary data. [2025-11-04T17:59:16.928350] [DEBUG] [VALIDATOR ] CGV: All expected trading days are present. [2025-11-04T17:59:16.928424] [DEBUG] [VALIDATOR ] CGV: Primary data validation passed. [2025-11-04T17:59:16.928512] [INFO ] [MAIN ] CGV: Fetched primary data is valid. [2025-11-04T17:59:16.928548] [DEBUG] [LOCAL ] CGV: Saving primary data. [2025-11-04T17:59:16.928890] [DEBUG] [LOCAL ] CGV: Trimmed 2185 records to 1350 (keeping 200 days of data) [2025-11-04T17:59:16.937208] [DEBUG] [LOCAL ] CGV: Primary data saved (1350 records). [2025-11-04T17:59:16.937265] [INFO ] [MAIN ] CGV: Full history saved. [2025-11-04T17:59:16.937328] [INFO ] [MAIN ] Update Progress: 433/2367 symbols completed. CGV took 22.0s. Estimated remaining: 4h 24m 32s. Rate: 7.3 symbols/min [2025-11-04T17:59:16.968634] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-07. [2025-11-04T17:59:17.107369] [INFO ] [MAIN ] CHD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:59:17.156772] [DEBUG] [LOCAL ] CHD: Loaded 52362 existing records from primary storage. [2025-11-04T17:59:17.157088] [INFO ] [MAIN ] CHD: Loaded 52362 existing records. [2025-11-04T17:59:17.336296] [WARN ] [MAIN ] CHD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:59:17.336510] [INFO ] [MAIN ] CHD: Wiping existing primary data before full refresh. [2025-11-04T17:59:17.336556] [INFO ] [LOCAL ] CHD: Starting deletion of primary data. [2025-11-04T17:59:17.336767] [DEBUG] [LOCAL ] CHD: Deleted primary data file. [2025-11-04T17:59:17.336842] [INFO ] [MAIN ] CHD: Starting full primary refresh. [2025-11-04T17:59:17.718500] [INFO ] [REMOTE ] CHD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:59:17.766995] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-10. [2025-11-04T17:59:17.903927] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-11-04T17:59:18.567076] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-08. [2025-11-04T17:59:18.809089] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-11-04T17:59:19.367015] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-07. [2025-11-04T17:59:19.406826] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-11-04T17:59:20.167058] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-11. [2025-11-04T17:59:20.550687] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T17:59:20.967009] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-09. [2025-11-04T17:59:21.055126] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T17:59:21.056591] [INFO ] [REMOTE ] CFR: Full fetch complete (48673 records). [2025-11-04T17:59:21.056692] [DEBUG] [VALIDATOR ] CFR: Validating primary data. [2025-11-04T17:59:21.440169] [DEBUG] [VALIDATOR ] CFR: All expected trading days are present. [2025-11-04T17:59:21.440247] [DEBUG] [VALIDATOR ] CFR: Primary data validation passed. [2025-11-04T17:59:21.440834] [INFO ] [MAIN ] CFR: Fetched primary data is valid. [2025-11-04T17:59:21.440895] [DEBUG] [LOCAL ] CFR: Saving primary data. [2025-11-04T17:59:21.441881] [DEBUG] [LOCAL ] CFR: Trimmed 48673 records to 32116 (keeping 200 days of data) [2025-11-04T17:59:21.638546] [DEBUG] [LOCAL ] CFR: Primary data saved (32116 records). [2025-11-04T17:59:21.638743] [INFO ] [MAIN ] CFR: Full history saved. [2025-11-04T17:59:21.638828] [INFO ] [MAIN ] Update Progress: 434/2367 symbols completed. CFR took 40.5s. Estimated remaining: 4h 24m 9s. Rate: 7.3 symbols/min [2025-11-04T17:59:21.778005] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-01. [2025-11-04T17:59:21.809127] [INFO ] [MAIN ] CHE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:59:21.827925] [DEBUG] [LOCAL ] CHE: Loaded 15893 existing records from primary storage. [2025-11-04T17:59:21.828184] [INFO ] [MAIN ] CHE: Loaded 15893 existing records. [2025-11-04T17:59:22.012619] [WARN ] [MAIN ] CHE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:59:22.012986] [INFO ] [MAIN ] CHE: Wiping existing primary data before full refresh. [2025-11-04T17:59:22.013099] [INFO ] [LOCAL ] CHE: Starting deletion of primary data. [2025-11-04T17:59:22.018500] [DEBUG] [LOCAL ] CHE: Deleted primary data file. [2025-11-04T17:59:22.025577] [INFO ] [MAIN ] CHE: Starting full primary refresh. [2025-11-04T17:59:22.058498] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-11-04T17:59:22.384368] [INFO ] [REMOTE ] CHE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:59:22.567039] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-10. [2025-11-04T17:59:23.251978] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T17:59:23.366998] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-02. [2025-11-04T17:59:23.414003] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-11-04T17:59:24.167075] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-01. [2025-11-04T17:59:24.570829] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T17:59:24.884358] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-11-04T17:59:24.967018] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-03. [2025-11-04T17:59:25.767054] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-11. [2025-11-04T17:59:26.263389] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T17:59:26.492600] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T17:59:26.494201] [INFO ] [REMOTE ] CHCT: Full fetch complete (34780 records). [2025-11-04T17:59:26.494284] [DEBUG] [VALIDATOR ] CHCT: Validating primary data. [2025-11-04T17:59:26.572229] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-02. [2025-11-04T17:59:26.891006] [DEBUG] [VALIDATOR ] CHCT: All expected trading days are present. [2025-11-04T17:59:26.891088] [DEBUG] [VALIDATOR ] CHCT: Primary data validation passed. [2025-11-04T17:59:26.891607] [INFO ] [MAIN ] CHCT: Fetched primary data is valid. [2025-11-04T17:59:26.891682] [DEBUG] [LOCAL ] CHCT: Saving primary data. [2025-11-04T17:59:26.892442] [DEBUG] [LOCAL ] CHCT: Trimmed 34780 records to 23919 (keeping 200 days of data) [2025-11-04T17:59:27.029021] [DEBUG] [LOCAL ] CHCT: Primary data saved (23919 records). [2025-11-04T17:59:27.029147] [INFO ] [MAIN ] CHCT: Full history saved. [2025-11-04T17:59:27.029223] [INFO ] [MAIN ] Update Progress: 435/2367 symbols completed. CHCT took 29.5s. Estimated remaining: 4h 23m 48s. Rate: 7.3 symbols/min [2025-11-04T17:59:27.200496] [INFO ] [MAIN ] CHGG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:59:27.210043] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T17:59:27.253412] [DEBUG] [LOCAL ] CHGG: Loaded 47575 existing records from primary storage. [2025-11-04T17:59:27.253773] [INFO ] [MAIN ] CHGG: Loaded 47575 existing records. [2025-11-04T17:59:27.372206] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-04. [2025-11-04T17:59:27.438751] [WARN ] [MAIN ] CHGG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:59:27.439026] [INFO ] [MAIN ] CHGG: Wiping existing primary data before full refresh. [2025-11-04T17:59:27.439078] [INFO ] [LOCAL ] CHGG: Starting deletion of primary data. [2025-11-04T17:59:27.439306] [DEBUG] [LOCAL ] CHGG: Deleted primary data file. [2025-11-04T17:59:27.439347] [INFO ] [MAIN ] CHGG: Starting full primary refresh. [2025-11-04T17:59:27.841792] [INFO ] [REMOTE ] CHGG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:59:28.167046] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-01. [2025-11-04T17:59:28.765814] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T17:59:28.967093] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-05. [2025-11-04T17:59:29.566627] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T17:59:29.767048] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-02. [2025-11-04T17:59:30.537589] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-11-04T17:59:30.567001] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-06. [2025-11-04T17:59:30.985870] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T17:59:31.367081] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-03. [2025-11-04T17:59:31.957885] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T17:59:32.167083] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-03. [2025-11-04T17:59:32.787631] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T17:59:32.805295] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T17:59:32.967013] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-04. [2025-11-04T17:59:33.767071] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-07. [2025-11-04T17:59:34.361675] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T17:59:34.567014] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-05. [2025-11-04T17:59:35.023239] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T17:59:35.367062] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-04. [2025-11-04T17:59:35.802928] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-11-04T17:59:36.167049] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-08. [2025-11-04T17:59:36.203848] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-11-04T17:59:36.967067] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-06. [2025-11-04T17:59:37.450308] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-11-04T17:59:37.767098] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-08. [2025-11-04T17:59:38.486826] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-11-04T17:59:38.567098] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-07. [2025-11-04T17:59:39.008938] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T17:59:39.367056] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-05. [2025-11-04T17:59:39.913225] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T17:59:39.933396] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-11-04T17:59:40.167040] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-08. [2025-11-04T17:59:40.967052] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-09. [2025-11-04T17:59:41.381302] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T17:59:41.767018] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-09. [2025-11-04T17:59:42.182364] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T17:59:42.567016] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-09. [2025-11-04T17:59:43.249063] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-11-04T17:59:43.367007] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-10. [2025-11-04T17:59:43.857752] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T17:59:44.167060] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-10. [2025-11-04T17:59:44.719968] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T17:59:44.967036] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-06. [2025-11-04T17:59:45.570167] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T17:59:45.590591] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-11-04T17:59:45.767007] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-11. [2025-11-04T17:59:46.567053] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-07. [2025-11-04T17:59:47.006085] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T17:59:47.008080] [INFO ] [REMOTE ] CGAU: Full fetch complete (62905 records). [2025-11-04T17:59:47.008173] [DEBUG] [VALIDATOR ] CGAU: Validating primary data. [2025-11-04T17:59:47.372603] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-11. [2025-11-04T17:59:47.394095] [DEBUG] [VALIDATOR ] CGAU: All expected trading days are present. [2025-11-04T17:59:47.394206] [DEBUG] [VALIDATOR ] CGAU: Primary data validation passed. [2025-11-04T17:59:47.395033] [INFO ] [MAIN ] CGAU: Fetched primary data is valid. [2025-11-04T17:59:47.395096] [DEBUG] [LOCAL ] CGAU: Saving primary data. [2025-11-04T17:59:47.396369] [DEBUG] [LOCAL ] CGAU: Trimmed 62905 records to 44142 (keeping 200 days of data) [2025-11-04T17:59:47.522844] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T17:59:47.664917] [DEBUG] [LOCAL ] CGAU: Primary data saved (44142 records). [2025-11-04T17:59:47.665048] [INFO ] [MAIN ] CGAU: Full history saved. [2025-11-04T17:59:47.665127] [INFO ] [MAIN ] Update Progress: 436/2367 symbols completed. CGAU took 56.2s. Estimated remaining: 4h 24m 35s. Rate: 7.3 symbols/min [2025-11-04T17:59:47.832001] [INFO ] [MAIN ] CHH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T17:59:47.866406] [DEBUG] [LOCAL ] CHH: Loaded 32066 existing records from primary storage. [2025-11-04T17:59:47.866726] [INFO ] [MAIN ] CHH: Loaded 32066 existing records. [2025-11-04T17:59:48.040353] [WARN ] [MAIN ] CHH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T17:59:48.040581] [INFO ] [MAIN ] CHH: Wiping existing primary data before full refresh. [2025-11-04T17:59:48.040627] [INFO ] [LOCAL ] CHH: Starting deletion of primary data. [2025-11-04T17:59:48.040849] [DEBUG] [LOCAL ] CHH: Deleted primary data file. [2025-11-04T17:59:48.040905] [INFO ] [MAIN ] CHH: Starting full primary refresh. [2025-11-04T17:59:48.176751] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-08. [2025-11-04T17:59:48.392517] [INFO ] [REMOTE ] CHH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T17:59:48.626130] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T17:59:48.627920] [INFO ] [REMOTE ] CHGG: Full fetch complete (69586 records). [2025-11-04T17:59:48.628023] [DEBUG] [VALIDATOR ] CHGG: Validating primary data. [2025-11-04T17:59:48.973780] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-01. [2025-11-04T17:59:49.028440] [DEBUG] [VALIDATOR ] CHGG: All expected trading days are present. [2025-11-04T17:59:49.029053] [DEBUG] [VALIDATOR ] CHGG: Primary data validation passed. [2025-11-04T17:59:49.029843] [INFO ] [MAIN ] CHGG: Fetched primary data is valid. [2025-11-04T17:59:49.030333] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-11-04T17:59:49.030541] [DEBUG] [LOCAL ] CHGG: Saving primary data. [2025-11-04T17:59:49.035849] [DEBUG] [LOCAL ] CHGG: Trimmed 69586 records to 47471 (keeping 200 days of data) [2025-11-04T17:59:49.301206] [DEBUG] [LOCAL ] CHGG: Primary data saved (47471 records). [2025-11-04T17:59:49.301328] [INFO ] [MAIN ] CHGG: Full history saved. [2025-11-04T17:59:49.301403] [INFO ] [MAIN ] Update Progress: 437/2367 symbols completed. CHGG took 22.3s. Estimated remaining: 4h 23m 58s. Rate: 7.3 symbols/min [2025-11-04T17:59:49.767050] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-09. [2025-11-04T17:59:50.209603] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-11-04T17:59:50.448053] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T17:59:50.567078] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-10. [2025-11-04T17:59:51.367047] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-02. [2025-11-04T17:59:51.389947] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T17:59:52.167068] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-11. [2025-11-04T17:59:52.285927] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-11-04T17:59:52.967115] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-03. [2025-11-04T17:59:53.175703] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-11 (1.01 seconds). [2025-11-04T17:59:53.177768] [INFO ] [REMOTE ] CHE: Full fetch complete (21369 records). [2025-11-04T17:59:53.177868] [DEBUG] [VALIDATOR ] CHE: Validating primary data. [2025-11-04T17:59:53.576894] [DEBUG] [VALIDATOR ] CHE: All expected trading days are present. [2025-11-04T17:59:53.576972] [DEBUG] [VALIDATOR ] CHE: Primary data validation passed. [2025-11-04T17:59:53.577363] [INFO ] [MAIN ] CHE: Fetched primary data is valid. [2025-11-04T17:59:53.577404] [DEBUG] [LOCAL ] CHE: Saving primary data. [2025-11-04T17:59:53.578009] [DEBUG] [LOCAL ] CHE: Trimmed 21369 records to 15995 (keeping 200 days of data) [2025-11-04T17:59:53.678388] [DEBUG] [LOCAL ] CHE: Primary data saved (15995 records). [2025-11-04T17:59:53.678551] [INFO ] [MAIN ] CHE: Full history saved. [2025-11-04T17:59:53.678669] [INFO ] [MAIN ] Update Progress: 438/2367 symbols completed. CHE took 32.0s. Estimated remaining: 4h 23m 32s. Rate: 7.3 symbols/min [2025-11-04T17:59:53.767032] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-10. [2025-11-04T17:59:54.702636] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-11-04T17:59:54.710210] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-04. [2025-11-04T17:59:55.891276] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-11-04T17:59:55.904182] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-11. [2025-11-04T17:59:55.927497] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T17:59:56.167125] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-05. [2025-11-04T17:59:57.138045] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T17:59:57.141290] [INFO ] [REMOTE ] CHD: Full fetch complete (75112 records). [2025-11-04T17:59:57.141498] [DEBUG] [VALIDATOR ] CHD: Validating primary data. [2025-11-04T17:59:57.665150] [DEBUG] [VALIDATOR ] CHD: All expected trading days are present. [2025-11-04T17:59:57.665353] [DEBUG] [VALIDATOR ] CHD: Primary data validation passed. [2025-11-04T17:59:57.666415] [INFO ] [MAIN ] CHD: Fetched primary data is valid. [2025-11-04T17:59:57.666501] [DEBUG] [LOCAL ] CHD: Saving primary data. [2025-11-04T17:59:57.691103] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-11-04T17:59:57.701065] [DEBUG] [LOCAL ] CHD: Trimmed 75112 records to 52373 (keeping 200 days of data) [2025-11-04T17:59:57.702742] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-06. [2025-11-04T17:59:58.021173] [DEBUG] [LOCAL ] CHD: Primary data saved (52373 records). [2025-11-04T17:59:58.021314] [INFO ] [MAIN ] CHD: Full history saved. [2025-11-04T17:59:58.022139] [INFO ] [MAIN ] Update Progress: 439/2367 symbols completed. CHD took 41.1s. Estimated remaining: 4h 23m 7s. Rate: 7.3 symbols/min [2025-11-04T17:59:58.568896] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T17:59:58.576019] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-07. [2025-11-04T17:59:59.602926] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T17:59:59.610892] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-08. [2025-11-04T18:00:00.835923] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T18:00:00.842730] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-09. [2025-11-04T18:00:02.129509] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T18:00:02.136535] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-10. [2025-11-04T18:00:03.206284] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T18:00:03.214816] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-11. [2025-11-04T18:00:04.607176] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-11 (1.39 seconds). [2025-11-04T18:00:04.608450] [INFO ] [REMOTE ] CHH: Full fetch complete (47501 records). [2025-11-04T18:00:04.608552] [DEBUG] [VALIDATOR ] CHH: Validating primary data. [2025-11-04T18:00:04.609130] [FATAL] [VALIDATOR ] CHH: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:00:04.609232] [FATAL] [MAIN ] CHH: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:00:04.609340] [WARN ] [MAIN ] CHH: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:00:04.609431] [INFO ] [LOCAL ] CHH: Starting deletion of primary data. [2025-11-04T18:00:04.609511] [DEBUG] [LOCAL ] CHH: No primary data file found to delete. [2025-11-04T18:00:04.609547] [INFO ] [LOCAL ] CHH: Starting deletion of derivative data. [2025-11-04T18:00:04.609752] [DEBUG] [LOCAL ] CHH: Deleted derivative data file. [2025-11-04T18:00:04.609839] [INFO ] [MAIN ] Update Progress: 440/2367 symbols completed. CHH took 16.9s. Estimated remaining: 4h 22m 52s. Rate: 7.3 symbols/min [2025-11-04T18:00:05.208950] [INFO ] [MAIN ] CHMI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:00:05.330476] [INFO ] [MAIN ] CHT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:00:05.362340] [INFO ] [MAIN ] CHPT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:00:05.416315] [INFO ] [MAIN ] CHWY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:00:05.441450] [DEBUG] [LOCAL ] CHMI: Loaded 20472 existing records from primary storage. [2025-11-04T18:00:05.442331] [DEBUG] [LOCAL ] CHT: Loaded 18698 existing records from primary storage. [2025-11-04T18:00:05.442829] [INFO ] [MAIN ] CHMI: Loaded 20472 existing records. [2025-11-04T18:00:05.454137] [INFO ] [MAIN ] CHT: Loaded 18698 existing records. [2025-11-04T18:00:05.800238] [WARN ] [MAIN ] CHMI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:00:05.807078] [DEBUG] [LOCAL ] CHPT: Loaded 45583 existing records from primary storage. [2025-11-04T18:00:05.844050] [INFO ] [MAIN ] CHMI: Wiping existing primary data before full refresh. [2025-11-04T18:00:05.825070] [INFO ] [MAIN ] CHPT: Loaded 45583 existing records. [2025-11-04T18:00:05.873385] [INFO ] [LOCAL ] CHMI: Starting deletion of primary data. [2025-11-04T18:00:05.873293] [DEBUG] [LOCAL ] CHWY: Loaded 53654 existing records from primary storage. [2025-11-04T18:00:05.950496] [DEBUG] [LOCAL ] CHMI: Deleted primary data file. [2025-11-04T18:00:05.954147] [WARN ] [MAIN ] CHT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:00:05.968471] [INFO ] [MAIN ] CHMI: Starting full primary refresh. [2025-11-04T18:00:05.951303] [INFO ] [MAIN ] CHWY: Loaded 53654 existing records. [2025-11-04T18:00:05.968751] [INFO ] [MAIN ] CHT: Wiping existing primary data before full refresh. [2025-11-04T18:00:05.988241] [INFO ] [LOCAL ] CHT: Starting deletion of primary data. [2025-11-04T18:00:06.110087] [DEBUG] [LOCAL ] CHT: Deleted primary data file. [2025-11-04T18:00:06.133553] [INFO ] [MAIN ] CHT: Starting full primary refresh. [2025-11-04T18:00:06.474280] [WARN ] [MAIN ] CHPT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:00:06.509049] [WARN ] [MAIN ] CHWY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:00:06.538525] [INFO ] [MAIN ] CHPT: Wiping existing primary data before full refresh. [2025-11-04T18:00:06.574690] [INFO ] [LOCAL ] CHPT: Starting deletion of primary data. [2025-11-04T18:00:06.568537] [INFO ] [MAIN ] CHWY: Wiping existing primary data before full refresh. [2025-11-04T18:00:06.635161] [INFO ] [LOCAL ] CHWY: Starting deletion of primary data. [2025-11-04T18:00:06.658558] [DEBUG] [LOCAL ] CHPT: Deleted primary data file. [2025-11-04T18:00:06.694804] [INFO ] [MAIN ] CHPT: Starting full primary refresh. [2025-11-04T18:00:06.709978] [DEBUG] [LOCAL ] CHWY: Deleted primary data file. [2025-11-04T18:00:06.743754] [INFO ] [MAIN ] CHWY: Starting full primary refresh. [2025-11-04T18:00:07.281743] [INFO ] [REMOTE ] CHMI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:00:07.403135] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-01. [2025-11-04T18:00:07.510631] [INFO ] [REMOTE ] CHT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:00:07.561865] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-01. [2025-11-04T18:00:07.756985] [INFO ] [REMOTE ] CHPT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:00:07.779601] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-01. [2025-11-04T18:00:07.779297] [INFO ] [REMOTE ] CHWY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:00:07.779941] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-01. [2025-11-04T18:00:08.537481] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T18:00:08.542078] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-02. [2025-11-04T18:00:08.724861] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T18:00:08.729734] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-02. [2025-11-04T18:00:09.307821] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-11-04T18:00:09.312128] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-03. [2025-11-04T18:00:09.345694] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-11-04T18:00:09.350079] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-03. [2025-11-04T18:00:09.411633] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-11-04T18:00:09.422490] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-02. [2025-11-04T18:00:09.619750] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-11-04T18:00:09.630193] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-02. [2025-11-04T18:00:10.060787] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-11-04T18:00:10.066037] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-04. [2025-11-04T18:00:10.241921] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-11-04T18:00:10.567065] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-04. [2025-11-04T18:00:10.898750] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-11-04T18:00:10.954976] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-11-04T18:00:11.239751] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-11-04T18:00:11.366998] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-03. [2025-11-04T18:00:11.532435] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-11-04T18:00:12.167071] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-05. [2025-11-04T18:00:12.967058] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-03. [2025-11-04T18:00:13.057286] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-11-04T18:00:13.102089] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T18:00:13.767111] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-04. [2025-11-04T18:00:14.304093] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T18:00:14.567001] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-06. [2025-11-04T18:00:15.367017] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-04. [2025-11-04T18:00:15.408742] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T18:00:15.699437] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-11-04T18:00:16.167058] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-05. [2025-11-04T18:00:16.794412] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T18:00:16.966999] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-07. [2025-11-04T18:00:17.067260] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T18:00:17.673820] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-11-04T18:00:17.767048] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-08. [2025-11-04T18:00:18.567074] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-05. [2025-11-04T18:00:18.990412] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T18:00:19.367029] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-06. [2025-11-04T18:00:20.167024] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-05. [2025-11-04T18:00:20.256835] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T18:00:20.348870] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-11-04T18:00:20.967027] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-06. [2025-11-04T18:00:21.779857] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-07. [2025-11-04T18:00:21.801727] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-11-04T18:00:22.497942] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-11-04T18:00:22.567055] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-07. [2025-11-04T18:00:22.566988] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-11-04T18:00:23.367060] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-08. [2025-11-04T18:00:23.967836] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-11-04T18:00:24.179362] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-06. [2025-11-04T18:00:24.182482] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-11-04T18:00:24.967037] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-09. [2025-11-04T18:00:25.554267] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-11-04T18:00:25.634968] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T18:00:25.767076] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-07. [2025-11-04T18:00:26.567072] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-09. [2025-11-04T18:00:27.367047] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-10. [2025-11-04T18:00:27.411101] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-11-04T18:00:27.451256] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-11-04T18:00:27.976517] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-11-04T18:00:28.167061] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-11. [2025-11-04T18:00:28.747700] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-11 (0.58 seconds). [2025-11-04T18:00:28.748926] [INFO ] [REMOTE ] CHT: Full fetch complete (27558 records). [2025-11-04T18:00:28.748999] [DEBUG] [VALIDATOR ] CHT: Validating primary data. [2025-11-04T18:00:28.972278] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-08. [2025-11-04T18:00:29.141624] [DEBUG] [VALIDATOR ] CHT: All expected trading days are present. [2025-11-04T18:00:29.141728] [DEBUG] [VALIDATOR ] CHT: Primary data validation passed. [2025-11-04T18:00:29.142145] [INFO ] [MAIN ] CHT: Fetched primary data is valid. [2025-11-04T18:00:29.142183] [DEBUG] [LOCAL ] CHT: Saving primary data. [2025-11-04T18:00:29.142868] [DEBUG] [LOCAL ] CHT: Trimmed 27558 records to 18666 (keeping 200 days of data) [2025-11-04T18:00:29.248700] [DEBUG] [LOCAL ] CHT: Primary data saved (18666 records). [2025-11-04T18:00:29.248840] [INFO ] [MAIN ] CHT: Full history saved. [2025-11-04T18:00:29.248929] [INFO ] [MAIN ] Update Progress: 441/2367 symbols completed. CHT took 24.6s. Estimated remaining: 4h 23m 56s. Rate: 7.3 symbols/min [2025-11-04T18:00:29.416982] [INFO ] [MAIN ] CI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:00:29.460437] [DEBUG] [LOCAL ] CI: Loaded 49506 existing records from primary storage. [2025-11-04T18:00:29.460747] [INFO ] [MAIN ] CI: Loaded 49506 existing records. [2025-11-04T18:00:29.639379] [WARN ] [MAIN ] CI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:00:29.639607] [INFO ] [MAIN ] CI: Wiping existing primary data before full refresh. [2025-11-04T18:00:29.639708] [INFO ] [LOCAL ] CI: Starting deletion of primary data. [2025-11-04T18:00:29.639922] [DEBUG] [LOCAL ] CI: Deleted primary data file. [2025-11-04T18:00:29.639977] [INFO ] [MAIN ] CI: Starting full primary refresh. [2025-11-04T18:00:29.776831] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-10. [2025-11-04T18:00:29.990034] [INFO ] [REMOTE ] CI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:00:30.567028] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-08. [2025-11-04T18:00:30.817254] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-11-04T18:00:30.882492] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-11-04T18:00:31.367045] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-11. [2025-11-04T18:00:31.987416] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T18:00:32.014219] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-11 (0.65 seconds). [2025-11-04T18:00:32.015241] [INFO ] [REMOTE ] CHMI: Full fetch complete (31222 records). [2025-11-04T18:00:32.015311] [DEBUG] [VALIDATOR ] CHMI: Validating primary data. [2025-11-04T18:00:32.170862] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-01. [2025-11-04T18:00:32.389188] [DEBUG] [VALIDATOR ] CHMI: All expected trading days are present. [2025-11-04T18:00:32.389294] [DEBUG] [VALIDATOR ] CHMI: Primary data validation passed. [2025-11-04T18:00:32.389753] [INFO ] [MAIN ] CHMI: Fetched primary data is valid. [2025-11-04T18:00:32.389813] [DEBUG] [LOCAL ] CHMI: Saving primary data. [2025-11-04T18:00:32.390500] [DEBUG] [LOCAL ] CHMI: Trimmed 31222 records to 20404 (keeping 200 days of data) [2025-11-04T18:00:32.502992] [DEBUG] [LOCAL ] CHMI: Primary data saved (20404 records). [2025-11-04T18:00:32.503112] [INFO ] [MAIN ] CHMI: Full history saved. [2025-11-04T18:00:32.503183] [INFO ] [MAIN ] Update Progress: 442/2367 symbols completed. CHMI took 27.9s. Estimated remaining: 4h 23m 26s. Rate: 7.3 symbols/min [2025-11-04T18:00:32.667816] [INFO ] [MAIN ] CIA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:00:32.683541] [DEBUG] [LOCAL ] CIA: Loaded 16100 existing records from primary storage. [2025-11-04T18:00:32.683789] [INFO ] [MAIN ] CIA: Loaded 16100 existing records. [2025-11-04T18:00:32.854984] [WARN ] [MAIN ] CIA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:00:32.855219] [INFO ] [MAIN ] CIA: Wiping existing primary data before full refresh. [2025-11-04T18:00:32.855267] [INFO ] [LOCAL ] CIA: Starting deletion of primary data. [2025-11-04T18:00:32.855419] [DEBUG] [LOCAL ] CIA: Deleted primary data file. [2025-11-04T18:00:32.855457] [INFO ] [MAIN ] CIA: Starting full primary refresh. [2025-11-04T18:00:32.972222] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-09. [2025-11-04T18:00:33.242155] [INFO ] [REMOTE ] CIA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:00:33.767023] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-09. [2025-11-04T18:00:33.825121] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-11-04T18:00:34.245954] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-11-04T18:00:34.567094] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-02. [2025-11-04T18:00:35.380110] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-01. [2025-11-04T18:00:35.386550] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-11-04T18:00:35.894430] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T18:00:36.167052] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-10. [2025-11-04T18:00:36.188223] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T18:00:36.967062] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-02. [2025-11-04T18:00:37.383299] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T18:00:37.441620] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-11-04T18:00:37.767056] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-11. [2025-11-04T18:00:38.567123] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-10. [2025-11-04T18:00:38.926510] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T18:00:38.928390] [INFO ] [REMOTE ] CHPT: Full fetch complete (69604 records). [2025-11-04T18:00:38.928539] [DEBUG] [VALIDATOR ] CHPT: Validating primary data. [2025-11-04T18:00:39.324593] [DEBUG] [VALIDATOR ] CHPT: All expected trading days are present. [2025-11-04T18:00:39.324692] [DEBUG] [VALIDATOR ] CHPT: Primary data validation passed. [2025-11-04T18:00:39.325722] [INFO ] [MAIN ] CHPT: Fetched primary data is valid. [2025-11-04T18:00:39.325784] [DEBUG] [LOCAL ] CHPT: Saving primary data. [2025-11-04T18:00:39.330153] [DEBUG] [LOCAL ] CHPT: Trimmed 69604 records to 45538 (keeping 200 days of data) [2025-11-04T18:00:39.381366] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-03. [2025-11-04T18:00:39.592397] [DEBUG] [LOCAL ] CHPT: Primary data saved (45538 records). [2025-11-04T18:00:39.592520] [INFO ] [MAIN ] CHPT: Full history saved. [2025-11-04T18:00:39.592633] [INFO ] [MAIN ] Update Progress: 443/2367 symbols completed. CHPT took 35.0s. Estimated remaining: 4h 23m 13s. Rate: 7.3 symbols/min [2025-11-04T18:00:39.759544] [INFO ] [MAIN ] CIB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:00:39.786524] [DEBUG] [LOCAL ] CIB: Loaded 29148 existing records from primary storage. [2025-11-04T18:00:39.786841] [INFO ] [MAIN ] CIB: Loaded 29148 existing records. [2025-11-04T18:00:39.966489] [WARN ] [MAIN ] CIB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:00:39.966907] [INFO ] [MAIN ] CIB: Wiping existing primary data before full refresh. [2025-11-04T18:00:39.967158] [INFO ] [LOCAL ] CIB: Starting deletion of primary data. [2025-11-04T18:00:39.967403] [DEBUG] [LOCAL ] CIB: Deleted primary data file. [2025-11-04T18:00:39.967503] [INFO ] [MAIN ] CIB: Starting full primary refresh. [2025-11-04T18:00:40.081497] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T18:00:40.168268] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-04. [2025-11-04T18:00:40.358962] [INFO ] [REMOTE ] CIB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:00:40.391623] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-11-04T18:00:40.967009] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-03. [2025-11-04T18:00:40.989980] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T18:00:41.767043] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-01. [2025-11-04T18:00:42.287076] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T18:00:42.567079] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-05. [2025-11-04T18:00:42.935020] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T18:00:43.212675] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-11-04T18:00:43.367024] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-06. [2025-11-04T18:00:44.104017] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-11-04T18:00:44.167104] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-07. [2025-11-04T18:00:44.967021] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-02. [2025-11-04T18:00:44.983289] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T18:00:45.767016] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-04. [2025-11-04T18:00:45.845576] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-11-04T18:00:46.567059] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-03. [2025-11-04T18:00:47.323678] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T18:00:47.367004] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-05. [2025-11-04T18:00:47.543172] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T18:00:48.167087] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-04. [2025-11-04T18:00:48.954901] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-11-04T18:00:48.966998] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-06. [2025-11-04T18:00:49.162305] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-11-04T18:00:49.767049] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-05. [2025-11-04T18:00:50.206902] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T18:00:50.567076] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-11. [2025-11-04T18:00:50.743426] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-11-04T18:00:51.367084] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-07. [2025-11-04T18:00:51.992384] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-11 (1.43 seconds). [2025-11-04T18:00:51.995034] [INFO ] [REMOTE ] CHWY: Full fetch complete (77780 records). [2025-11-04T18:00:51.995222] [DEBUG] [VALIDATOR ] CHWY: Validating primary data. [2025-11-04T18:00:52.171970] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-06. [2025-11-04T18:00:52.495100] [DEBUG] [VALIDATOR ] CHWY: All expected trading days are present. [2025-11-04T18:00:52.495209] [DEBUG] [VALIDATOR ] CHWY: Primary data validation passed. [2025-11-04T18:00:52.496169] [INFO ] [MAIN ] CHWY: Fetched primary data is valid. [2025-11-04T18:00:52.496236] [DEBUG] [LOCAL ] CHWY: Saving primary data. [2025-11-04T18:00:52.497798] [DEBUG] [LOCAL ] CHWY: Trimmed 77780 records to 53654 (keeping 200 days of data) [2025-11-04T18:00:52.861889] [DEBUG] [LOCAL ] CHWY: Primary data saved (53654 records). [2025-11-04T18:00:52.878711] [INFO ] [MAIN ] CHWY: Full history saved. [2025-11-04T18:00:52.878947] [INFO ] [MAIN ] Update Progress: 444/2367 symbols completed. CHWY took 48.3s. Estimated remaining: 4h 23m 27s. Rate: 7.3 symbols/min [2025-11-04T18:00:52.914133] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-11-04T18:00:52.967775] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-08. [2025-11-04T18:00:53.061775] [INFO ] [MAIN ] CICB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:00:53.061900] [DEBUG] [LOCAL ] CICB: No existing data file found in primary storage. [2025-11-04T18:00:53.061935] [WARN ] [MAIN ] CICB: No existing data found, performing full refresh instead. [2025-11-04T18:00:53.062037] [INFO ] [MAIN ] CICB: Wiping existing primary data before full refresh. [2025-11-04T18:00:53.062077] [INFO ] [LOCAL ] CICB: Starting deletion of primary data. [2025-11-04T18:00:53.062116] [DEBUG] [LOCAL ] CICB: No primary data file found to delete. [2025-11-04T18:00:53.062147] [INFO ] [MAIN ] CICB: Starting full primary refresh. [2025-11-04T18:00:53.431178] [INFO ] [REMOTE ] CICB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:00:53.448693] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T18:00:53.767013] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-08. [2025-11-04T18:00:53.813046] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-11-04T18:00:54.567045] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-07. [2025-11-04T18:00:55.068743] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T18:00:55.367058] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-09. [2025-11-04T18:00:55.428547] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-11-04T18:00:56.167111] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-08. [2025-11-04T18:00:56.611055] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-11-04T18:00:56.967112] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-09. [2025-11-04T18:00:57.105015] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-11-04T18:00:57.558694] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-11-04T18:00:57.767052] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-10. [2025-11-04T18:00:58.567017] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-01. [2025-11-04T18:00:59.092314] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T18:00:59.367053] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-09. [2025-11-04T18:00:59.606059] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-11-04T18:01:00.167048] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-11. [2025-11-04T18:01:00.337934] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-11-04T18:01:00.404262] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-11 (0.24 seconds). [2025-11-04T18:01:00.405741] [INFO ] [REMOTE ] CI: Full fetch complete (71696 records). [2025-11-04T18:01:00.405836] [DEBUG] [VALIDATOR ] CI: Validating primary data. [2025-11-04T18:01:00.406441] [FATAL] [VALIDATOR ] CI: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:01:00.406525] [FATAL] [MAIN ] CI: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:01:00.406568] [WARN ] [MAIN ] CI: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:01:00.406633] [INFO ] [LOCAL ] CI: Starting deletion of primary data. [2025-11-04T18:01:00.406748] [DEBUG] [LOCAL ] CI: No primary data file found to delete. [2025-11-04T18:01:00.406822] [INFO ] [LOCAL ] CI: Starting deletion of derivative data. [2025-11-04T18:01:00.406958] [DEBUG] [LOCAL ] CI: Deleted derivative data file. [2025-11-04T18:01:00.407429] [INFO ] [MAIN ] Update Progress: 445/2367 symbols completed. CI took 31.2s. Estimated remaining: 4h 23m 15s. Rate: 7.3 symbols/min [2025-11-04T18:01:00.585393] [INFO ] [MAIN ] CIEN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:01:00.585569] [DEBUG] [LOCAL ] CIEN: No existing data file found in primary storage. [2025-11-04T18:01:00.585605] [WARN ] [MAIN ] CIEN: No existing data found, performing full refresh instead. [2025-11-04T18:01:00.585744] [INFO ] [MAIN ] CIEN: Wiping existing primary data before full refresh. [2025-11-04T18:01:00.585784] [INFO ] [LOCAL ] CIEN: Starting deletion of primary data. [2025-11-04T18:01:00.585823] [DEBUG] [LOCAL ] CIEN: No primary data file found to delete. [2025-11-04T18:01:00.585853] [INFO ] [MAIN ] CIEN: Starting full primary refresh. [2025-11-04T18:01:00.942161] [INFO ] [REMOTE ] CIEN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:01:00.967064] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-02. [2025-11-04T18:01:01.437516] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-11-04T18:01:01.767105] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-03. [2025-11-04T18:01:02.064988] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-03 (0.30 seconds). [2025-11-04T18:01:02.567083] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-01. [2025-11-04T18:01:03.367078] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-10. [2025-11-04T18:01:04.123699] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-11-04T18:01:04.141444] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-11-04T18:01:04.167065] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-02. [2025-11-04T18:01:04.967099] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-11. [2025-11-04T18:01:05.486159] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-11 (0.52 seconds). [2025-11-04T18:01:05.487397] [INFO ] [REMOTE ] CIA: Full fetch complete (22991 records). [2025-11-04T18:01:05.487565] [DEBUG] [VALIDATOR ] CIA: Validating primary data. [2025-11-04T18:01:05.772231] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-04. [2025-11-04T18:01:05.779608] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-11-04T18:01:05.912702] [DEBUG] [VALIDATOR ] CIA: All expected trading days are present. [2025-11-04T18:01:05.912824] [DEBUG] [VALIDATOR ] CIA: Primary data validation passed. [2025-11-04T18:01:05.913308] [INFO ] [MAIN ] CIA: Fetched primary data is valid. [2025-11-04T18:01:05.913351] [DEBUG] [LOCAL ] CIA: Saving primary data. [2025-11-04T18:01:05.914196] [DEBUG] [LOCAL ] CIA: Trimmed 22991 records to 16099 (keeping 200 days of data) [2025-11-04T18:01:06.004198] [DEBUG] [LOCAL ] CIA: Primary data saved (16099 records). [2025-11-04T18:01:06.004340] [INFO ] [MAIN ] CIA: Full history saved. [2025-11-04T18:01:06.004433] [INFO ] [MAIN ] Update Progress: 446/2367 symbols completed. CIA took 33.5s. Estimated remaining: 4h 22m 56s. Rate: 7.3 symbols/min [2025-11-04T18:01:06.139175] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-11-04T18:01:06.181318] [INFO ] [MAIN ] CIF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:01:06.187935] [DEBUG] [LOCAL ] CIF: Loaded 3836 existing records from primary storage. [2025-11-04T18:01:06.188117] [INFO ] [MAIN ] CIF: Loaded 3836 existing records. [2025-11-04T18:01:06.364609] [WARN ] [MAIN ] CIF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:01:06.364902] [INFO ] [MAIN ] CIF: Wiping existing primary data before full refresh. [2025-11-04T18:01:06.364951] [INFO ] [LOCAL ] CIF: Starting deletion of primary data. [2025-11-04T18:01:06.365101] [DEBUG] [LOCAL ] CIF: Deleted primary data file. [2025-11-04T18:01:06.365159] [INFO ] [MAIN ] CIF: Starting full primary refresh. [2025-11-04T18:01:06.572126] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-10. [2025-11-04T18:01:06.739926] [INFO ] [REMOTE ] CIF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:01:07.367106] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-03. [2025-11-04T18:01:07.782316] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-11-04T18:01:08.167046] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-05. [2025-11-04T18:01:08.420323] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-05 (0.25 seconds). [2025-11-04T18:01:08.730043] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T18:01:08.967098] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-04. [2025-11-04T18:01:09.767081] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-11. [2025-11-04T18:01:10.241727] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T18:01:10.548349] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T18:01:10.549468] [INFO ] [REMOTE ] CIB: Full fetch complete (45042 records). [2025-11-04T18:01:10.549558] [DEBUG] [VALIDATOR ] CIB: Validating primary data. [2025-11-04T18:01:10.550219] [FATAL] [VALIDATOR ] CIB: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:01:10.550318] [FATAL] [MAIN ] CIB: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:01:10.550406] [WARN ] [MAIN ] CIB: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:01:10.550499] [INFO ] [LOCAL ] CIB: Starting deletion of primary data. [2025-11-04T18:01:10.550578] [DEBUG] [LOCAL ] CIB: No primary data file found to delete. [2025-11-04T18:01:10.550618] [INFO ] [LOCAL ] CIB: Starting deletion of derivative data. [2025-11-04T18:01:10.550805] [DEBUG] [LOCAL ] CIB: Deleted derivative data file. [2025-11-04T18:01:10.550896] [INFO ] [MAIN ] Update Progress: 447/2367 symbols completed. CIB took 31.0s. Estimated remaining: 4h 22m 32s. Rate: 7.3 symbols/min [2025-11-04T18:01:10.572131] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-06. [2025-11-04T18:01:10.728418] [INFO ] [MAIN ] CIG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:01:10.728591] [DEBUG] [LOCAL ] CIG: No existing data file found in primary storage. [2025-11-04T18:01:10.728630] [WARN ] [MAIN ] CIG: No existing data found, performing full refresh instead. [2025-11-04T18:01:10.728761] [INFO ] [MAIN ] CIG: Wiping existing primary data before full refresh. [2025-11-04T18:01:10.728804] [INFO ] [LOCAL ] CIG: Starting deletion of primary data. [2025-11-04T18:01:10.728854] [DEBUG] [LOCAL ] CIG: No primary data file found to delete. [2025-11-04T18:01:10.728888] [INFO ] [MAIN ] CIG: Starting full primary refresh. [2025-11-04T18:01:11.101575] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-11-04T18:01:11.107392] [INFO ] [REMOTE ] CIG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:01:11.367059] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-01. [2025-11-04T18:01:12.167062] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-07. [2025-11-04T18:01:12.455060] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-07 (0.29 seconds). [2025-11-04T18:01:12.954796] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-11-04T18:01:12.967002] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-08. [2025-11-04T18:01:13.204351] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-08 (0.24 seconds). [2025-11-04T18:01:13.767033] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-01. [2025-11-04T18:01:14.328069] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T18:01:14.567126] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-02. [2025-11-04T18:01:15.367030] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-02. [2025-11-04T18:01:15.592832] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T18:01:15.860724] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-11-04T18:01:16.167048] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-03. [2025-11-04T18:01:16.967055] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-09. [2025-11-04T18:01:17.251173] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-09 (0.28 seconds). [2025-11-04T18:01:17.281908] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T18:01:17.767084] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-10. [2025-11-04T18:01:18.050280] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-10 (0.28 seconds). [2025-11-04T18:01:18.567051] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-05. [2025-11-04T18:01:19.367062] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-11. [2025-11-04T18:01:19.662183] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-11 (0.30 seconds). [2025-11-04T18:01:19.662853] [INFO ] [REMOTE ] CICB: Full fetch complete (1896 records). [2025-11-04T18:01:19.662920] [DEBUG] [VALIDATOR ] CICB: Validating primary data. [2025-11-04T18:01:19.852041] [WARN ] [VALIDATOR ] CICB: Validation failed - insufficient historical data. Have 196 unique days, need 200 days [2025-11-04T18:01:19.852348] [FATAL] [MAIN ] CICB: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:01:19.852502] [WARN ] [MAIN ] CICB: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:01:19.852743] [INFO ] [LOCAL ] CICB: Starting deletion of primary data. [2025-11-04T18:01:19.852865] [DEBUG] [LOCAL ] CICB: No primary data file found to delete. [2025-11-04T18:01:19.853004] [INFO ] [LOCAL ] CICB: Starting deletion of derivative data. [2025-11-04T18:01:19.853075] [DEBUG] [LOCAL ] CICB: No derivative data file found to delete. [2025-11-04T18:01:19.853149] [INFO ] [MAIN ] Update Progress: 448/2367 symbols completed. CICB took 27.0s. Estimated remaining: 4h 22m 28s. Rate: 7.3 symbols/min [2025-11-04T18:01:20.032817] [INFO ] [MAIN ] CII: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:01:20.033079] [DEBUG] [LOCAL ] CII: No existing data file found in primary storage. [2025-11-04T18:01:20.033148] [WARN ] [MAIN ] CII: No existing data found, performing full refresh instead. [2025-11-04T18:01:20.033279] [INFO ] [MAIN ] CII: Wiping existing primary data before full refresh. [2025-11-04T18:01:20.033452] [INFO ] [LOCAL ] CII: Starting deletion of primary data. [2025-11-04T18:01:20.033591] [DEBUG] [LOCAL ] CII: No primary data file found to delete. [2025-11-04T18:01:20.033636] [INFO ] [MAIN ] CII: Starting full primary refresh. [2025-11-04T18:01:20.191194] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-04. [2025-11-04T18:01:20.300916] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-11-04T18:01:20.462246] [INFO ] [REMOTE ] CII: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:01:20.967032] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-06. [2025-11-04T18:01:21.550289] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T18:01:21.767081] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-05. [2025-11-04T18:01:22.207578] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T18:01:22.567074] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-01. [2025-11-04T18:01:23.109773] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T18:01:23.186891] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-11-04T18:01:23.367076] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-02. [2025-11-04T18:01:23.906124] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-11-04T18:01:24.167099] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-03. [2025-11-04T18:01:24.836341] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-11-04T18:01:24.967086] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-07. [2025-11-04T18:01:25.767041] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-03. [2025-11-04T18:01:26.406287] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T18:01:26.567076] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-04. [2025-11-04T18:01:27.301599] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T18:01:27.367016] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-05. [2025-11-04T18:01:27.484880] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-11-04T18:01:27.957295] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T18:01:28.167072] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-06. [2025-11-04T18:01:28.738745] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-11-04T18:01:28.967062] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-08. [2025-11-04T18:01:29.767091] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-04. [2025-11-04T18:01:30.206011] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-11-04T18:01:30.362411] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T18:01:30.567035] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-07. [2025-11-04T18:01:31.235822] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T18:01:31.367054] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-06. [2025-11-04T18:01:32.167089] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-08. [2025-11-04T18:01:32.653664] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-11-04T18:01:32.841427] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T18:01:32.967112] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-07. [2025-11-04T18:01:33.767072] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-09. [2025-11-04T18:01:34.248754] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T18:01:34.567056] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-08. [2025-11-04T18:01:35.318309] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-11-04T18:01:35.367023] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-10. [2025-11-04T18:01:35.695253] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T18:01:36.167088] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-09. [2025-11-04T18:01:36.927077] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-11-04T18:01:36.967094] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-09. [2025-11-04T18:01:37.355605] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T18:01:37.642423] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T18:01:37.767065] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-11. [2025-11-04T18:01:38.567068] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-10. [2025-11-04T18:01:39.071018] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T18:01:39.072986] [INFO ] [REMOTE ] CIEN: Full fetch complete (75855 records). [2025-11-04T18:01:39.073070] [DEBUG] [VALIDATOR ] CIEN: Validating primary data. [2025-11-04T18:01:39.372479] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-10. [2025-11-04T18:01:39.479377] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-11-04T18:01:39.501887] [DEBUG] [VALIDATOR ] CIEN: All expected trading days are present. [2025-11-04T18:01:39.502157] [DEBUG] [VALIDATOR ] CIEN: Primary data validation passed. [2025-11-04T18:01:39.503546] [INFO ] [MAIN ] CIEN: Fetched primary data is valid. [2025-11-04T18:01:39.504032] [DEBUG] [LOCAL ] CIEN: Saving primary data. [2025-11-04T18:01:39.505496] [DEBUG] [LOCAL ] CIEN: Trimmed 75855 records to 52067 (keeping 200 days of data) [2025-11-04T18:01:39.805996] [DEBUG] [LOCAL ] CIEN: Primary data saved (52067 records). [2025-11-04T18:01:39.806175] [INFO ] [MAIN ] CIEN: Full history saved. [2025-11-04T18:01:39.806267] [INFO ] [MAIN ] Update Progress: 449/2367 symbols completed. CIEN took 39.4s. Estimated remaining: 4h 23m 10s. Rate: 7.3 symbols/min [2025-11-04T18:01:39.989781] [INFO ] [MAIN ] CIM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:01:40.025141] [DEBUG] [LOCAL ] CIM: Loaded 36872 existing records from primary storage. [2025-11-04T18:01:40.025441] [INFO ] [MAIN ] CIM: Loaded 36872 existing records. [2025-11-04T18:01:40.178532] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-05. [2025-11-04T18:01:40.221876] [WARN ] [MAIN ] CIM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:01:40.222084] [INFO ] [MAIN ] CIM: Wiping existing primary data before full refresh. [2025-11-04T18:01:40.222137] [INFO ] [LOCAL ] CIM: Starting deletion of primary data. [2025-11-04T18:01:40.222401] [DEBUG] [LOCAL ] CIM: Deleted primary data file. [2025-11-04T18:01:40.222457] [INFO ] [MAIN ] CIM: Starting full primary refresh. [2025-11-04T18:01:40.611174] [INFO ] [REMOTE ] CIM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:01:40.760537] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-11-04T18:01:40.760341] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T18:01:40.967102] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-11. [2025-11-04T18:01:41.684083] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T18:01:41.685428] [INFO ] [REMOTE ] CII: Full fetch complete (23000 records). [2025-11-04T18:01:41.685538] [DEBUG] [VALIDATOR ] CII: Validating primary data. [2025-11-04T18:01:41.772274] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-11. [2025-11-04T18:01:42.089290] [DEBUG] [VALIDATOR ] CII: All expected trading days are present. [2025-11-04T18:01:42.089389] [DEBUG] [VALIDATOR ] CII: Primary data validation passed. [2025-11-04T18:01:42.089873] [INFO ] [MAIN ] CII: Fetched primary data is valid. [2025-11-04T18:01:42.089937] [DEBUG] [LOCAL ] CII: Saving primary data. [2025-11-04T18:01:42.090824] [DEBUG] [LOCAL ] CII: Trimmed 23000 records to 14436 (keeping 200 days of data) [2025-11-04T18:01:42.177348] [DEBUG] [LOCAL ] CII: Primary data saved (14436 records). [2025-11-04T18:01:42.177488] [INFO ] [MAIN ] CII: Full history saved. [2025-11-04T18:01:42.177564] [INFO ] [MAIN ] Update Progress: 450/2367 symbols completed. CII took 22.3s. Estimated remaining: 4h 22m 37s. Rate: 7.3 symbols/min [2025-11-04T18:01:42.343992] [INFO ] [MAIN ] CIMN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:01:42.348810] [DEBUG] [LOCAL ] CIMN: Loaded 2632 existing records from primary storage. [2025-11-04T18:01:42.348968] [INFO ] [MAIN ] CIMN: Loaded 2632 existing records. [2025-11-04T18:01:42.524064] [WARN ] [MAIN ] CIMN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:01:42.524294] [INFO ] [MAIN ] CIMN: Wiping existing primary data before full refresh. [2025-11-04T18:01:42.524339] [INFO ] [LOCAL ] CIMN: Starting deletion of primary data. [2025-11-04T18:01:42.524471] [DEBUG] [LOCAL ] CIMN: Deleted primary data file. [2025-11-04T18:01:42.524508] [INFO ] [MAIN ] CIMN: Starting full primary refresh. [2025-11-04T18:01:42.578613] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-01. [2025-11-04T18:01:42.904923] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T18:01:42.908519] [INFO ] [REMOTE ] CIMN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:01:42.910910] [INFO ] [REMOTE ] CIG: Full fetch complete (63162 records). [2025-11-04T18:01:42.911033] [DEBUG] [VALIDATOR ] CIG: Validating primary data. [2025-11-04T18:01:43.292070] [DEBUG] [VALIDATOR ] CIG: All expected trading days are present. [2025-11-04T18:01:43.292145] [DEBUG] [VALIDATOR ] CIG: Primary data validation passed. [2025-11-04T18:01:43.292842] [INFO ] [MAIN ] CIG: Fetched primary data is valid. [2025-11-04T18:01:43.292903] [DEBUG] [LOCAL ] CIG: Saving primary data. [2025-11-04T18:01:43.294064] [DEBUG] [LOCAL ] CIG: Trimmed 63162 records to 44885 (keeping 200 days of data) [2025-11-04T18:01:43.376164] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-01. [2025-11-04T18:01:43.545574] [DEBUG] [LOCAL ] CIG: Primary data saved (44885 records). [2025-11-04T18:01:43.545737] [INFO ] [MAIN ] CIG: Full history saved. [2025-11-04T18:01:43.545817] [INFO ] [MAIN ] Update Progress: 451/2367 symbols completed. CIG took 33.0s. Estimated remaining: 4h 21m 60s. Rate: 7.3 symbols/min [2025-11-04T18:01:43.722179] [INFO ] [MAIN ] CIMO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:01:43.746321] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T18:01:43.756545] [DEBUG] [LOCAL ] CIMO: Loaded 3178 existing records from primary storage. [2025-11-04T18:01:43.756816] [INFO ] [MAIN ] CIMO: Loaded 3178 existing records. [2025-11-04T18:01:43.860614] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-01 (0.48 seconds). [2025-11-04T18:01:43.936511] [WARN ] [MAIN ] CIMO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:01:43.936757] [INFO ] [MAIN ] CIMO: Wiping existing primary data before full refresh. [2025-11-04T18:01:43.936844] [INFO ] [LOCAL ] CIMO: Starting deletion of primary data. [2025-11-04T18:01:43.936991] [DEBUG] [LOCAL ] CIMO: Deleted primary data file. [2025-11-04T18:01:43.937028] [INFO ] [MAIN ] CIMO: Starting full primary refresh. [2025-11-04T18:01:44.172286] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-02. [2025-11-04T18:01:44.287043] [INFO ] [REMOTE ] CIMO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:01:44.967089] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-01. [2025-11-04T18:01:45.297338] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T18:01:45.404273] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-11-04T18:01:45.767086] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-02. [2025-11-04T18:01:46.067629] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-02 (0.30 seconds). [2025-11-04T18:01:46.567093] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-06. [2025-11-04T18:01:46.894554] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-11-04T18:01:47.367060] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-03. [2025-11-04T18:01:48.167073] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-07. [2025-11-04T18:01:48.431761] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-11-04T18:01:48.564416] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-11-04T18:01:48.967046] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-02. [2025-11-04T18:01:49.288000] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T18:01:49.767065] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-04. [2025-11-04T18:01:50.567057] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-03. [2025-11-04T18:01:50.847797] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T18:01:50.938741] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-11-04T18:01:51.367058] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-03. [2025-11-04T18:01:51.649589] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-03 (0.28 seconds). [2025-11-04T18:01:52.167090] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-04. [2025-11-04T18:01:52.546465] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T18:01:52.967034] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-05. [2025-11-04T18:01:53.347015] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-11-04T18:01:53.767113] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-08. [2025-11-04T18:01:54.249210] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-11-04T18:01:54.567034] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-09. [2025-11-04T18:01:54.893836] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-09 (0.33 seconds). [2025-11-04T18:01:55.367047] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-10. [2025-11-04T18:01:55.821427] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-11-04T18:01:56.167045] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-11. [2025-11-04T18:01:56.592432] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T18:01:56.593231] [INFO ] [REMOTE ] CIF: Full fetch complete (6545 records). [2025-11-04T18:01:56.593301] [DEBUG] [VALIDATOR ] CIF: Validating primary data. [2025-11-04T18:01:56.961409] [DEBUG] [VALIDATOR ] CIF: All expected trading days are present. [2025-11-04T18:01:56.961493] [DEBUG] [VALIDATOR ] CIF: Primary data validation passed. [2025-11-04T18:01:56.961711] [INFO ] [MAIN ] CIF: Fetched primary data is valid. [2025-11-04T18:01:56.961776] [DEBUG] [LOCAL ] CIF: Saving primary data. [2025-11-04T18:01:56.962310] [DEBUG] [LOCAL ] CIF: Trimmed 6545 records to 3839 (keeping 200 days of data) [2025-11-04T18:01:56.971689] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-05. [2025-11-04T18:01:56.987713] [DEBUG] [LOCAL ] CIF: Primary data saved (3839 records). [2025-11-04T18:01:56.987836] [INFO ] [MAIN ] CIF: Full history saved. [2025-11-04T18:01:56.987918] [INFO ] [MAIN ] Update Progress: 452/2367 symbols completed. CIF took 51.0s. Estimated remaining: 4h 22m 14s. Rate: 7.3 symbols/min [2025-11-04T18:01:57.162841] [INFO ] [MAIN ] CIMP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:01:57.163004] [DEBUG] [LOCAL ] CIMP: No existing data file found in primary storage. [2025-11-04T18:01:57.163042] [WARN ] [MAIN ] CIMP: No existing data found, performing full refresh instead. [2025-11-04T18:01:57.163153] [INFO ] [MAIN ] CIMP: Wiping existing primary data before full refresh. [2025-11-04T18:01:57.163195] [INFO ] [LOCAL ] CIMP: Starting deletion of primary data. [2025-11-04T18:01:57.163236] [DEBUG] [LOCAL ] CIMP: No primary data file found to delete. [2025-11-04T18:01:57.163269] [INFO ] [MAIN ] CIMP: Starting full primary refresh. [2025-11-04T18:01:57.533090] [INFO ] [REMOTE ] CIMP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:01:57.767044] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-06. [2025-11-04T18:01:58.204490] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T18:01:58.278737] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T18:01:58.567043] [INFO ] [REMOTE ] CIMP: Fetching primary data for 2025-01. [2025-11-04T18:01:58.716016] [DEBUG] [REMOTE ] CIMP: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:01:58.716390] [WARN ] [REMOTE ] CIMP: No data returned for month 2025-01 [2025-11-04T18:01:58.716504] [FATAL] [REMOTE ] CIMP: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:01:58.716566] [FATAL] [MAIN ] CIMP: Failed to fetch primary data. [2025-11-04T18:01:58.716662] [WARN ] [MAIN ] CIMP: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:01:58.716965] [INFO ] [LOCAL ] CIMP: Starting deletion of primary data. [2025-11-04T18:01:58.717066] [DEBUG] [LOCAL ] CIMP: No primary data file found to delete. [2025-11-04T18:01:58.717125] [INFO ] [LOCAL ] CIMP: Starting deletion of derivative data. [2025-11-04T18:01:58.717188] [DEBUG] [LOCAL ] CIMP: No derivative data file found to delete. [2025-11-04T18:01:58.717253] [INFO ] [MAIN ] Update Progress: 453/2367 symbols completed. CIMP took 1.7s. Estimated remaining: 4h 21m 38s. Rate: 7.3 symbols/min [2025-11-04T18:01:58.943613] [INFO ] [MAIN ] CINT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:01:58.943816] [DEBUG] [LOCAL ] CINT: No existing data file found in primary storage. [2025-11-04T18:01:58.943860] [WARN ] [MAIN ] CINT: No existing data found, performing full refresh instead. [2025-11-04T18:01:58.943968] [INFO ] [MAIN ] CINT: Wiping existing primary data before full refresh. [2025-11-04T18:01:58.944011] [INFO ] [LOCAL ] CINT: Starting deletion of primary data. [2025-11-04T18:01:58.944053] [DEBUG] [LOCAL ] CINT: No primary data file found to delete. [2025-11-04T18:01:58.944087] [INFO ] [MAIN ] CINT: Starting full primary refresh. [2025-11-04T18:01:59.310756] [INFO ] [REMOTE ] CINT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:01:59.367037] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-01. [2025-11-04T18:02:00.067713] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-11-04T18:02:00.167065] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-02. [2025-11-04T18:02:00.934887] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-11-04T18:02:00.967084] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-03. [2025-11-04T18:02:01.767020] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-06. [2025-11-04T18:02:01.919042] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-11-04T18:02:02.567040] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-04. [2025-11-04T18:02:02.806569] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T18:02:03.247244] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T18:02:03.367040] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-05. [2025-11-04T18:02:04.076850] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-11-04T18:02:04.167088] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-07. [2025-11-04T18:02:04.460922] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-07 (0.29 seconds). [2025-11-04T18:02:04.967048] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-08. [2025-11-04T18:02:05.253848] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-11-04T18:02:05.767041] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-09. [2025-11-04T18:02:06.058448] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-09 (0.29 seconds). [2025-11-04T18:02:06.567069] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-06. [2025-11-04T18:02:07.308190] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-11-04T18:02:07.367037] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-10. [2025-11-04T18:02:07.740080] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-11-04T18:02:08.167024] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-07. [2025-11-04T18:02:08.812478] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-11-04T18:02:08.967045] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-08. [2025-11-04T18:02:09.682027] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T18:02:09.767085] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-09. [2025-11-04T18:02:10.567028] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-11. [2025-11-04T18:02:10.726466] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T18:02:11.367038] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-07. [2025-11-04T18:02:11.652120] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T18:02:11.652882] [INFO ] [REMOTE ] CIMN: Full fetch complete (3864 records). [2025-11-04T18:02:11.652949] [DEBUG] [VALIDATOR ] CIMN: Validating primary data. [2025-11-04T18:02:11.653288] [FATAL] [VALIDATOR ] CIMN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:02:11.653368] [FATAL] [MAIN ] CIMN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:02:11.653410] [WARN ] [MAIN ] CIMN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:02:11.653474] [INFO ] [LOCAL ] CIMN: Starting deletion of primary data. [2025-11-04T18:02:11.653568] [DEBUG] [LOCAL ] CIMN: No primary data file found to delete. [2025-11-04T18:02:11.653616] [INFO ] [LOCAL ] CIMN: Starting deletion of derivative data. [2025-11-04T18:02:11.653796] [DEBUG] [LOCAL ] CIMN: Deleted derivative data file. [2025-11-04T18:02:11.653881] [INFO ] [MAIN ] Update Progress: 454/2367 symbols completed. CIMN took 29.5s. Estimated remaining: 4h 21m 50s. Rate: 7.3 symbols/min [2025-11-04T18:02:11.868102] [INFO ] [MAIN ] CIO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:02:11.869052] [DEBUG] [LOCAL ] CIO: No existing data file found in primary storage. [2025-11-04T18:02:11.869107] [WARN ] [MAIN ] CIO: No existing data found, performing full refresh instead. [2025-11-04T18:02:11.869290] [INFO ] [MAIN ] CIO: Wiping existing primary data before full refresh. [2025-11-04T18:02:11.869345] [INFO ] [LOCAL ] CIO: Starting deletion of primary data. [2025-11-04T18:02:11.869391] [DEBUG] [LOCAL ] CIO: No primary data file found to delete. [2025-11-04T18:02:11.869423] [INFO ] [MAIN ] CIO: Starting full primary refresh. [2025-11-04T18:02:12.172230] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-04. [2025-11-04T18:02:12.267879] [INFO ] [REMOTE ] CIO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:02:12.453292] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-11-04T18:02:12.551247] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T18:02:12.967040] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-05. [2025-11-04T18:02:13.248242] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-11-04T18:02:13.767045] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-10. [2025-11-04T18:02:14.567013] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-06. [2025-11-04T18:02:14.766619] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-11-04T18:02:14.777500] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-06 (0.21 seconds). [2025-11-04T18:02:15.367035] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-01. [2025-11-04T18:02:16.167031] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-11. [2025-11-04T18:02:16.215090] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-11-04T18:02:16.810753] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T18:02:16.811809] [INFO ] [REMOTE ] CINT: Full fetch complete (28886 records). [2025-11-04T18:02:16.811896] [DEBUG] [VALIDATOR ] CINT: Validating primary data. [2025-11-04T18:02:16.812329] [FATAL] [VALIDATOR ] CINT: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:02:16.812408] [FATAL] [MAIN ] CINT: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:02:16.812450] [WARN ] [MAIN ] CINT: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:02:16.812487] [INFO ] [LOCAL ] CINT: Starting deletion of primary data. [2025-11-04T18:02:16.812542] [DEBUG] [LOCAL ] CINT: No primary data file found to delete. [2025-11-04T18:02:16.812575] [INFO ] [LOCAL ] CINT: Starting deletion of derivative data. [2025-11-04T18:02:16.812612] [DEBUG] [LOCAL ] CINT: No derivative data file found to delete. [2025-11-04T18:02:16.812697] [INFO ] [MAIN ] Update Progress: 455/2367 symbols completed. CINT took 18.1s. Estimated remaining: 4h 21m 29s. Rate: 7.3 symbols/min [2025-11-04T18:02:16.969499] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-07. [2025-11-04T18:02:16.998296] [INFO ] [MAIN ] CION: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:02:16.998450] [DEBUG] [LOCAL ] CION: No existing data file found in primary storage. [2025-11-04T18:02:16.998500] [WARN ] [MAIN ] CION: No existing data found, performing full refresh instead. [2025-11-04T18:02:16.998632] [INFO ] [MAIN ] CION: Wiping existing primary data before full refresh. [2025-11-04T18:02:16.998738] [INFO ] [LOCAL ] CION: Starting deletion of primary data. [2025-11-04T18:02:16.998808] [DEBUG] [LOCAL ] CION: No primary data file found to delete. [2025-11-04T18:02:16.998858] [INFO ] [MAIN ] CION: Starting full primary refresh. [2025-11-04T18:02:17.390132] [INFO ] [REMOTE ] CION: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:02:17.393191] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-11-04T18:02:17.767087] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-08. [2025-11-04T18:02:18.172846] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-11-04T18:02:18.567036] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-09. [2025-11-04T18:02:19.050871] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-11-04T18:02:19.367107] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-02. [2025-11-04T18:02:20.009794] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T18:02:20.167031] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-08. [2025-11-04T18:02:20.967041] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-10. [2025-11-04T18:02:21.168098] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T18:02:21.285620] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-10 (0.32 seconds). [2025-11-04T18:02:21.767102] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-01. [2025-11-04T18:02:22.567065] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-03. [2025-11-04T18:02:22.765844] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T18:02:23.371688] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-02. [2025-11-04T18:02:23.375112] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-11-04T18:02:24.151717] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-11-04T18:02:24.167129] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-03. [2025-11-04T18:02:24.967072] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-04. [2025-11-04T18:02:25.053931] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-11-04T18:02:25.767031] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-09. [2025-11-04T18:02:25.799137] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-11-04T18:02:26.567099] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-11. [2025-11-04T18:02:26.978024] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T18:02:27.034772] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T18:02:27.035740] [INFO ] [REMOTE ] CIMO: Full fetch complete (4621 records). [2025-11-04T18:02:27.035824] [DEBUG] [VALIDATOR ] CIMO: Validating primary data. [2025-11-04T18:02:27.372229] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-05. [2025-11-04T18:02:27.465521] [DEBUG] [VALIDATOR ] CIMO: All expected trading days are present. [2025-11-04T18:02:27.465596] [DEBUG] [VALIDATOR ] CIMO: Primary data validation passed. [2025-11-04T18:02:27.465757] [INFO ] [MAIN ] CIMO: Fetched primary data is valid. [2025-11-04T18:02:27.465816] [DEBUG] [LOCAL ] CIMO: Saving primary data. [2025-11-04T18:02:27.466157] [DEBUG] [LOCAL ] CIMO: Trimmed 4621 records to 3188 (keeping 200 days of data) [2025-11-04T18:02:27.486103] [DEBUG] [LOCAL ] CIMO: Primary data saved (3188 records). [2025-11-04T18:02:27.486237] [INFO ] [MAIN ] CIMO: Full history saved. [2025-11-04T18:02:27.486317] [INFO ] [MAIN ] Update Progress: 456/2367 symbols completed. CIMO took 43.9s. Estimated remaining: 4h 21m 31s. Rate: 7.3 symbols/min [2025-11-04T18:02:27.657711] [INFO ] [MAIN ] CIVI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:02:27.657920] [DEBUG] [LOCAL ] CIVI: No existing data file found in primary storage. [2025-11-04T18:02:27.657975] [WARN ] [MAIN ] CIVI: No existing data found, performing full refresh instead. [2025-11-04T18:02:27.658087] [INFO ] [MAIN ] CIVI: Wiping existing primary data before full refresh. [2025-11-04T18:02:27.658129] [INFO ] [LOCAL ] CIVI: Starting deletion of primary data. [2025-11-04T18:02:27.658173] [DEBUG] [LOCAL ] CIVI: No primary data file found to delete. [2025-11-04T18:02:27.658205] [INFO ] [MAIN ] CIVI: Starting full primary refresh. [2025-11-04T18:02:28.019423] [INFO ] [REMOTE ] CIVI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:02:28.142174] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-11-04T18:02:28.167014] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-04. [2025-11-04T18:02:28.967092] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-01. [2025-11-04T18:02:29.283789] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T18:02:29.767087] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-06. [2025-11-04T18:02:30.294808] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T18:02:30.411802] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-11-04T18:02:30.567131] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-02. [2025-11-04T18:02:31.367048] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-05. [2025-11-04T18:02:31.733869] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T18:02:32.167054] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-07. [2025-11-04T18:02:32.310207] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T18:02:32.967033] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-10. [2025-11-04T18:02:33.021499] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T18:02:33.767044] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-08. [2025-11-04T18:02:34.080163] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-11-04T18:02:34.571944] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-06. [2025-11-04T18:02:34.574379] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-11-04T18:02:35.367239] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-03. [2025-11-04T18:02:35.614315] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T18:02:36.167063] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-07. [2025-11-04T18:02:36.736332] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T18:02:36.938894] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T18:02:36.967025] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-09. [2025-11-04T18:02:37.715256] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-11-04T18:02:37.767059] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-10. [2025-11-04T18:02:38.567079] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-04. [2025-11-04T18:02:38.941600] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-11-04T18:02:39.367092] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-11. [2025-11-04T18:02:39.892492] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T18:02:40.167086] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-05. [2025-11-04T18:02:40.568702] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T18:02:40.570378] [INFO ] [REMOTE ] CIM: Full fetch complete (54953 records). [2025-11-04T18:02:40.570463] [DEBUG] [VALIDATOR ] CIM: Validating primary data. [2025-11-04T18:02:40.938515] [DEBUG] [VALIDATOR ] CIM: All expected trading days are present. [2025-11-04T18:02:40.938593] [DEBUG] [VALIDATOR ] CIM: Primary data validation passed. [2025-11-04T18:02:40.939212] [INFO ] [MAIN ] CIM: Fetched primary data is valid. [2025-11-04T18:02:40.939273] [DEBUG] [LOCAL ] CIM: Saving primary data. [2025-11-04T18:02:40.940331] [DEBUG] [LOCAL ] CIM: Trimmed 54953 records to 36857 (keeping 200 days of data) [2025-11-04T18:02:40.991062] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-11. [2025-11-04T18:02:41.156415] [DEBUG] [LOCAL ] CIM: Primary data saved (36857 records). [2025-11-04T18:02:41.156538] [INFO ] [MAIN ] CIM: Full history saved. [2025-11-04T18:02:41.157178] [INFO ] [MAIN ] Update Progress: 457/2367 symbols completed. CIM took 1m 1s. Estimated remaining: 4h 21m 46s. Rate: 7.3 symbols/min [2025-11-04T18:02:41.326259] [INFO ] [MAIN ] CL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:02:41.378226] [DEBUG] [LOCAL ] CL: Loaded 53649 existing records from primary storage. [2025-11-04T18:02:41.378558] [INFO ] [MAIN ] CL: Loaded 53649 existing records. [2025-11-04T18:02:41.555312] [WARN ] [MAIN ] CL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:02:41.555669] [INFO ] [MAIN ] CL: Wiping existing primary data before full refresh. [2025-11-04T18:02:41.555729] [INFO ] [LOCAL ] CL: Starting deletion of primary data. [2025-11-04T18:02:41.556053] [DEBUG] [LOCAL ] CL: Deleted primary data file. [2025-11-04T18:02:41.556108] [INFO ] [MAIN ] CL: Starting full primary refresh. [2025-11-04T18:02:41.772186] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-08. [2025-11-04T18:02:41.796953] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T18:02:41.828532] [INFO ] [REMOTE ] CIO: Full fetch complete (30019 records). [2025-11-04T18:02:41.836438] [DEBUG] [VALIDATOR ] CIO: Validating primary data. [2025-11-04T18:02:41.866595] [FATAL] [VALIDATOR ] CIO: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:02:41.891400] [FATAL] [MAIN ] CIO: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:02:41.898815] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-11-04T18:02:41.897474] [WARN ] [MAIN ] CIO: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:02:41.949478] [INFO ] [LOCAL ] CIO: Starting deletion of primary data. [2025-11-04T18:02:41.986547] [DEBUG] [LOCAL ] CIO: No primary data file found to delete. [2025-11-04T18:02:41.986671] [INFO ] [LOCAL ] CIO: Starting deletion of derivative data. [2025-11-04T18:02:41.986445] [INFO ] [REMOTE ] CL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:02:41.987060] [DEBUG] [LOCAL ] CIO: No derivative data file found to delete. [2025-11-04T18:02:41.987155] [INFO ] [MAIN ] Update Progress: 458/2367 symbols completed. CIO took 30.3s. Estimated remaining: 4h 21m 7s. Rate: 7.3 symbols/min [2025-11-04T18:02:42.152917] [INFO ] [MAIN ] CLB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:02:42.182918] [DEBUG] [LOCAL ] CLB: Loaded 33483 existing records from primary storage. [2025-11-04T18:02:42.183200] [INFO ] [MAIN ] CLB: Loaded 33483 existing records. [2025-11-04T18:02:42.355304] [WARN ] [MAIN ] CLB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:02:42.355507] [INFO ] [MAIN ] CLB: Wiping existing primary data before full refresh. [2025-11-04T18:02:42.355577] [INFO ] [LOCAL ] CLB: Starting deletion of primary data. [2025-11-04T18:02:42.355837] [DEBUG] [LOCAL ] CLB: Deleted primary data file. [2025-11-04T18:02:42.355896] [INFO ] [MAIN ] CLB: Starting full primary refresh. [2025-11-04T18:02:42.569327] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-01. [2025-11-04T18:02:42.735360] [INFO ] [REMOTE ] CLB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:02:42.737102] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-11-04T18:02:43.367060] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-01. [2025-11-04T18:02:43.973588] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T18:02:44.167070] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-02. [2025-11-04T18:02:44.345187] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T18:02:44.967102] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-06. [2025-11-04T18:02:45.495123] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T18:02:45.767059] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-02. [2025-11-04T18:02:46.287592] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T18:02:46.567035] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-07. [2025-11-04T18:02:46.606692] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-11-04T18:02:47.367028] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-03. [2025-11-04T18:02:47.906355] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T18:02:48.167010] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-03. [2025-11-04T18:02:48.310155] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-11-04T18:02:48.967037] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-08. [2025-11-04T18:02:49.767075] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-04. [2025-11-04T18:02:49.889395] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-11-04T18:02:50.378597] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-11-04T18:02:50.567048] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-09. [2025-11-04T18:02:51.007870] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T18:02:51.367043] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-09. [2025-11-04T18:02:51.552328] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T18:02:52.167028] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-05. [2025-11-04T18:02:52.794474] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T18:02:52.967029] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-10. [2025-11-04T18:02:53.048007] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T18:02:53.767044] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-06. [2025-11-04T18:02:54.388826] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-11-04T18:02:54.567146] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-11. [2025-11-04T18:02:54.880328] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T18:02:55.367070] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-07. [2025-11-04T18:02:55.724735] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T18:02:55.726807] [INFO ] [REMOTE ] CIVI: Full fetch complete (76101 records). [2025-11-04T18:02:55.726919] [DEBUG] [VALIDATOR ] CIVI: Validating primary data. [2025-11-04T18:02:56.138120] [DEBUG] [VALIDATOR ] CIVI: All expected trading days are present. [2025-11-04T18:02:56.138200] [DEBUG] [VALIDATOR ] CIVI: Primary data validation passed. [2025-11-04T18:02:56.139003] [INFO ] [MAIN ] CIVI: Fetched primary data is valid. [2025-11-04T18:02:56.139068] [DEBUG] [LOCAL ] CIVI: Saving primary data. [2025-11-04T18:02:56.140485] [DEBUG] [LOCAL ] CIVI: Trimmed 76101 records to 52609 (keeping 200 days of data) [2025-11-04T18:02:56.193941] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-10. [2025-11-04T18:02:56.470855] [DEBUG] [LOCAL ] CIVI: Primary data saved (52609 records). [2025-11-04T18:02:56.470993] [INFO ] [MAIN ] CIVI: Full history saved. [2025-11-04T18:02:56.471070] [INFO ] [MAIN ] Update Progress: 459/2367 symbols completed. CIVI took 29.0s. Estimated remaining: 4h 21m 25s. Rate: 7.3 symbols/min [2025-11-04T18:02:56.653043] [INFO ] [MAIN ] CLCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:02:56.675819] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T18:02:56.693769] [DEBUG] [LOCAL ] CLCO: Loaded 12635 existing records from primary storage. [2025-11-04T18:02:56.694002] [INFO ] [MAIN ] CLCO: Loaded 12635 existing records. [2025-11-04T18:02:56.869699] [WARN ] [MAIN ] CLCO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:02:56.869901] [INFO ] [MAIN ] CLCO: Wiping existing primary data before full refresh. [2025-11-04T18:02:56.869991] [INFO ] [LOCAL ] CLCO: Starting deletion of primary data. [2025-11-04T18:02:56.870156] [DEBUG] [LOCAL ] CLCO: Deleted primary data file. [2025-11-04T18:02:56.870195] [INFO ] [MAIN ] CLCO: Starting full primary refresh. [2025-11-04T18:02:56.970548] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-08. [2025-11-04T18:02:57.239019] [INFO ] [REMOTE ] CLCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:02:57.249995] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-11-04T18:02:57.767092] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-01. [2025-11-04T18:02:58.114787] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T18:02:58.538708] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-11-04T18:02:58.566997] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-02. [2025-11-04T18:02:59.244613] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-11-04T18:02:59.367031] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-11. [2025-11-04T18:03:00.167027] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-03. [2025-11-04T18:03:00.188026] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-11 (0.82 seconds). [2025-11-04T18:03:00.189150] [INFO ] [REMOTE ] CION: Full fetch complete (40363 records). [2025-11-04T18:03:00.189231] [DEBUG] [VALIDATOR ] CION: Validating primary data. [2025-11-04T18:03:00.189824] [FATAL] [VALIDATOR ] CION: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:03:00.189921] [FATAL] [MAIN ] CION: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:03:00.189997] [WARN ] [MAIN ] CION: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:03:00.190094] [INFO ] [LOCAL ] CION: Starting deletion of primary data. [2025-11-04T18:03:00.190172] [DEBUG] [LOCAL ] CION: No primary data file found to delete. [2025-11-04T18:03:00.190217] [INFO ] [LOCAL ] CION: Starting deletion of derivative data. [2025-11-04T18:03:00.190268] [DEBUG] [LOCAL ] CION: No derivative data file found to delete. [2025-11-04T18:03:00.190357] [INFO ] [MAIN ] Update Progress: 460/2367 symbols completed. CION took 43.4s. Estimated remaining: 4h 20m 58s. Rate: 7.3 symbols/min [2025-11-04T18:03:00.362140] [INFO ] [MAIN ] CLDT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:03:00.383077] [DEBUG] [LOCAL ] CLDT: Loaded 24357 existing records from primary storage. [2025-11-04T18:03:00.383320] [INFO ] [MAIN ] CLDT: Loaded 24357 existing records. [2025-11-04T18:03:00.559561] [WARN ] [MAIN ] CLDT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:03:00.559821] [INFO ] [MAIN ] CLDT: Wiping existing primary data before full refresh. [2025-11-04T18:03:00.559881] [INFO ] [LOCAL ] CLDT: Starting deletion of primary data. [2025-11-04T18:03:00.560126] [DEBUG] [LOCAL ] CLDT: Deleted primary data file. [2025-11-04T18:03:00.560165] [INFO ] [MAIN ] CLDT: Starting full primary refresh. [2025-11-04T18:03:00.953197] [INFO ] [REMOTE ] CLDT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:03:00.966997] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-01. [2025-11-04T18:03:01.017539] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-11-04T18:03:01.767038] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-04. [2025-11-04T18:03:02.032555] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-11-04T18:03:02.333776] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-11-04T18:03:02.567047] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-02. [2025-11-04T18:03:03.356989] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T18:03:03.367001] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-03. [2025-11-04T18:03:04.167036] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-09. [2025-11-04T18:03:04.359693] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T18:03:04.967049] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-04. [2025-11-04T18:03:05.120388] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-11-04T18:03:05.767039] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-04. [2025-11-04T18:03:06.381195] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-11-04T18:03:06.567037] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-05. [2025-11-04T18:03:06.783623] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T18:03:07.367027] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-05. [2025-11-04T18:03:07.909457] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-11-04T18:03:08.120399] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-11-04T18:03:08.167002] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-06. [2025-11-04T18:03:08.967035] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-06. [2025-11-04T18:03:09.429688] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-11-04T18:03:09.626011] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-11-04T18:03:09.767044] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-05. [2025-11-04T18:03:10.567053] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-10. [2025-11-04T18:03:10.635741] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T18:03:11.367077] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-06. [2025-11-04T18:03:11.564560] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-11-04T18:03:12.167047] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-07. [2025-11-04T18:03:12.301291] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T18:03:12.967041] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-11. [2025-11-04T18:03:13.583775] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-11-04T18:03:13.767048] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-07. [2025-11-04T18:03:13.978085] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-11 (1.01 seconds). [2025-11-04T18:03:13.979745] [INFO ] [REMOTE ] CLB: Full fetch complete (48399 records). [2025-11-04T18:03:13.979836] [DEBUG] [VALIDATOR ] CLB: Validating primary data. [2025-11-04T18:03:14.348543] [DEBUG] [VALIDATOR ] CLB: All expected trading days are present. [2025-11-04T18:03:14.348623] [DEBUG] [VALIDATOR ] CLB: Primary data validation passed. [2025-11-04T18:03:14.349268] [INFO ] [MAIN ] CLB: Fetched primary data is valid. [2025-11-04T18:03:14.349320] [DEBUG] [LOCAL ] CLB: Saving primary data. [2025-11-04T18:03:14.350245] [DEBUG] [LOCAL ] CLB: Trimmed 48399 records to 33453 (keeping 200 days of data) [2025-11-04T18:03:14.541487] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T18:03:14.566997] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-07. [2025-11-04T18:03:14.571788] [DEBUG] [LOCAL ] CLB: Primary data saved (33453 records). [2025-11-04T18:03:14.572063] [INFO ] [MAIN ] CLB: Full history saved. [2025-11-04T18:03:14.572157] [INFO ] [MAIN ] Update Progress: 461/2367 symbols completed. CLB took 32.6s. Estimated remaining: 4h 21m 15s. Rate: 7.3 symbols/min [2025-11-04T18:03:14.741340] [INFO ] [MAIN ] CLF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:03:14.789634] [DEBUG] [LOCAL ] CLF: Loaded 53741 existing records from primary storage. [2025-11-04T18:03:14.789954] [INFO ] [MAIN ] CLF: Loaded 53741 existing records. [2025-11-04T18:03:14.963291] [WARN ] [MAIN ] CLF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:03:14.963485] [INFO ] [MAIN ] CLF: Wiping existing primary data before full refresh. [2025-11-04T18:03:14.963530] [INFO ] [LOCAL ] CLF: Starting deletion of primary data. [2025-11-04T18:03:14.963739] [DEBUG] [LOCAL ] CLF: Deleted primary data file. [2025-11-04T18:03:14.963795] [INFO ] [MAIN ] CLF: Starting full primary refresh. [2025-11-04T18:03:15.165216] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T18:03:15.353055] [INFO ] [REMOTE ] CLF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:03:15.367003] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-01. [2025-11-04T18:03:16.167089] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-08. [2025-11-04T18:03:16.684216] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-11-04T18:03:16.967040] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-09. [2025-11-04T18:03:17.096619] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-11-04T18:03:17.513248] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-11-04T18:03:17.767101] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-08. [2025-11-04T18:03:18.557583] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T18:03:18.566991] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-09. [2025-11-04T18:03:19.304183] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-11-04T18:03:19.367027] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-10. [2025-11-04T18:03:20.167047] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-10. [2025-11-04T18:03:20.218639] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-11-04T18:03:20.963345] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T18:03:20.968119] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-11. [2025-11-04T18:03:21.726681] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-11 (0.76 seconds). [2025-11-04T18:03:21.728011] [INFO ] [REMOTE ] CLDT: Full fetch complete (37706 records). [2025-11-04T18:03:21.728098] [DEBUG] [VALIDATOR ] CLDT: Validating primary data. [2025-11-04T18:03:21.772251] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-02. [2025-11-04T18:03:22.104969] [DEBUG] [VALIDATOR ] CLDT: All expected trading days are present. [2025-11-04T18:03:22.105063] [DEBUG] [VALIDATOR ] CLDT: Primary data validation passed. [2025-11-04T18:03:22.105568] [INFO ] [MAIN ] CLDT: Fetched primary data is valid. [2025-11-04T18:03:22.105610] [DEBUG] [LOCAL ] CLDT: Saving primary data. [2025-11-04T18:03:22.106407] [DEBUG] [LOCAL ] CLDT: Trimmed 37706 records to 24281 (keeping 200 days of data) [2025-11-04T18:03:22.239875] [DEBUG] [LOCAL ] CLDT: Primary data saved (24281 records). [2025-11-04T18:03:22.239995] [INFO ] [MAIN ] CLDT: Full history saved. [2025-11-04T18:03:22.240069] [INFO ] [MAIN ] Update Progress: 462/2367 symbols completed. CLDT took 22.0s. Estimated remaining: 4h 21m 4s. Rate: 7.3 symbols/min [2025-11-04T18:03:22.428636] [INFO ] [MAIN ] CLH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:03:22.454118] [DEBUG] [LOCAL ] CLH: Loaded 27475 existing records from primary storage. [2025-11-04T18:03:22.454353] [INFO ] [MAIN ] CLH: Loaded 27475 existing records. [2025-11-04T18:03:22.572262] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-11. [2025-11-04T18:03:22.634456] [WARN ] [MAIN ] CLH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:03:22.634632] [INFO ] [MAIN ] CLH: Wiping existing primary data before full refresh. [2025-11-04T18:03:22.634710] [INFO ] [LOCAL ] CLH: Starting deletion of primary data. [2025-11-04T18:03:22.634876] [DEBUG] [LOCAL ] CLH: Deleted primary data file. [2025-11-04T18:03:22.634914] [INFO ] [MAIN ] CLH: Starting full primary refresh. [2025-11-04T18:03:23.000102] [INFO ] [REMOTE ] CLH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:03:23.239457] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T18:03:23.241116] [INFO ] [REMOTE ] CLCO: Full fetch complete (22242 records). [2025-11-04T18:03:23.241421] [DEBUG] [VALIDATOR ] CLCO: Validating primary data. [2025-11-04T18:03:23.367479] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-08. [2025-11-04T18:03:23.572438] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-11-04T18:03:23.675934] [DEBUG] [VALIDATOR ] CLCO: All expected trading days are present. [2025-11-04T18:03:23.676014] [DEBUG] [VALIDATOR ] CLCO: Primary data validation passed. [2025-11-04T18:03:23.676414] [INFO ] [MAIN ] CLCO: Fetched primary data is valid. [2025-11-04T18:03:23.676452] [DEBUG] [LOCAL ] CLCO: Saving primary data. [2025-11-04T18:03:23.677095] [DEBUG] [LOCAL ] CLCO: Trimmed 22242 records to 12637 (keeping 200 days of data) [2025-11-04T18:03:23.745536] [DEBUG] [LOCAL ] CLCO: Primary data saved (12637 records). [2025-11-04T18:03:23.745694] [INFO ] [MAIN ] CLCO: Full history saved. [2025-11-04T18:03:23.745785] [INFO ] [MAIN ] Update Progress: 463/2367 symbols completed. CLCO took 27.3s. Estimated remaining: 4h 20m 29s. Rate: 7.3 symbols/min [2025-11-04T18:03:23.914962] [INFO ] [MAIN ] CLPR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:03:23.915131] [DEBUG] [LOCAL ] CLPR: No existing data file found in primary storage. [2025-11-04T18:03:23.915168] [WARN ] [MAIN ] CLPR: No existing data found, performing full refresh instead. [2025-11-04T18:03:23.915274] [INFO ] [MAIN ] CLPR: Wiping existing primary data before full refresh. [2025-11-04T18:03:23.915315] [INFO ] [LOCAL ] CLPR: Starting deletion of primary data. [2025-11-04T18:03:23.915353] [DEBUG] [LOCAL ] CLPR: No primary data file found to delete. [2025-11-04T18:03:23.915384] [INFO ] [MAIN ] CLPR: Starting full primary refresh. [2025-11-04T18:03:24.177426] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-01. [2025-11-04T18:03:24.266702] [INFO ] [REMOTE ] CLPR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:03:24.967018] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-03. [2025-11-04T18:03:25.028429] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-11-04T18:03:25.319174] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T18:03:25.767060] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-01. [2025-11-04T18:03:26.435095] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-11-04T18:03:26.567069] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-09. [2025-11-04T18:03:26.840118] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-11-04T18:03:27.367037] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-04. [2025-11-04T18:03:27.996932] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T18:03:28.167031] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-10. [2025-11-04T18:03:28.967047] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-02. [2025-11-04T18:03:29.218881] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-11-04T18:03:29.481544] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T18:03:29.762964] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-11-04T18:03:29.774416] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-03. [2025-11-04T18:03:30.425755] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-11-04T18:03:30.567089] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-02. [2025-11-04T18:03:31.367046] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-04. [2025-11-04T18:03:31.526245] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T18:03:31.994541] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-11-04T18:03:32.167081] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-05. [2025-11-04T18:03:32.649016] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-11-04T18:03:32.967086] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-11. [2025-11-04T18:03:33.395141] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T18:03:33.397152] [INFO ] [REMOTE ] CL: Full fetch complete (77819 records). [2025-11-04T18:03:33.397263] [DEBUG] [VALIDATOR ] CL: Validating primary data. [2025-11-04T18:03:33.772800] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-05. [2025-11-04T18:03:33.777666] [DEBUG] [VALIDATOR ] CL: All expected trading days are present. [2025-11-04T18:03:33.777763] [DEBUG] [VALIDATOR ] CL: Primary data validation passed. [2025-11-04T18:03:33.778575] [INFO ] [MAIN ] CL: Fetched primary data is valid. [2025-11-04T18:03:33.778777] [DEBUG] [LOCAL ] CL: Saving primary data. [2025-11-04T18:03:33.780324] [DEBUG] [LOCAL ] CL: Trimmed 77819 records to 53649 (keeping 200 days of data) [2025-11-04T18:03:34.107240] [DEBUG] [LOCAL ] CL: Primary data saved (53649 records). [2025-11-04T18:03:34.107378] [INFO ] [MAIN ] CL: Full history saved. [2025-11-04T18:03:34.107923] [INFO ] [MAIN ] Update Progress: 464/2367 symbols completed. CL took 53.0s. Estimated remaining: 4h 20m 29s. Rate: 7.3 symbols/min [2025-11-04T18:03:34.274728] [INFO ] [MAIN ] CLS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:03:34.275026] [DEBUG] [LOCAL ] CLS: No existing data file found in primary storage. [2025-11-04T18:03:34.275081] [WARN ] [MAIN ] CLS: No existing data found, performing full refresh instead. [2025-11-04T18:03:34.275193] [INFO ] [MAIN ] CLS: Wiping existing primary data before full refresh. [2025-11-04T18:03:34.275235] [INFO ] [LOCAL ] CLS: Starting deletion of primary data. [2025-11-04T18:03:34.275274] [DEBUG] [LOCAL ] CLS: No primary data file found to delete. [2025-11-04T18:03:34.275306] [INFO ] [MAIN ] CLS: Starting full primary refresh. [2025-11-04T18:03:34.572194] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-03. [2025-11-04T18:03:34.634601] [INFO ] [REMOTE ] CLS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:03:35.367015] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-01. [2025-11-04T18:03:35.491023] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-11-04T18:03:35.833203] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-11-04T18:03:36.167043] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-06. [2025-11-04T18:03:36.832279] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-11-04T18:03:36.967044] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-07. [2025-11-04T18:03:37.223995] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-11-04T18:03:37.628791] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-11-04T18:03:37.767013] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-06. [2025-11-04T18:03:38.567085] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-08. [2025-11-04T18:03:39.115613] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-11-04T18:03:39.367090] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-04. [2025-11-04T18:03:39.453394] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-11-04T18:03:40.167057] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-02. [2025-11-04T18:03:40.504333] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T18:03:40.967050] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-05. [2025-11-04T18:03:41.767021] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-09. [2025-11-04T18:03:41.938696] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-11-04T18:03:41.972275] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-11-04T18:03:42.201731] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-11-04T18:03:42.567097] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-10. [2025-11-04T18:03:43.091361] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-11-04T18:03:43.367045] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-06. [2025-11-04T18:03:44.080222] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T18:03:44.167076] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-03. [2025-11-04T18:03:44.967037] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-07. [2025-11-04T18:03:45.767088] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-07. [2025-11-04T18:03:45.987846] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T18:03:46.069594] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-11-04T18:03:46.567033] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-11. [2025-11-04T18:03:47.205029] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T18:03:47.205913] [INFO ] [REMOTE ] CLPR: Full fetch complete (15410 records). [2025-11-04T18:03:47.205986] [DEBUG] [VALIDATOR ] CLPR: Validating primary data. [2025-11-04T18:03:47.370619] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-04. [2025-11-04T18:03:47.600859] [DEBUG] [VALIDATOR ] CLPR: All expected trading days are present. [2025-11-04T18:03:47.601098] [DEBUG] [VALIDATOR ] CLPR: Primary data validation passed. [2025-11-04T18:03:47.601521] [INFO ] [MAIN ] CLPR: Fetched primary data is valid. [2025-11-04T18:03:47.601597] [DEBUG] [LOCAL ] CLPR: Saving primary data. [2025-11-04T18:03:47.602423] [DEBUG] [LOCAL ] CLPR: Trimmed 15410 records to 9395 (keeping 200 days of data) [2025-11-04T18:03:47.657550] [DEBUG] [LOCAL ] CLPR: Primary data saved (9395 records). [2025-11-04T18:03:47.658069] [INFO ] [MAIN ] CLPR: Full history saved. [2025-11-04T18:03:47.658169] [INFO ] [MAIN ] Update Progress: 465/2367 symbols completed. CLPR took 23.9s. Estimated remaining: 4h 20m 43s. Rate: 7.3 symbols/min [2025-11-04T18:03:47.812730] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-07 (2.05 seconds). [2025-11-04T18:03:47.891371] [INFO ] [MAIN ] CLVT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:03:47.891539] [DEBUG] [LOCAL ] CLVT: No existing data file found in primary storage. [2025-11-04T18:03:47.891576] [WARN ] [MAIN ] CLVT: No existing data found, performing full refresh instead. [2025-11-04T18:03:47.891711] [INFO ] [MAIN ] CLVT: Wiping existing primary data before full refresh. [2025-11-04T18:03:47.891760] [INFO ] [LOCAL ] CLVT: Starting deletion of primary data. [2025-11-04T18:03:47.891855] [DEBUG] [LOCAL ] CLVT: No primary data file found to delete. [2025-11-04T18:03:47.891903] [INFO ] [MAIN ] CLVT: Starting full primary refresh. [2025-11-04T18:03:48.172203] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-08. [2025-11-04T18:03:48.245584] [INFO ] [REMOTE ] CLVT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:03:48.967075] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-01. [2025-11-04T18:03:49.345915] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-11-04T18:03:49.767017] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-05. [2025-11-04T18:03:49.880300] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-11-04T18:03:50.244193] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T18:03:50.567115] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-09. [2025-11-04T18:03:51.374513] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-02. [2025-11-04T18:03:51.404894] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-11-04T18:03:51.892612] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T18:03:52.167094] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-10. [2025-11-04T18:03:52.349203] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T18:03:52.967032] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-06. [2025-11-04T18:03:53.767020] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-08. [2025-11-04T18:03:53.847491] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-11-04T18:03:54.567027] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-11. [2025-11-04T18:03:54.636409] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-11-04T18:03:54.719411] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-11-04T18:03:55.367042] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-03. [2025-11-04T18:03:56.068131] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-11 (1.50 seconds). [2025-11-04T18:03:56.069606] [INFO ] [REMOTE ] CLF: Full fetch complete (77530 records). [2025-11-04T18:03:56.069734] [DEBUG] [VALIDATOR ] CLF: Validating primary data. [2025-11-04T18:03:56.070482] [FATAL] [VALIDATOR ] CLF: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:03:56.070566] [FATAL] [MAIN ] CLF: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:03:56.070620] [WARN ] [MAIN ] CLF: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:03:56.070875] [INFO ] [LOCAL ] CLF: Starting deletion of primary data. [2025-11-04T18:03:56.070954] [DEBUG] [LOCAL ] CLF: No primary data file found to delete. [2025-11-04T18:03:56.071004] [INFO ] [LOCAL ] CLF: Starting deletion of derivative data. [2025-11-04T18:03:56.071194] [DEBUG] [LOCAL ] CLF: Deleted derivative data file. [2025-11-04T18:03:56.071299] [INFO ] [MAIN ] Update Progress: 466/2367 symbols completed. CLF took 41.5s. Estimated remaining: 4h 20m 35s. Rate: 7.3 symbols/min [2025-11-04T18:03:56.172547] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-09. [2025-11-04T18:03:56.248099] [INFO ] [MAIN ] CLW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:03:56.273853] [DEBUG] [LOCAL ] CLW: Loaded 21349 existing records from primary storage. [2025-11-04T18:03:56.274177] [INFO ] [MAIN ] CLW: Loaded 21349 existing records. [2025-11-04T18:03:56.484774] [WARN ] [MAIN ] CLW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:03:56.484991] [INFO ] [MAIN ] CLW: Wiping existing primary data before full refresh. [2025-11-04T18:03:56.485038] [INFO ] [LOCAL ] CLW: Starting deletion of primary data. [2025-11-04T18:03:56.485218] [DEBUG] [LOCAL ] CLW: Deleted primary data file. [2025-11-04T18:03:56.485274] [INFO ] [MAIN ] CLW: Starting full primary refresh. [2025-11-04T18:03:56.840527] [INFO ] [REMOTE ] CLW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:03:56.879699] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-11-04T18:03:56.967040] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-01. [2025-11-04T18:03:57.209780] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-11-04T18:03:57.767030] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-04. [2025-11-04T18:03:57.967170] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T18:03:58.567045] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-10. [2025-11-04T18:03:58.859444] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T18:03:59.367036] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-07. [2025-11-04T18:03:59.460131] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-11-04T18:04:00.167035] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-11. [2025-11-04T18:04:00.967042] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-02. [2025-11-04T18:04:01.183346] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T18:04:01.184895] [INFO ] [REMOTE ] CLH: Full fetch complete (42823 records). [2025-11-04T18:04:01.184989] [DEBUG] [VALIDATOR ] CLH: Validating primary data. [2025-11-04T18:04:01.581345] [DEBUG] [VALIDATOR ] CLH: All expected trading days are present. [2025-11-04T18:04:01.581529] [DEBUG] [VALIDATOR ] CLH: Primary data validation passed. [2025-11-04T18:04:01.582387] [INFO ] [MAIN ] CLH: Fetched primary data is valid. [2025-11-04T18:04:01.582619] [DEBUG] [LOCAL ] CLH: Saving primary data. [2025-11-04T18:04:01.584207] [DEBUG] [LOCAL ] CLH: Trimmed 42823 records to 27561 (keeping 200 days of data) [2025-11-04T18:04:01.734426] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-07 (2.37 seconds). [2025-11-04T18:04:01.769396] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-05. [2025-11-04T18:04:01.837421] [DEBUG] [LOCAL ] CLH: Primary data saved (27561 records). [2025-11-04T18:04:01.837772] [INFO ] [MAIN ] CLH: Full history saved. [2025-11-04T18:04:01.837964] [INFO ] [MAIN ] Update Progress: 467/2367 symbols completed. CLH took 39.6s. Estimated remaining: 4h 20m 17s. Rate: 7.3 symbols/min [2025-11-04T18:04:01.932538] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T18:04:02.067263] [INFO ] [MAIN ] CLX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:04:02.123235] [DEBUG] [LOCAL ] CLX: Loaded 51962 existing records from primary storage. [2025-11-04T18:04:02.123612] [INFO ] [MAIN ] CLX: Loaded 51962 existing records. [2025-11-04T18:04:02.339051] [WARN ] [MAIN ] CLX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:04:02.339293] [INFO ] [MAIN ] CLX: Wiping existing primary data before full refresh. [2025-11-04T18:04:02.339360] [INFO ] [LOCAL ] CLX: Starting deletion of primary data. [2025-11-04T18:04:02.339576] [DEBUG] [LOCAL ] CLX: Deleted primary data file. [2025-11-04T18:04:02.339635] [INFO ] [MAIN ] CLX: Starting full primary refresh. [2025-11-04T18:04:02.572282] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-03. [2025-11-04T18:04:02.782512] [INFO ] [REMOTE ] CLX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:04:03.367016] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-01. [2025-11-04T18:04:03.409208] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-11-04T18:04:03.437347] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-11-04T18:04:04.167092] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-06. [2025-11-04T18:04:04.803102] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T18:04:04.967102] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-02. [2025-11-04T18:04:05.285712] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T18:04:05.767081] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-04. [2025-11-04T18:04:06.208605] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T18:04:06.567019] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-03. [2025-11-04T18:04:06.703902] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-11-04T18:04:07.367039] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-05. [2025-11-04T18:04:07.894499] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T18:04:08.167020] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-07. [2025-11-04T18:04:08.320920] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-11-04T18:04:08.967032] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-06. [2025-11-04T18:04:09.481545] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T18:04:09.710073] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-11-04T18:04:09.767001] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-07. [2025-11-04T18:04:10.505883] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T18:04:10.567030] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-04. [2025-11-04T18:04:11.367067] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-08. [2025-11-04T18:04:11.863662] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T18:04:12.167142] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-05. [2025-11-04T18:04:12.633970] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T18:04:12.967054] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-09. [2025-11-04T18:04:13.555196] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-11-04T18:04:13.767047] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-06. [2025-11-04T18:04:14.330481] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T18:04:14.567092] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-10. [2025-11-04T18:04:15.362373] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-11-04T18:04:15.372224] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-08. [2025-11-04T18:04:15.850753] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-11-04T18:04:16.167086] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-07. [2025-11-04T18:04:16.967072] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-11. [2025-11-04T18:04:17.363530] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-11-04T18:04:17.679251] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T18:04:17.766998] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-08. [2025-11-04T18:04:17.920195] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-11 (0.95 seconds). [2025-11-04T18:04:17.922200] [INFO ] [REMOTE ] CLVT: Full fetch complete (75433 records). [2025-11-04T18:04:17.922335] [DEBUG] [VALIDATOR ] CLVT: Validating primary data. [2025-11-04T18:04:18.325031] [DEBUG] [VALIDATOR ] CLVT: All expected trading days are present. [2025-11-04T18:04:18.325116] [DEBUG] [VALIDATOR ] CLVT: Primary data validation passed. [2025-11-04T18:04:18.325920] [INFO ] [MAIN ] CLVT: Fetched primary data is valid. [2025-11-04T18:04:18.325982] [DEBUG] [LOCAL ] CLVT: Saving primary data. [2025-11-04T18:04:18.327509] [DEBUG] [LOCAL ] CLVT: Trimmed 75433 records to 51869 (keeping 200 days of data) [2025-11-04T18:04:18.569788] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-08. [2025-11-04T18:04:18.612282] [DEBUG] [LOCAL ] CLVT: Primary data saved (51869 records). [2025-11-04T18:04:18.612375] [INFO ] [MAIN ] CLVT: Full history saved. [2025-11-04T18:04:18.612444] [INFO ] [MAIN ] Update Progress: 468/2367 symbols completed. CLVT took 31.0s. Estimated remaining: 4h 20m 44s. Rate: 7.3 symbols/min [2025-11-04T18:04:18.789508] [INFO ] [MAIN ] CM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:04:18.789634] [DEBUG] [LOCAL ] CM: No existing data file found in primary storage. [2025-11-04T18:04:18.789710] [WARN ] [MAIN ] CM: No existing data found, performing full refresh instead. [2025-11-04T18:04:18.789820] [INFO ] [MAIN ] CM: Wiping existing primary data before full refresh. [2025-11-04T18:04:18.789863] [INFO ] [LOCAL ] CM: Starting deletion of primary data. [2025-11-04T18:04:18.789905] [DEBUG] [LOCAL ] CM: No primary data file found to delete. [2025-11-04T18:04:18.789938] [INFO ] [MAIN ] CM: Starting full primary refresh. [2025-11-04T18:04:19.157826] [INFO ] [REMOTE ] CM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:04:19.190304] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T18:04:19.367092] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-09. [2025-11-04T18:04:19.647541] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-11-04T18:04:20.167111] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-09. [2025-11-04T18:04:20.967023] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-01. [2025-11-04T18:04:21.085696] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-11-04T18:04:21.475523] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T18:04:21.767077] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-10. [2025-11-04T18:04:22.063864] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-11-04T18:04:22.567053] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-02. [2025-11-04T18:04:23.367071] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-10. [2025-11-04T18:04:23.705733] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T18:04:24.046114] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-10 (2.28 seconds). [2025-11-04T18:04:24.167038] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-11. [2025-11-04T18:04:24.691668] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T18:04:24.967085] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-11. [2025-11-04T18:04:25.426839] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-11 (0.46 seconds). [2025-11-04T18:04:25.428290] [INFO ] [REMOTE ] CLX: Full fetch complete (73848 records). [2025-11-04T18:04:25.428403] [DEBUG] [VALIDATOR ] CLX: Validating primary data. [2025-11-04T18:04:25.429295] [FATAL] [VALIDATOR ] CLX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:04:25.429385] [FATAL] [MAIN ] CLX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:04:25.429435] [WARN ] [MAIN ] CLX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:04:25.429539] [INFO ] [LOCAL ] CLX: Starting deletion of primary data. [2025-11-04T18:04:25.429617] [DEBUG] [LOCAL ] CLX: No primary data file found to delete. [2025-11-04T18:04:25.429710] [INFO ] [LOCAL ] CLX: Starting deletion of derivative data. [2025-11-04T18:04:25.429882] [DEBUG] [LOCAL ] CLX: Deleted derivative data file. [2025-11-04T18:04:25.429979] [INFO ] [MAIN ] Update Progress: 469/2367 symbols completed. CLX took 23.6s. Estimated remaining: 4h 20m 30s. Rate: 7.3 symbols/min [2025-11-04T18:04:25.643578] [INFO ] [MAIN ] CMA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:04:25.686098] [DEBUG] [LOCAL ] CMA: Loaded 50069 existing records from primary storage. [2025-11-04T18:04:25.686495] [INFO ] [MAIN ] CMA: Loaded 50069 existing records. [2025-11-04T18:04:25.772162] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-03. [2025-11-04T18:04:25.877944] [WARN ] [MAIN ] CMA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:04:25.878163] [INFO ] [MAIN ] CMA: Wiping existing primary data before full refresh. [2025-11-04T18:04:25.878211] [INFO ] [LOCAL ] CMA: Starting deletion of primary data. [2025-11-04T18:04:25.878479] [DEBUG] [LOCAL ] CMA: Deleted primary data file. [2025-11-04T18:04:25.878537] [INFO ] [MAIN ] CMA: Starting full primary refresh. [2025-11-04T18:04:25.996101] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-11 (1.83 seconds). [2025-11-04T18:04:26.027989] [INFO ] [REMOTE ] CLS: Full fetch complete (77689 records). [2025-11-04T18:04:26.048190] [DEBUG] [VALIDATOR ] CLS: Validating primary data. [2025-11-04T18:04:26.395356] [INFO ] [REMOTE ] CMA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:04:26.572313] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-09. [2025-11-04T18:04:26.642509] [DEBUG] [VALIDATOR ] CLS: All expected trading days are present. [2025-11-04T18:04:26.642591] [DEBUG] [VALIDATOR ] CLS: Primary data validation passed. [2025-11-04T18:04:26.643435] [INFO ] [MAIN ] CLS: Fetched primary data is valid. [2025-11-04T18:04:26.643497] [DEBUG] [LOCAL ] CLS: Saving primary data. [2025-11-04T18:04:26.644947] [DEBUG] [LOCAL ] CLS: Trimmed 77689 records to 53525 (keeping 200 days of data) [2025-11-04T18:04:26.973265] [DEBUG] [LOCAL ] CLS: Primary data saved (53525 records). [2025-11-04T18:04:26.973403] [INFO ] [MAIN ] CLS: Full history saved. [2025-11-04T18:04:26.973474] [INFO ] [MAIN ] Update Progress: 470/2367 symbols completed. CLS took 52.9s. Estimated remaining: 4h 19m 54s. Rate: 7.3 symbols/min [2025-11-04T18:04:27.150475] [INFO ] [MAIN ] CMBT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:04:27.182875] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T18:04:27.213686] [DEBUG] [LOCAL ] CMBT: Loaded 29786 existing records from primary storage. [2025-11-04T18:04:27.213965] [INFO ] [MAIN ] CMBT: Loaded 29786 existing records. [2025-11-04T18:04:27.374823] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-01. [2025-11-04T18:04:27.396434] [WARN ] [MAIN ] CMBT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:04:27.396974] [INFO ] [MAIN ] CMBT: Wiping existing primary data before full refresh. [2025-11-04T18:04:27.397150] [INFO ] [LOCAL ] CMBT: Starting deletion of primary data. [2025-11-04T18:04:27.397464] [DEBUG] [LOCAL ] CMBT: Deleted primary data file. [2025-11-04T18:04:27.397523] [INFO ] [MAIN ] CMBT: Starting full primary refresh. [2025-11-04T18:04:27.557478] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T18:04:27.786759] [INFO ] [REMOTE ] CMBT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:04:28.167089] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-10. [2025-11-04T18:04:28.828291] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T18:04:28.967046] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-01. [2025-11-04T18:04:29.394348] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T18:04:29.683456] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-11-04T18:04:29.767000] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-02. [2025-11-04T18:04:30.392451] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-11-04T18:04:30.567034] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-02. [2025-11-04T18:04:31.367040] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-11. [2025-11-04T18:04:31.889622] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-11-04T18:04:32.167085] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-04. [2025-11-04T18:04:32.231987] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T18:04:32.233344] [INFO ] [REMOTE ] CLW: Full fetch complete (33125 records). [2025-11-04T18:04:32.233440] [DEBUG] [VALIDATOR ] CLW: Validating primary data. [2025-11-04T18:04:32.234044] [FATAL] [VALIDATOR ] CLW: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:04:32.234143] [FATAL] [MAIN ] CLW: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:04:32.234200] [WARN ] [MAIN ] CLW: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:04:32.234280] [INFO ] [LOCAL ] CLW: Starting deletion of primary data. [2025-11-04T18:04:32.234360] [DEBUG] [LOCAL ] CLW: No primary data file found to delete. [2025-11-04T18:04:32.234409] [INFO ] [LOCAL ] CLW: Starting deletion of derivative data. [2025-11-04T18:04:32.234565] [DEBUG] [LOCAL ] CLW: Deleted derivative data file. [2025-11-04T18:04:32.234687] [INFO ] [MAIN ] Update Progress: 471/2367 symbols completed. CLW took 36.2s. Estimated remaining: 4h 19m 34s. Rate: 7.3 symbols/min [2025-11-04T18:04:32.409855] [INFO ] [MAIN ] CMC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:04:32.410109] [DEBUG] [LOCAL ] CMC: No existing data file found in primary storage. [2025-11-04T18:04:32.410159] [WARN ] [MAIN ] CMC: No existing data found, performing full refresh instead. [2025-11-04T18:04:32.410283] [INFO ] [MAIN ] CMC: Wiping existing primary data before full refresh. [2025-11-04T18:04:32.410325] [INFO ] [LOCAL ] CMC: Starting deletion of primary data. [2025-11-04T18:04:32.410366] [DEBUG] [LOCAL ] CMC: No primary data file found to delete. [2025-11-04T18:04:32.410399] [INFO ] [MAIN ] CMC: Starting full primary refresh. [2025-11-04T18:04:32.793264] [INFO ] [REMOTE ] CMC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:04:32.967056] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-03. [2025-11-04T18:04:33.477836] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T18:04:33.645251] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-11-04T18:04:33.767042] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-05. [2025-11-04T18:04:34.567036] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-04. [2025-11-04T18:04:35.060919] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T18:04:35.325724] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-11-04T18:04:35.367081] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-01. [2025-11-04T18:04:36.167030] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-05. [2025-11-04T18:04:36.666251] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T18:04:36.956910] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-11-04T18:04:36.966996] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-06. [2025-11-04T18:04:37.652344] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-11-04T18:04:37.767016] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-06. [2025-11-04T18:04:38.567170] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-02. [2025-11-04T18:04:38.938269] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T18:04:39.367052] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-07. [2025-11-04T18:04:39.851745] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T18:04:40.167040] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-03. [2025-11-04T18:04:40.567631] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T18:04:40.967031] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-03. [2025-11-04T18:04:41.674542] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-11-04T18:04:41.767013] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-07. [2025-11-04T18:04:42.223877] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-11-04T18:04:42.567012] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-04. [2025-11-04T18:04:42.597722] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T18:04:43.367044] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-08. [2025-11-04T18:04:44.005194] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T18:04:44.167067] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-05. [2025-11-04T18:04:44.457002] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-11-04T18:04:44.967084] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-08. [2025-11-04T18:04:45.381768] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T18:04:45.767083] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-04. [2025-11-04T18:04:46.299878] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-11-04T18:04:46.567109] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-06. [2025-11-04T18:04:46.984863] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T18:04:47.367049] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-09. [2025-11-04T18:04:47.840884] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T18:04:48.167043] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-05. [2025-11-04T18:04:48.619321] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T18:04:48.967046] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-07. [2025-11-04T18:04:49.279288] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T18:04:49.767063] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-06. [2025-11-04T18:04:50.068521] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T18:04:50.567087] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-09. [2025-11-04T18:04:51.027814] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-11-04T18:04:51.367058] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-08. [2025-11-04T18:04:51.851606] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T18:04:52.167085] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-07. [2025-11-04T18:04:52.506377] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T18:04:52.967034] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-10. [2025-11-04T18:04:53.767018] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-10. [2025-11-04T18:04:53.810095] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-11-04T18:04:54.509226] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-11-04T18:04:54.567003] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-11. [2025-11-04T18:04:54.993352] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T18:04:55.367070] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-11. [2025-11-04T18:04:56.167105] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-09. [2025-11-04T18:04:56.487331] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T18:04:56.489252] [INFO ] [REMOTE ] CM: Full fetch complete (69603 records). [2025-11-04T18:04:56.489377] [DEBUG] [VALIDATOR ] CM: Validating primary data. [2025-11-04T18:04:56.618467] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-11 (2.05 seconds). [2025-11-04T18:04:56.623462] [INFO ] [REMOTE ] CMBT: Full fetch complete (35816 records). [2025-11-04T18:04:56.634150] [DEBUG] [VALIDATOR ] CMBT: Validating primary data. [2025-11-04T18:04:56.651089] [FATAL] [VALIDATOR ] CMBT: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:04:56.678542] [FATAL] [MAIN ] CMBT: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:04:56.685272] [WARN ] [MAIN ] CMBT: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:04:56.720265] [INFO ] [LOCAL ] CMBT: Starting deletion of primary data. [2025-11-04T18:04:56.740073] [DEBUG] [LOCAL ] CMBT: No primary data file found to delete. [2025-11-04T18:04:56.780098] [INFO ] [LOCAL ] CMBT: Starting deletion of derivative data. [2025-11-04T18:04:56.812316] [DEBUG] [LOCAL ] CMBT: Deleted derivative data file. [2025-11-04T18:04:56.822494] [INFO ] [MAIN ] Update Progress: 472/2367 symbols completed. CMBT took 29.8s. Estimated remaining: 4h 20m 32s. Rate: 7.3 symbols/min [2025-11-04T18:04:56.971891] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-08. [2025-11-04T18:04:57.095898] [DEBUG] [VALIDATOR ] CM: All expected trading days are present. [2025-11-04T18:04:57.101116] [DEBUG] [VALIDATOR ] CM: Primary data validation passed. [2025-11-04T18:04:57.119409] [INFO ] [MAIN ] CM: Fetched primary data is valid. [2025-11-04T18:04:57.131372] [DEBUG] [LOCAL ] CM: Saving primary data. [2025-11-04T18:04:57.150637] [INFO ] [MAIN ] CMCM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:04:57.151788] [DEBUG] [LOCAL ] CM: Trimmed 69603 records to 48017 (keeping 200 days of data) [2025-11-04T18:04:57.255844] [DEBUG] [LOCAL ] CMCM: Loaded 4815 existing records from primary storage. [2025-11-04T18:04:57.259012] [INFO ] [MAIN ] CMCM: Loaded 4815 existing records. [2025-11-04T18:04:57.464194] [WARN ] [MAIN ] CMCM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:04:57.470676] [INFO ] [MAIN ] CMCM: Wiping existing primary data before full refresh. [2025-11-04T18:04:57.470879] [INFO ] [LOCAL ] CMCM: Starting deletion of primary data. [2025-11-04T18:04:57.471087] [DEBUG] [LOCAL ] CMCM: Deleted primary data file. [2025-11-04T18:04:57.471168] [INFO ] [MAIN ] CMCM: Starting full primary refresh. [2025-11-04T18:04:57.883471] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-11-04T18:04:57.971698] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-10. [2025-11-04T18:04:58.024544] [DEBUG] [LOCAL ] CM: Primary data saved (48017 records). [2025-11-04T18:04:58.062492] [INFO ] [REMOTE ] CMCM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:04:58.058595] [INFO ] [MAIN ] CM: Full history saved. [2025-11-04T18:04:58.063036] [INFO ] [MAIN ] Update Progress: 473/2367 symbols completed. CM took 39.5s. Estimated remaining: 4h 19m 56s. Rate: 7.3 symbols/min [2025-11-04T18:04:58.240529] [INFO ] [MAIN ] CMDB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:04:58.240853] [DEBUG] [LOCAL ] CMDB: No existing data file found in primary storage. [2025-11-04T18:04:58.240924] [WARN ] [MAIN ] CMDB: No existing data found, performing full refresh instead. [2025-11-04T18:04:58.241140] [INFO ] [MAIN ] CMDB: Wiping existing primary data before full refresh. [2025-11-04T18:04:58.241201] [INFO ] [LOCAL ] CMDB: Starting deletion of primary data. [2025-11-04T18:04:58.241277] [DEBUG] [LOCAL ] CMDB: No primary data file found to delete. [2025-11-04T18:04:58.241331] [INFO ] [MAIN ] CMDB: Starting full primary refresh. [2025-11-04T18:04:58.448195] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T18:04:58.572134] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-01. [2025-11-04T18:04:58.641574] [INFO ] [REMOTE ] CMDB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:04:59.064122] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-01 (0.49 seconds). [2025-11-04T18:04:59.367015] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-09. [2025-11-04T18:04:59.432817] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T18:05:00.167041] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-01. [2025-11-04T18:05:00.370492] [DEBUG] [REMOTE ] CMDB: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:05:00.370824] [WARN ] [REMOTE ] CMDB: No data returned for month 2025-01 [2025-11-04T18:05:00.370911] [FATAL] [REMOTE ] CMDB: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:05:00.370963] [FATAL] [MAIN ] CMDB: Failed to fetch primary data. [2025-11-04T18:05:00.371034] [WARN ] [MAIN ] CMDB: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:05:00.371107] [INFO ] [LOCAL ] CMDB: Starting deletion of primary data. [2025-11-04T18:05:00.371168] [DEBUG] [LOCAL ] CMDB: No primary data file found to delete. [2025-11-04T18:05:00.371203] [INFO ] [LOCAL ] CMDB: Starting deletion of derivative data. [2025-11-04T18:05:00.371245] [DEBUG] [LOCAL ] CMDB: No derivative data file found to delete. [2025-11-04T18:05:00.371290] [INFO ] [MAIN ] Update Progress: 474/2367 symbols completed. CMDB took 2.3s. Estimated remaining: 4h 19m 24s. Rate: 7.3 symbols/min [2025-11-04T18:05:00.544601] [INFO ] [MAIN ] CMG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:05:00.544847] [DEBUG] [LOCAL ] CMG: No existing data file found in primary storage. [2025-11-04T18:05:00.544928] [WARN ] [MAIN ] CMG: No existing data found, performing full refresh instead. [2025-11-04T18:05:00.545067] [INFO ] [MAIN ] CMG: Wiping existing primary data before full refresh. [2025-11-04T18:05:00.545127] [INFO ] [LOCAL ] CMG: Starting deletion of primary data. [2025-11-04T18:05:00.545229] [DEBUG] [LOCAL ] CMG: No primary data file found to delete. [2025-11-04T18:05:00.545274] [INFO ] [MAIN ] CMG: Starting full primary refresh. [2025-11-04T18:05:00.932277] [INFO ] [REMOTE ] CMG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:05:00.927978] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-11-04T18:05:00.966995] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-01. [2025-11-04T18:05:01.767042] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-11. [2025-11-04T18:05:02.567012] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-02. [2025-11-04T18:05:02.666636] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-11-04T18:05:02.896078] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T18:05:02.897958] [INFO ] [REMOTE ] CMC: Full fetch complete (67114 records). [2025-11-04T18:05:02.898042] [DEBUG] [VALIDATOR ] CMC: Validating primary data. [2025-11-04T18:05:03.197681] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-11-04T18:05:03.308294] [DEBUG] [VALIDATOR ] CMC: All expected trading days are present. [2025-11-04T18:05:03.308381] [DEBUG] [VALIDATOR ] CMC: Primary data validation passed. [2025-11-04T18:05:03.309303] [INFO ] [MAIN ] CMC: Fetched primary data is valid. [2025-11-04T18:05:03.309934] [DEBUG] [LOCAL ] CMC: Saving primary data. [2025-11-04T18:05:03.311578] [DEBUG] [LOCAL ] CMC: Trimmed 67114 records to 45482 (keeping 200 days of data) [2025-11-04T18:05:03.370017] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-03. [2025-11-04T18:05:03.597868] [DEBUG] [LOCAL ] CMC: Primary data saved (45482 records). [2025-11-04T18:05:03.598008] [INFO ] [MAIN ] CMC: Full history saved. [2025-11-04T18:05:03.598084] [INFO ] [MAIN ] Update Progress: 475/2367 symbols completed. CMC took 31.4s. Estimated remaining: 4h 18m 56s. Rate: 7.3 symbols/min [2025-11-04T18:05:03.775245] [INFO ] [MAIN ] CMI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:05:03.801226] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-11-04T18:05:03.817915] [DEBUG] [LOCAL ] CMI: Loaded 41394 existing records from primary storage. [2025-11-04T18:05:03.818223] [INFO ] [MAIN ] CMI: Loaded 41394 existing records. [2025-11-04T18:05:04.002998] [WARN ] [MAIN ] CMI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:05:04.003241] [INFO ] [MAIN ] CMI: Wiping existing primary data before full refresh. [2025-11-04T18:05:04.003299] [INFO ] [LOCAL ] CMI: Starting deletion of primary data. [2025-11-04T18:05:04.003780] [DEBUG] [LOCAL ] CMI: Deleted primary data file. [2025-11-04T18:05:04.003839] [INFO ] [MAIN ] CMI: Starting full primary refresh. [2025-11-04T18:05:04.176715] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-04. [2025-11-04T18:05:04.386320] [INFO ] [REMOTE ] CMI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:05:04.642638] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-11-04T18:05:04.967043] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-05. [2025-11-04T18:05:05.273575] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-11-04T18:05:05.767048] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-06. [2025-11-04T18:05:06.223164] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-11-04T18:05:06.567037] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-07. [2025-11-04T18:05:06.986846] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-11-04T18:05:07.367040] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-10. [2025-11-04T18:05:08.167030] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-01. [2025-11-04T18:05:08.871190] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-11-04T18:05:08.967033] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-11. [2025-11-04T18:05:09.530378] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T18:05:09.767037] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-02. [2025-11-04T18:05:10.216196] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T18:05:10.218172] [INFO ] [REMOTE ] CMA: Full fetch complete (73434 records). [2025-11-04T18:05:10.218262] [DEBUG] [VALIDATOR ] CMA: Validating primary data. [2025-11-04T18:05:10.574241] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-02. [2025-11-04T18:05:10.615993] [DEBUG] [VALIDATOR ] CMA: All expected trading days are present. [2025-11-04T18:05:10.616089] [DEBUG] [VALIDATOR ] CMA: Primary data validation passed. [2025-11-04T18:05:10.616918] [INFO ] [MAIN ] CMA: Fetched primary data is valid. [2025-11-04T18:05:10.616980] [DEBUG] [LOCAL ] CMA: Saving primary data. [2025-11-04T18:05:10.618535] [DEBUG] [LOCAL ] CMA: Trimmed 73434 records to 50065 (keeping 200 days of data) [2025-11-04T18:05:10.919679] [DEBUG] [LOCAL ] CMA: Primary data saved (50065 records). [2025-11-04T18:05:10.919816] [INFO ] [MAIN ] CMA: Full history saved. [2025-11-04T18:05:10.919894] [INFO ] [MAIN ] Update Progress: 476/2367 symbols completed. CMA took 45.5s. Estimated remaining: 4h 18m 44s. Rate: 7.3 symbols/min [2025-11-04T18:05:11.097468] [INFO ] [MAIN ] CMP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:05:11.123273] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T18:05:11.159665] [DEBUG] [LOCAL ] CMP: Loaded 34724 existing records from primary storage. [2025-11-04T18:05:11.159941] [INFO ] [MAIN ] CMP: Loaded 34724 existing records. [2025-11-04T18:05:11.337394] [WARN ] [MAIN ] CMP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:05:11.337637] [INFO ] [MAIN ] CMP: Wiping existing primary data before full refresh. [2025-11-04T18:05:11.337720] [INFO ] [LOCAL ] CMP: Starting deletion of primary data. [2025-11-04T18:05:11.337918] [DEBUG] [LOCAL ] CMP: Deleted primary data file. [2025-11-04T18:05:11.337974] [INFO ] [MAIN ] CMP: Starting full primary refresh. [2025-11-04T18:05:11.372133] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-03. [2025-11-04T18:05:11.733460] [INFO ] [REMOTE ] CMP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:05:12.175201] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-08. [2025-11-04T18:05:12.203943] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-11-04T18:05:12.708991] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T18:05:12.780125] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-11-04T18:05:12.967025] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-09. [2025-11-04T18:05:13.565806] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-11-04T18:05:13.767052] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-10. [2025-11-04T18:05:14.144729] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-11-04T18:05:14.567033] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-01. [2025-11-04T18:05:15.367071] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-11. [2025-11-04T18:05:15.611355] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-11-04T18:05:15.860448] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-11 (0.49 seconds). [2025-11-04T18:05:15.861138] [INFO ] [REMOTE ] CMCM: Full fetch complete (7038 records). [2025-11-04T18:05:15.861204] [DEBUG] [VALIDATOR ] CMCM: Validating primary data. [2025-11-04T18:05:15.861544] [FATAL] [VALIDATOR ] CMCM: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:05:15.861622] [FATAL] [MAIN ] CMCM: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:05:15.861728] [WARN ] [MAIN ] CMCM: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:05:15.861802] [INFO ] [LOCAL ] CMCM: Starting deletion of primary data. [2025-11-04T18:05:15.861877] [DEBUG] [LOCAL ] CMCM: No primary data file found to delete. [2025-11-04T18:05:15.861928] [INFO ] [LOCAL ] CMCM: Starting deletion of derivative data. [2025-11-04T18:05:15.862069] [DEBUG] [LOCAL ] CMCM: Deleted derivative data file. [2025-11-04T18:05:15.862162] [INFO ] [MAIN ] Update Progress: 477/2367 symbols completed. CMCM took 19.0s. Estimated remaining: 4h 18m 23s. Rate: 7.3 symbols/min [2025-11-04T18:05:16.167079] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-04. [2025-11-04T18:05:16.967108] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-02. [2025-11-04T18:05:17.420489] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T18:05:17.767019] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-05. [2025-11-04T18:05:17.935842] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T18:05:18.567031] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-03. [2025-11-04T18:05:18.891823] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-11-04T18:05:19.367007] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-06. [2025-11-04T18:05:19.572085] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T18:05:20.167015] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-04. [2025-11-04T18:05:20.392548] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T18:05:20.967072] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-07. [2025-11-04T18:05:21.188386] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T18:05:21.767039] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-03. [2025-11-04T18:05:22.104969] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T18:05:22.567028] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-08. [2025-11-04T18:05:23.367043] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-05. [2025-11-04T18:05:23.688738] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-11-04T18:05:23.787447] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T18:05:24.167028] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-09. [2025-11-04T18:05:24.579713] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T18:05:24.967037] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-06. [2025-11-04T18:05:25.767019] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-04. [2025-11-04T18:05:26.046824] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-11-04T18:05:26.148038] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T18:05:26.567048] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-10. [2025-11-04T18:05:27.367056] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-07. [2025-11-04T18:05:27.546376] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-11-04T18:05:27.821694] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T18:05:28.167061] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-11. [2025-11-04T18:05:28.416284] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-11-04T18:05:28.967062] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-05. [2025-11-04T18:05:29.395002] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T18:05:29.396865] [INFO ] [REMOTE ] CMI: Full fetch complete (61324 records). [2025-11-04T18:05:29.396969] [DEBUG] [VALIDATOR ] CMI: Validating primary data. [2025-11-04T18:05:29.770593] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-08. [2025-11-04T18:05:29.770862] [DEBUG] [VALIDATOR ] CMI: All expected trading days are present. [2025-11-04T18:05:29.771675] [DEBUG] [VALIDATOR ] CMI: Primary data validation passed. [2025-11-04T18:05:29.772374] [INFO ] [MAIN ] CMI: Fetched primary data is valid. [2025-11-04T18:05:29.772416] [DEBUG] [LOCAL ] CMI: Saving primary data. [2025-11-04T18:05:29.773671] [DEBUG] [LOCAL ] CMI: Trimmed 61324 records to 41459 (keeping 200 days of data) [2025-11-04T18:05:30.024923] [DEBUG] [LOCAL ] CMI: Primary data saved (41459 records). [2025-11-04T18:05:30.025043] [INFO ] [MAIN ] CMI: Full history saved. [2025-11-04T18:05:30.025117] [INFO ] [MAIN ] Update Progress: 478/2367 symbols completed. CMI took 26.4s. Estimated remaining: 4h 18m 38s. Rate: 7.3 symbols/min [2025-11-04T18:05:30.625109] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-11-04T18:05:30.637924] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-06. [2025-11-04T18:05:30.658750] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T18:05:31.367055] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-09. [2025-11-04T18:05:32.259522] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-11-04T18:05:32.270702] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-07. [2025-11-04T18:05:32.295769] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-11-04T18:05:32.967080] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-10. [2025-11-04T18:05:33.893465] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-11-04T18:05:33.900421] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-11. [2025-11-04T18:05:34.159309] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-11-04T18:05:34.567121] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-08. [2025-11-04T18:05:34.771497] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T18:05:34.772915] [INFO ] [REMOTE ] CMP: Full fetch complete (51154 records). [2025-11-04T18:05:34.773028] [DEBUG] [VALIDATOR ] CMP: Validating primary data. [2025-11-04T18:05:35.147448] [DEBUG] [VALIDATOR ] CMP: All expected trading days are present. [2025-11-04T18:05:35.147560] [DEBUG] [VALIDATOR ] CMP: Primary data validation passed. [2025-11-04T18:05:35.148251] [INFO ] [MAIN ] CMP: Fetched primary data is valid. [2025-11-04T18:05:35.148324] [DEBUG] [LOCAL ] CMP: Saving primary data. [2025-11-04T18:05:35.149403] [DEBUG] [LOCAL ] CMP: Trimmed 51154 records to 34741 (keeping 200 days of data) [2025-11-04T18:05:35.359318] [DEBUG] [LOCAL ] CMP: Primary data saved (34741 records). [2025-11-04T18:05:35.359461] [INFO ] [MAIN ] CMP: Full history saved. [2025-11-04T18:05:35.359550] [INFO ] [MAIN ] Update Progress: 479/2367 symbols completed. CMP took 24.4s. Estimated remaining: 4h 18m 18s. Rate: 7.3 symbols/min [2025-11-04T18:05:36.435329] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-11-04T18:05:36.447925] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-09. [2025-11-04T18:05:38.397196] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-11-04T18:05:38.410131] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-10. [2025-11-04T18:05:42.650669] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-10 (4.24 seconds). [2025-11-04T18:05:42.664058] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-11. [2025-11-04T18:05:44.368839] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-11 (1.70 seconds). [2025-11-04T18:05:44.370866] [INFO ] [REMOTE ] CMG: Full fetch complete (77895 records). [2025-11-04T18:05:44.370977] [DEBUG] [VALIDATOR ] CMG: Validating primary data. [2025-11-04T18:05:44.758191] [DEBUG] [VALIDATOR ] CMG: All expected trading days are present. [2025-11-04T18:05:44.758299] [DEBUG] [VALIDATOR ] CMG: Primary data validation passed. [2025-11-04T18:05:44.759187] [INFO ] [MAIN ] CMG: Fetched primary data is valid. [2025-11-04T18:05:44.759251] [DEBUG] [LOCAL ] CMG: Saving primary data. [2025-11-04T18:05:44.760754] [DEBUG] [LOCAL ] CMG: Trimmed 77895 records to 53725 (keeping 200 days of data) [2025-11-04T18:05:45.077353] [DEBUG] [LOCAL ] CMG: Primary data saved (53725 records). [2025-11-04T18:05:45.077478] [INFO ] [MAIN ] CMG: Full history saved. [2025-11-04T18:05:45.077549] [INFO ] [MAIN ] Update Progress: 480/2367 symbols completed. CMG took 44.7s. Estimated remaining: 4h 18m 16s. Rate: 7.3 symbols/min [2025-11-04T18:05:45.762189] [INFO ] [MAIN ] CMS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:05:45.792178] [INFO ] [MAIN ] CMPO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:05:45.820282] [INFO ] [MAIN ] CMSA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:05:45.859108] [INFO ] [MAIN ] CMRE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:05:45.868271] [DEBUG] [LOCAL ] CMSA: Loaded 3356 existing records from primary storage. [2025-11-04T18:05:45.869005] [INFO ] [MAIN ] CMSA: Loaded 3356 existing records. [2025-11-04T18:05:46.084026] [DEBUG] [LOCAL ] CMRE: Loaded 35048 existing records from primary storage. [2025-11-04T18:05:46.132455] [WARN ] [MAIN ] CMSA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:05:46.136337] [INFO ] [MAIN ] CMSA: Wiping existing primary data before full refresh. [2025-11-04T18:05:46.136040] [DEBUG] [LOCAL ] CMPO: Loaded 37707 existing records from primary storage. [2025-11-04T18:05:46.133023] [INFO ] [MAIN ] CMRE: Loaded 35048 existing records. [2025-11-04T18:05:46.137301] [INFO ] [MAIN ] CMPO: Loaded 37707 existing records. [2025-11-04T18:05:46.136537] [INFO ] [LOCAL ] CMSA: Starting deletion of primary data. [2025-11-04T18:05:46.204527] [DEBUG] [LOCAL ] CMSA: Deleted primary data file. [2025-11-04T18:05:46.224717] [DEBUG] [LOCAL ] CMS: Loaded 52515 existing records from primary storage. [2025-11-04T18:05:46.204884] [INFO ] [MAIN ] CMSA: Starting full primary refresh. [2025-11-04T18:05:46.276573] [INFO ] [MAIN ] CMS: Loaded 52515 existing records. [2025-11-04T18:05:46.683812] [WARN ] [MAIN ] CMPO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:05:46.734186] [INFO ] [MAIN ] CMPO: Wiping existing primary data before full refresh. [2025-11-04T18:05:46.791046] [INFO ] [LOCAL ] CMPO: Starting deletion of primary data. [2025-11-04T18:05:46.839287] [DEBUG] [LOCAL ] CMPO: Deleted primary data file. [2025-11-04T18:05:46.848881] [WARN ] [MAIN ] CMRE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:05:46.839777] [INFO ] [MAIN ] CMPO: Starting full primary refresh. [2025-11-04T18:05:46.980130] [INFO ] [MAIN ] CMRE: Wiping existing primary data before full refresh. [2025-11-04T18:05:47.044699] [WARN ] [MAIN ] CMS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:05:47.039870] [INFO ] [LOCAL ] CMRE: Starting deletion of primary data. [2025-11-04T18:05:47.079068] [INFO ] [MAIN ] CMS: Wiping existing primary data before full refresh. [2025-11-04T18:05:47.093214] [DEBUG] [LOCAL ] CMRE: Deleted primary data file. [2025-11-04T18:05:47.108965] [INFO ] [LOCAL ] CMS: Starting deletion of primary data. [2025-11-04T18:05:47.114452] [INFO ] [MAIN ] CMRE: Starting full primary refresh. [2025-11-04T18:05:47.173733] [DEBUG] [LOCAL ] CMS: Deleted primary data file. [2025-11-04T18:05:47.244274] [INFO ] [MAIN ] CMS: Starting full primary refresh. [2025-11-04T18:05:47.709113] [INFO ] [REMOTE ] CMSA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:05:47.743508] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-01. [2025-11-04T18:05:48.106029] [INFO ] [REMOTE ] CMPO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:05:48.119907] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-01. [2025-11-04T18:05:48.252969] [INFO ] [REMOTE ] CMRE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:05:48.281857] [INFO ] [REMOTE ] CMS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:05:48.282238] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-01. [2025-11-04T18:05:48.282665] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-01. [2025-11-04T18:05:48.421505] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-11-04T18:05:48.422995] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-02. [2025-11-04T18:05:48.717069] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-11-04T18:05:48.718286] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-03. [2025-11-04T18:05:49.038524] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-11-04T18:05:49.040124] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-04. [2025-11-04T18:05:49.195526] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-11-04T18:05:49.202238] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-02. [2025-11-04T18:05:49.458099] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-11-04T18:05:49.461483] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-05. [2025-11-04T18:05:49.460850] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T18:05:49.471341] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-02. [2025-11-04T18:05:49.876351] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-11-04T18:05:49.876562] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-11-04T18:05:49.890102] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-06. [2025-11-04T18:05:49.890957] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-02. [2025-11-04T18:05:50.265054] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T18:05:50.272196] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-03. [2025-11-04T18:05:50.279817] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-11-04T18:05:50.281216] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-07. [2025-11-04T18:05:50.442242] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T18:05:50.450547] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-03. [2025-11-04T18:05:50.674778] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-11-04T18:05:50.676259] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-08. [2025-11-04T18:05:51.012427] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-11-04T18:05:51.013910] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-09. [2025-11-04T18:05:51.317906] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-11-04T18:05:51.367014] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-04. [2025-11-04T18:05:51.384871] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-11-04T18:05:51.995146] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-02 (2.10 seconds). [2025-11-04T18:05:52.167026] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-10. [2025-11-04T18:05:52.521254] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T18:05:52.568273] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-11-04T18:05:52.817219] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-03 (2.37 seconds). [2025-11-04T18:05:52.967035] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-04. [2025-11-04T18:05:53.767079] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-03. [2025-11-04T18:05:54.106335] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T18:05:54.567069] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-05. [2025-11-04T18:05:55.067070] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T18:05:55.367036] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-04. [2025-11-04T18:05:55.661793] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-11-04T18:05:56.167047] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-06. [2025-11-04T18:05:56.640465] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T18:05:56.967062] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-05. [2025-11-04T18:05:57.094844] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T18:05:57.767050] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-05. [2025-11-04T18:05:57.893074] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-11-04T18:05:58.567059] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-11. [2025-11-04T18:05:59.097815] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T18:05:59.220246] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-11 (0.65 seconds). [2025-11-04T18:05:59.221224] [INFO ] [REMOTE ] CMSA: Full fetch complete (4692 records). [2025-11-04T18:05:59.221318] [DEBUG] [VALIDATOR ] CMSA: Validating primary data. [2025-11-04T18:05:59.221886] [FATAL] [VALIDATOR ] CMSA: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:05:59.221988] [FATAL] [MAIN ] CMSA: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:05:59.222049] [WARN ] [MAIN ] CMSA: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:05:59.222111] [INFO ] [LOCAL ] CMSA: Starting deletion of primary data. [2025-11-04T18:05:59.222194] [DEBUG] [LOCAL ] CMSA: No primary data file found to delete. [2025-11-04T18:05:59.222302] [INFO ] [LOCAL ] CMSA: Starting deletion of derivative data. [2025-11-04T18:05:59.222513] [DEBUG] [LOCAL ] CMSA: Deleted derivative data file. [2025-11-04T18:05:59.222624] [INFO ] [MAIN ] Update Progress: 481/2367 symbols completed. CMSA took 14.1s. Estimated remaining: 4h 18m 31s. Rate: 7.3 symbols/min [2025-11-04T18:05:59.372169] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-06. [2025-11-04T18:05:59.471222] [INFO ] [MAIN ] CMSC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:05:59.479389] [DEBUG] [LOCAL ] CMSC: Loaded 4460 existing records from primary storage. [2025-11-04T18:05:59.479742] [INFO ] [MAIN ] CMSC: Loaded 4460 existing records. [2025-11-04T18:05:59.715205] [WARN ] [MAIN ] CMSC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:05:59.715467] [INFO ] [MAIN ] CMSC: Wiping existing primary data before full refresh. [2025-11-04T18:05:59.715515] [INFO ] [LOCAL ] CMSC: Starting deletion of primary data. [2025-11-04T18:05:59.715701] [DEBUG] [LOCAL ] CMSC: Deleted primary data file. [2025-11-04T18:05:59.715757] [INFO ] [MAIN ] CMSC: Starting full primary refresh. [2025-11-04T18:06:00.100370] [INFO ] [REMOTE ] CMSC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:06:00.167070] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-01. [2025-11-04T18:06:00.678864] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-11-04T18:06:00.788428] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-11-04T18:06:00.967036] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-07. [2025-11-04T18:06:01.767051] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-07. [2025-11-04T18:06:02.412840] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T18:06:02.567030] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-08. [2025-11-04T18:06:02.902583] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T18:06:03.367039] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-08. [2025-11-04T18:06:04.038496] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-11-04T18:06:04.167028] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-09. [2025-11-04T18:06:04.556961] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T18:06:04.967060] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-09. [2025-11-04T18:06:05.400739] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T18:06:05.767046] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-10. [2025-11-04T18:06:05.957514] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T18:06:06.567038] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-10. [2025-11-04T18:06:07.127586] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-11-04T18:06:07.367050] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-02. [2025-11-04T18:06:07.497911] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-11-04T18:06:07.714404] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-11-04T18:06:08.167044] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-11. [2025-11-04T18:06:08.466238] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-11 (0.30 seconds). [2025-11-04T18:06:08.469584] [INFO ] [REMOTE ] CMS: Full fetch complete (76290 records). [2025-11-04T18:06:08.469885] [DEBUG] [VALIDATOR ] CMS: Validating primary data. [2025-11-04T18:06:08.859226] [DEBUG] [VALIDATOR ] CMS: All expected trading days are present. [2025-11-04T18:06:08.859317] [DEBUG] [VALIDATOR ] CMS: Primary data validation passed. [2025-11-04T18:06:08.860265] [INFO ] [MAIN ] CMS: Fetched primary data is valid. [2025-11-04T18:06:08.860336] [DEBUG] [LOCAL ] CMS: Saving primary data. [2025-11-04T18:06:08.861907] [DEBUG] [LOCAL ] CMS: Trimmed 76290 records to 52519 (keeping 200 days of data) [2025-11-04T18:06:08.974717] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-03. [2025-11-04T18:06:09.170731] [DEBUG] [LOCAL ] CMS: Primary data saved (52519 records). [2025-11-04T18:06:09.170864] [INFO ] [MAIN ] CMS: Full history saved. [2025-11-04T18:06:09.170943] [INFO ] [MAIN ] Update Progress: 482/2367 symbols completed. CMS took 24.1s. Estimated remaining: 4h 18m 30s. Rate: 7.3 symbols/min [2025-11-04T18:06:09.336994] [INFO ] [MAIN ] CMSD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:06:09.345798] [DEBUG] [LOCAL ] CMSD: Loaded 7253 existing records from primary storage. [2025-11-04T18:06:09.345974] [INFO ] [MAIN ] CMSD: Loaded 7253 existing records. [2025-11-04T18:06:09.469063] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-11-04T18:06:09.533162] [WARN ] [MAIN ] CMSD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:06:09.533381] [INFO ] [MAIN ] CMSD: Wiping existing primary data before full refresh. [2025-11-04T18:06:09.533429] [INFO ] [LOCAL ] CMSD: Starting deletion of primary data. [2025-11-04T18:06:09.533570] [DEBUG] [LOCAL ] CMSD: Deleted primary data file. [2025-11-04T18:06:09.533607] [INFO ] [MAIN ] CMSD: Starting full primary refresh. [2025-11-04T18:06:09.772174] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-04. [2025-11-04T18:06:09.886079] [INFO ] [REMOTE ] CMSD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:06:10.292829] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-11-04T18:06:10.567044] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-06. [2025-11-04T18:06:11.367064] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-01. [2025-11-04T18:06:11.721618] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T18:06:12.040192] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-11-04T18:06:12.167038] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-07. [2025-11-04T18:06:12.967018] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-11. [2025-11-04T18:06:13.180023] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T18:06:13.767067] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-05. [2025-11-04T18:06:13.943943] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-11 (0.98 seconds). [2025-11-04T18:06:13.945412] [INFO ] [REMOTE ] CMPO: Full fetch complete (57189 records). [2025-11-04T18:06:13.945493] [DEBUG] [VALIDATOR ] CMPO: Validating primary data. [2025-11-04T18:06:13.946121] [FATAL] [VALIDATOR ] CMPO: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:06:13.946206] [FATAL] [MAIN ] CMPO: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:06:13.946260] [WARN ] [MAIN ] CMPO: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:06:13.946334] [INFO ] [LOCAL ] CMPO: Starting deletion of primary data. [2025-11-04T18:06:13.946396] [DEBUG] [LOCAL ] CMPO: No primary data file found to delete. [2025-11-04T18:06:13.946432] [INFO ] [LOCAL ] CMPO: Starting deletion of derivative data. [2025-11-04T18:06:13.946583] [DEBUG] [LOCAL ] CMPO: Deleted derivative data file. [2025-11-04T18:06:13.946695] [INFO ] [MAIN ] Update Progress: 483/2367 symbols completed. CMPO took 28.9s. Estimated remaining: 4h 18m 8s. Rate: 7.3 symbols/min [2025-11-04T18:06:14.134164] [INFO ] [MAIN ] CMTG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:06:14.167483] [DEBUG] [LOCAL ] CMTG: Loaded 33769 existing records from primary storage. [2025-11-04T18:06:14.167759] [INFO ] [MAIN ] CMTG: Loaded 33769 existing records. [2025-11-04T18:06:14.285305] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-11-04T18:06:14.355258] [WARN ] [MAIN ] CMTG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:06:14.355479] [INFO ] [MAIN ] CMTG: Wiping existing primary data before full refresh. [2025-11-04T18:06:14.355545] [INFO ] [LOCAL ] CMTG: Starting deletion of primary data. [2025-11-04T18:06:14.355763] [DEBUG] [LOCAL ] CMTG: Deleted primary data file. [2025-11-04T18:06:14.355822] [INFO ] [MAIN ] CMTG: Starting full primary refresh. [2025-11-04T18:06:14.572144] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-06. [2025-11-04T18:06:14.737428] [INFO ] [REMOTE ] CMTG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:06:14.949525] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-11-04T18:06:15.367046] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-01. [2025-11-04T18:06:16.167050] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-08. [2025-11-04T18:06:16.424720] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T18:06:16.967035] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-02. [2025-11-04T18:06:17.023724] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-11-04T18:06:17.385946] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-11-04T18:06:17.767055] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-07. [2025-11-04T18:06:18.167963] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-11-04T18:06:18.567058] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-09. [2025-11-04T18:06:19.367031] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-08. [2025-11-04T18:06:19.481733] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-11-04T18:06:19.805442] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T18:06:20.167044] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-03. [2025-11-04T18:06:20.683102] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-11-04T18:06:20.967045] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-04. [2025-11-04T18:06:21.470099] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-11-04T18:06:21.767073] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-10. [2025-11-04T18:06:22.567038] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-02. [2025-11-04T18:06:22.908710] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T18:06:23.367044] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-11. [2025-11-04T18:06:23.431871] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-11-04T18:06:24.167054] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-03. [2025-11-04T18:06:24.214286] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-11 (0.85 seconds). [2025-11-04T18:06:24.216200] [INFO ] [REMOTE ] CMRE: Full fetch complete (51328 records). [2025-11-04T18:06:24.216340] [DEBUG] [VALIDATOR ] CMRE: Validating primary data. [2025-11-04T18:06:24.596809] [DEBUG] [VALIDATOR ] CMRE: All expected trading days are present. [2025-11-04T18:06:24.596901] [DEBUG] [VALIDATOR ] CMRE: Primary data validation passed. [2025-11-04T18:06:24.597607] [INFO ] [MAIN ] CMRE: Fetched primary data is valid. [2025-11-04T18:06:24.597694] [DEBUG] [LOCAL ] CMRE: Saving primary data. [2025-11-04T18:06:24.598796] [DEBUG] [LOCAL ] CMRE: Trimmed 51328 records to 35079 (keeping 200 days of data) [2025-11-04T18:06:24.803915] [DEBUG] [LOCAL ] CMRE: Primary data saved (35079 records). [2025-11-04T18:06:24.804057] [INFO ] [MAIN ] CMRE: Full history saved. [2025-11-04T18:06:24.804136] [INFO ] [MAIN ] Update Progress: 484/2367 symbols completed. CMRE took 39.7s. Estimated remaining: 4h 18m 10s. Rate: 7.3 symbols/min [2025-11-04T18:06:24.970633] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-09. [2025-11-04T18:06:24.972179] [INFO ] [MAIN ] CMU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:06:24.980828] [DEBUG] [LOCAL ] CMU: Loaded 5508 existing records from primary storage. [2025-11-04T18:06:24.981046] [INFO ] [MAIN ] CMU: Loaded 5508 existing records. [2025-11-04T18:06:25.157515] [WARN ] [MAIN ] CMU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:06:25.157806] [INFO ] [MAIN ] CMU: Wiping existing primary data before full refresh. [2025-11-04T18:06:25.157858] [INFO ] [LOCAL ] CMU: Starting deletion of primary data. [2025-11-04T18:06:25.157990] [DEBUG] [LOCAL ] CMU: Deleted primary data file. [2025-11-04T18:06:25.158027] [INFO ] [MAIN ] CMU: Starting full primary refresh. [2025-11-04T18:06:25.561926] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-11-04T18:06:25.587623] [INFO ] [REMOTE ] CMU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:06:25.590897] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T18:06:25.767045] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-04. [2025-11-04T18:06:26.567020] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-10. [2025-11-04T18:06:26.873589] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-11-04T18:06:26.931990] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-11-04T18:06:27.367033] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-11. [2025-11-04T18:06:27.785121] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T18:06:27.785955] [INFO ] [REMOTE ] CMSC: Full fetch complete (6740 records). [2025-11-04T18:06:27.786039] [DEBUG] [VALIDATOR ] CMSC: Validating primary data. [2025-11-04T18:06:28.163070] [DEBUG] [VALIDATOR ] CMSC: All expected trading days are present. [2025-11-04T18:06:28.163146] [DEBUG] [VALIDATOR ] CMSC: Primary data validation passed. [2025-11-04T18:06:28.163322] [INFO ] [MAIN ] CMSC: Fetched primary data is valid. [2025-11-04T18:06:28.163358] [DEBUG] [LOCAL ] CMSC: Saving primary data. [2025-11-04T18:06:28.163758] [DEBUG] [LOCAL ] CMSC: Trimmed 6740 records to 4453 (keeping 200 days of data) [2025-11-04T18:06:28.175114] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-05. [2025-11-04T18:06:28.191539] [DEBUG] [LOCAL ] CMSC: Primary data saved (4453 records). [2025-11-04T18:06:28.191622] [INFO ] [MAIN ] CMSC: Full history saved. [2025-11-04T18:06:28.191728] [INFO ] [MAIN ] Update Progress: 485/2367 symbols completed. CMSC took 29.0s. Estimated remaining: 4h 17m 43s. Rate: 7.3 symbols/min [2025-11-04T18:06:28.357259] [INFO ] [MAIN ] CNA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:06:28.387212] [DEBUG] [LOCAL ] CNA: Loaded 32600 existing records from primary storage. [2025-11-04T18:06:28.387497] [INFO ] [MAIN ] CNA: Loaded 32600 existing records. [2025-11-04T18:06:28.560423] [WARN ] [MAIN ] CNA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:06:28.560679] [INFO ] [MAIN ] CNA: Wiping existing primary data before full refresh. [2025-11-04T18:06:28.560748] [INFO ] [LOCAL ] CNA: Starting deletion of primary data. [2025-11-04T18:06:28.560944] [DEBUG] [LOCAL ] CNA: Deleted primary data file. [2025-11-04T18:06:28.560999] [INFO ] [MAIN ] CNA: Starting full primary refresh. [2025-11-04T18:06:28.875609] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-11-04T18:06:28.949308] [INFO ] [REMOTE ] CNA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:06:28.967018] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-01. [2025-11-04T18:06:29.767052] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-05. [2025-11-04T18:06:29.827423] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T18:06:30.567034] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-02. [2025-11-04T18:06:30.710510] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T18:06:31.367021] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-06. [2025-11-04T18:06:31.736919] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T18:06:32.167038] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-01. [2025-11-04T18:06:32.443862] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T18:06:32.967064] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-07. [2025-11-04T18:06:33.043116] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T18:06:33.767081] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-02. [2025-11-04T18:06:33.996234] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T18:06:34.150031] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T18:06:34.567026] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-03. [2025-11-04T18:06:35.367020] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-03. [2025-11-04T18:06:35.485476] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-11-04T18:06:35.871467] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-11-04T18:06:36.167038] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-06. [2025-11-04T18:06:36.657335] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-11-04T18:06:36.967019] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-07. [2025-11-04T18:06:37.416234] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-11-04T18:06:37.767086] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-08. [2025-11-04T18:06:38.567014] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-04. [2025-11-04T18:06:39.077359] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-11-04T18:06:39.209372] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-11-04T18:06:39.367031] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-09. [2025-11-04T18:06:40.167065] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-04. [2025-11-04T18:06:40.260064] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-11-04T18:06:40.967041] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-08. [2025-11-04T18:06:41.211582] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-11-04T18:06:41.482379] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-11-04T18:06:41.767044] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-09. [2025-11-04T18:06:42.283767] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-11-04T18:06:42.567047] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-05. [2025-11-04T18:06:43.354558] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-11-04T18:06:43.366986] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-06. [2025-11-04T18:06:44.167038] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-10. [2025-11-04T18:06:44.392367] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T18:06:44.650011] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-11-04T18:06:44.967044] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-05. [2025-11-04T18:06:45.411083] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-11-04T18:06:45.767058] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-07. [2025-11-04T18:06:46.567096] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-06. [2025-11-04T18:06:46.899323] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T18:06:46.923520] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-06 (0.36 seconds). [2025-11-04T18:06:47.367070] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-10. [2025-11-04T18:06:48.155316] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T18:06:48.167005] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-11. [2025-11-04T18:06:48.967074] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-08. [2025-11-04T18:06:49.088211] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T18:06:49.090142] [INFO ] [REMOTE ] CMTG: Full fetch complete (52130 records). [2025-11-04T18:06:49.090447] [DEBUG] [VALIDATOR ] CMTG: Validating primary data. [2025-11-04T18:06:49.496231] [DEBUG] [VALIDATOR ] CMTG: All expected trading days are present. [2025-11-04T18:06:49.496319] [DEBUG] [VALIDATOR ] CMTG: Primary data validation passed. [2025-11-04T18:06:49.497013] [INFO ] [MAIN ] CMTG: Fetched primary data is valid. [2025-11-04T18:06:49.497090] [DEBUG] [LOCAL ] CMTG: Saving primary data. [2025-11-04T18:06:49.498199] [DEBUG] [LOCAL ] CMTG: Trimmed 52130 records to 33639 (keeping 200 days of data) [2025-11-04T18:06:49.688321] [DEBUG] [LOCAL ] CMTG: Primary data saved (33639 records). [2025-11-04T18:06:49.688472] [INFO ] [MAIN ] CMTG: Full history saved. [2025-11-04T18:06:49.689269] [INFO ] [MAIN ] Update Progress: 486/2367 symbols completed. CMTG took 35.7s. Estimated remaining: 4h 18m 26s. Rate: 7.3 symbols/min [2025-11-04T18:06:49.772456] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-07. [2025-11-04T18:06:49.861607] [INFO ] [MAIN ] CNC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:06:49.918361] [DEBUG] [LOCAL ] CNC: Loaded 53784 existing records from primary storage. [2025-11-04T18:06:49.918839] [INFO ] [MAIN ] CNC: Loaded 53784 existing records. [2025-11-04T18:06:50.102891] [WARN ] [MAIN ] CNC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:06:50.103445] [INFO ] [MAIN ] CNC: Wiping existing primary data before full refresh. [2025-11-04T18:06:50.103501] [INFO ] [LOCAL ] CNC: Starting deletion of primary data. [2025-11-04T18:06:50.103831] [DEBUG] [LOCAL ] CNC: Deleted primary data file. [2025-11-04T18:06:50.103889] [INFO ] [MAIN ] CNC: Starting full primary refresh. [2025-11-04T18:06:50.460874] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T18:06:50.484575] [INFO ] [REMOTE ] CNC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:06:50.536762] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-11-04T18:06:50.566995] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-08. [2025-11-04T18:06:50.985352] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-11-04T18:06:51.367031] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-09. [2025-11-04T18:06:51.835613] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-11-04T18:06:52.167252] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-10. [2025-11-04T18:06:52.533829] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-11-04T18:06:52.967032] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-09. [2025-11-04T18:06:53.767028] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-01. [2025-11-04T18:06:54.016318] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T18:06:54.567081] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-11. [2025-11-04T18:06:55.116107] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T18:06:55.117119] [INFO ] [REMOTE ] CMU: Full fetch complete (9452 records). [2025-11-04T18:06:55.117195] [DEBUG] [VALIDATOR ] CMU: Validating primary data. [2025-11-04T18:06:55.375677] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-10. [2025-11-04T18:06:55.458448] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-11-04T18:06:55.578739] [DEBUG] [VALIDATOR ] CMU: All expected trading days are present. [2025-11-04T18:06:55.578829] [DEBUG] [VALIDATOR ] CMU: Primary data validation passed. [2025-11-04T18:06:55.579132] [INFO ] [MAIN ] CMU: Fetched primary data is valid. [2025-11-04T18:06:55.579173] [DEBUG] [LOCAL ] CMU: Saving primary data. [2025-11-04T18:06:55.579599] [DEBUG] [LOCAL ] CMU: Trimmed 9452 records to 5328 (keeping 200 days of data) [2025-11-04T18:06:55.610885] [DEBUG] [LOCAL ] CMU: Primary data saved (5328 records). [2025-11-04T18:06:55.611015] [INFO ] [MAIN ] CMU: Full history saved. [2025-11-04T18:06:55.611104] [INFO ] [MAIN ] Update Progress: 487/2367 symbols completed. CMU took 30.8s. Estimated remaining: 4h 18m 9s. Rate: 7.3 symbols/min [2025-11-04T18:06:55.792346] [INFO ] [MAIN ] CNF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:06:55.804033] [DEBUG] [LOCAL ] CNF: Loaded 10562 existing records from primary storage. [2025-11-04T18:06:55.804245] [INFO ] [MAIN ] CNF: Loaded 10562 existing records. [2025-11-04T18:06:55.990923] [WARN ] [MAIN ] CNF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:06:55.991154] [INFO ] [MAIN ] CNF: Wiping existing primary data before full refresh. [2025-11-04T18:06:55.991202] [INFO ] [LOCAL ] CNF: Starting deletion of primary data. [2025-11-04T18:06:55.991373] [DEBUG] [LOCAL ] CNF: Deleted primary data file. [2025-11-04T18:06:55.991429] [INFO ] [MAIN ] CNF: Starting full primary refresh. [2025-11-04T18:06:56.172256] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-11. [2025-11-04T18:06:56.368839] [INFO ] [REMOTE ] CNF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:06:56.407308] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-11-04T18:06:56.837172] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T18:06:56.838182] [INFO ] [REMOTE ] CMSD: Full fetch complete (11191 records). [2025-11-04T18:06:56.838253] [DEBUG] [VALIDATOR ] CMSD: Validating primary data. [2025-11-04T18:06:56.982505] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-02. [2025-11-04T18:06:57.219864] [DEBUG] [VALIDATOR ] CMSD: All expected trading days are present. [2025-11-04T18:06:57.219938] [DEBUG] [VALIDATOR ] CMSD: Primary data validation passed. [2025-11-04T18:06:57.220225] [INFO ] [MAIN ] CMSD: Fetched primary data is valid. [2025-11-04T18:06:57.220286] [DEBUG] [LOCAL ] CMSD: Saving primary data. [2025-11-04T18:06:57.220775] [DEBUG] [LOCAL ] CMSD: Trimmed 11191 records to 7240 (keeping 200 days of data) [2025-11-04T18:06:57.265850] [DEBUG] [LOCAL ] CMSD: Primary data saved (7240 records). [2025-11-04T18:06:57.266014] [INFO ] [MAIN ] CMSD: Full history saved. [2025-11-04T18:06:57.266167] [INFO ] [MAIN ] Update Progress: 488/2367 symbols completed. CMSD took 48.1s. Estimated remaining: 4h 17m 35s. Rate: 7.3 symbols/min [2025-11-04T18:06:57.443295] [INFO ] [MAIN ] CNH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:06:57.495795] [DEBUG] [LOCAL ] CNH: Loaded 53535 existing records from primary storage. [2025-11-04T18:06:57.496105] [INFO ] [MAIN ] CNH: Loaded 53535 existing records. [2025-11-04T18:06:57.689278] [WARN ] [MAIN ] CNH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:06:57.689527] [INFO ] [MAIN ] CNH: Wiping existing primary data before full refresh. [2025-11-04T18:06:57.689584] [INFO ] [LOCAL ] CNH: Starting deletion of primary data. [2025-11-04T18:06:57.689851] [DEBUG] [LOCAL ] CNH: Deleted primary data file. [2025-11-04T18:06:57.689909] [INFO ] [MAIN ] CNH: Starting full primary refresh. [2025-11-04T18:06:57.772179] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-11. [2025-11-04T18:06:58.068101] [INFO ] [REMOTE ] CNH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:06:58.269927] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T18:06:58.567019] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-03. [2025-11-04T18:06:58.622523] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-11 (0.85 seconds). [2025-11-04T18:06:58.624417] [INFO ] [REMOTE ] CNA: Full fetch complete (46749 records). [2025-11-04T18:06:58.624494] [DEBUG] [VALIDATOR ] CNA: Validating primary data. [2025-11-04T18:06:59.008369] [DEBUG] [VALIDATOR ] CNA: All expected trading days are present. [2025-11-04T18:06:59.008479] [DEBUG] [VALIDATOR ] CNA: Primary data validation passed. [2025-11-04T18:06:59.009108] [INFO ] [MAIN ] CNA: Fetched primary data is valid. [2025-11-04T18:06:59.009171] [DEBUG] [LOCAL ] CNA: Saving primary data. [2025-11-04T18:06:59.010230] [DEBUG] [LOCAL ] CNA: Trimmed 46749 records to 32675 (keeping 200 days of data) [2025-11-04T18:06:59.205781] [DEBUG] [LOCAL ] CNA: Primary data saved (32675 records). [2025-11-04T18:06:59.205908] [INFO ] [MAIN ] CNA: Full history saved. [2025-11-04T18:06:59.205984] [INFO ] [MAIN ] Update Progress: 489/2367 symbols completed. CNA took 31.0s. Estimated remaining: 4h 17m 3s. Rate: 7.3 symbols/min [2025-11-04T18:06:59.367623] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-01. [2025-11-04T18:06:59.389337] [INFO ] [MAIN ] CNI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:06:59.440346] [DEBUG] [LOCAL ] CNI: Loaded 51551 existing records from primary storage. [2025-11-04T18:06:59.440668] [INFO ] [MAIN ] CNI: Loaded 51551 existing records. [2025-11-04T18:06:59.625348] [WARN ] [MAIN ] CNI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:06:59.626480] [INFO ] [MAIN ] CNI: Wiping existing primary data before full refresh. [2025-11-04T18:06:59.626539] [INFO ] [LOCAL ] CNI: Starting deletion of primary data. [2025-11-04T18:06:59.627213] [DEBUG] [LOCAL ] CNI: Deleted primary data file. [2025-11-04T18:06:59.627269] [INFO ] [MAIN ] CNI: Starting full primary refresh. [2025-11-04T18:06:59.996432] [INFO ] [REMOTE ] CNI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:07:00.029742] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T18:07:00.167039] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-04. [2025-11-04T18:07:00.923756] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-11-04T18:07:00.967032] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-01. [2025-11-04T18:07:01.553155] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T18:07:01.767038] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-02. [2025-11-04T18:07:02.567069] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-01. [2025-11-04T18:07:03.072297] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-11-04T18:07:03.213095] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-11-04T18:07:03.218984] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-11-04T18:07:03.367078] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-05. [2025-11-04T18:07:04.167016] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-03. [2025-11-04T18:07:04.922421] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-11-04T18:07:04.967008] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-06. [2025-11-04T18:07:05.475053] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T18:07:05.767022] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-04. [2025-11-04T18:07:06.372288] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-11-04T18:07:06.567060] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-07. [2025-11-04T18:07:07.051693] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T18:07:07.367038] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-02. [2025-11-04T18:07:07.748986] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T18:07:08.167031] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-02. [2025-11-04T18:07:08.613536] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-07 (2.05 seconds). [2025-11-04T18:07:08.967130] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-05. [2025-11-04T18:07:09.332985] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T18:07:09.767159] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-08. [2025-11-04T18:07:10.233213] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-11-04T18:07:10.567035] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-03. [2025-11-04T18:07:11.070716] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-11-04T18:07:11.367055] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-04. [2025-11-04T18:07:11.627017] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-11-04T18:07:11.750321] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T18:07:12.167101] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-06. [2025-11-04T18:07:12.967041] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-05. [2025-11-04T18:07:13.640268] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T18:07:13.767022] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-06. [2025-11-04T18:07:14.211731] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-06 (2.04 seconds). [2025-11-04T18:07:14.377915] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T18:07:14.567048] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-07. [2025-11-04T18:07:15.367098] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-09. [2025-11-04T18:07:15.977423] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T18:07:16.167051] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-08. [2025-11-04T18:07:16.967091] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-07. [2025-11-04T18:07:17.331361] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-11-04T18:07:17.342150] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-11-04T18:07:17.516053] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-11-04T18:07:17.767099] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-08. [2025-11-04T18:07:18.426245] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-11-04T18:07:18.567047] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-03. [2025-11-04T18:07:19.367030] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-09. [2025-11-04T18:07:20.102951] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-11-04T18:07:20.122462] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-11-04T18:07:20.167012] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-04. [2025-11-04T18:07:20.967047] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-10. [2025-11-04T18:07:21.359049] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-10 (0.39 seconds). [2025-11-04T18:07:21.517895] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T18:07:21.767047] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-11. [2025-11-04T18:07:22.154595] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-11 (0.39 seconds). [2025-11-04T18:07:22.155363] [INFO ] [REMOTE ] CNF: Full fetch complete (12952 records). [2025-11-04T18:07:22.155431] [DEBUG] [VALIDATOR ] CNF: Validating primary data. [2025-11-04T18:07:22.511516] [DEBUG] [VALIDATOR ] CNF: All expected trading days are present. [2025-11-04T18:07:22.511593] [DEBUG] [VALIDATOR ] CNF: Primary data validation passed. [2025-11-04T18:07:22.511985] [INFO ] [MAIN ] CNF: Fetched primary data is valid. [2025-11-04T18:07:22.512045] [DEBUG] [LOCAL ] CNF: Saving primary data. [2025-11-04T18:07:22.512513] [DEBUG] [LOCAL ] CNF: Trimmed 12952 records to 10566 (keeping 200 days of data) [2025-11-04T18:07:22.569265] [DEBUG] [LOCAL ] CNF: Primary data saved (10566 records). [2025-11-04T18:07:22.567050] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-05. [2025-11-04T18:07:22.569452] [INFO ] [MAIN ] CNF: Full history saved. [2025-11-04T18:07:22.570960] [INFO ] [MAIN ] Update Progress: 490/2367 symbols completed. CNF took 27.0s. Estimated remaining: 4h 17m 53s. Rate: 7.3 symbols/min [2025-11-04T18:07:22.749692] [INFO ] [MAIN ] CNK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:07:22.797038] [DEBUG] [LOCAL ] CNK: Loaded 52854 existing records from primary storage. [2025-11-04T18:07:22.797371] [INFO ] [MAIN ] CNK: Loaded 52854 existing records. [2025-11-04T18:07:22.998531] [WARN ] [MAIN ] CNK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:07:22.998837] [INFO ] [MAIN ] CNK: Wiping existing primary data before full refresh. [2025-11-04T18:07:22.998898] [INFO ] [LOCAL ] CNK: Starting deletion of primary data. [2025-11-04T18:07:22.999124] [DEBUG] [LOCAL ] CNK: Deleted primary data file. [2025-11-04T18:07:22.999192] [INFO ] [MAIN ] CNK: Starting full primary refresh. [2025-11-04T18:07:23.356423] [INFO ] [REMOTE ] CNK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:07:23.367016] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-01. [2025-11-04T18:07:23.941436] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T18:07:24.167079] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-10. [2025-11-04T18:07:24.620795] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T18:07:24.967085] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-09. [2025-11-04T18:07:25.767005] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-06. [2025-11-04T18:07:26.030201] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-11-04T18:07:26.391481] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T18:07:26.567007] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-10. [2025-11-04T18:07:27.007284] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T18:07:27.367075] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-11. [2025-11-04T18:07:27.685838] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-11 (0.32 seconds). [2025-11-04T18:07:27.688024] [INFO ] [REMOTE ] CNC: Full fetch complete (77872 records). [2025-11-04T18:07:27.688137] [DEBUG] [VALIDATOR ] CNC: Validating primary data. [2025-11-04T18:07:28.107368] [DEBUG] [VALIDATOR ] CNC: All expected trading days are present. [2025-11-04T18:07:28.107474] [DEBUG] [VALIDATOR ] CNC: Primary data validation passed. [2025-11-04T18:07:28.108356] [INFO ] [MAIN ] CNC: Fetched primary data is valid. [2025-11-04T18:07:28.108437] [DEBUG] [LOCAL ] CNC: Saving primary data. [2025-11-04T18:07:28.121186] [DEBUG] [LOCAL ] CNC: Trimmed 77872 records to 53785 (keeping 200 days of data) [2025-11-04T18:07:28.144309] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-11-04T18:07:28.189600] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-07. [2025-11-04T18:07:28.473253] [DEBUG] [LOCAL ] CNC: Primary data saved (53785 records). [2025-11-04T18:07:28.473414] [INFO ] [MAIN ] CNC: Full history saved. [2025-11-04T18:07:28.473507] [INFO ] [MAIN ] Update Progress: 491/2367 symbols completed. CNC took 38.8s. Estimated remaining: 4h 17m 36s. Rate: 7.3 symbols/min [2025-11-04T18:07:28.640195] [INFO ] [MAIN ] CNM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:07:28.685342] [DEBUG] [LOCAL ] CNM: Loaded 50862 existing records from primary storage. [2025-11-04T18:07:28.685629] [INFO ] [MAIN ] CNM: Loaded 50862 existing records. [2025-11-04T18:07:28.859213] [WARN ] [MAIN ] CNM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:07:28.859420] [INFO ] [MAIN ] CNM: Wiping existing primary data before full refresh. [2025-11-04T18:07:28.859466] [INFO ] [LOCAL ] CNM: Starting deletion of primary data. [2025-11-04T18:07:28.859696] [DEBUG] [LOCAL ] CNM: Deleted primary data file. [2025-11-04T18:07:28.859771] [INFO ] [MAIN ] CNM: Starting full primary refresh. [2025-11-04T18:07:28.972269] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-11. [2025-11-04T18:07:29.217618] [INFO ] [REMOTE ] CNM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:07:29.767001] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-01. [2025-11-04T18:07:30.124469] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-11-04T18:07:30.195305] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T18:07:30.198220] [INFO ] [REMOTE ] CNH: Full fetch complete (77707 records). [2025-11-04T18:07:30.198365] [DEBUG] [VALIDATOR ] CNH: Validating primary data. [2025-11-04T18:07:30.572295] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-08. [2025-11-04T18:07:30.649626] [DEBUG] [VALIDATOR ] CNH: All expected trading days are present. [2025-11-04T18:07:30.649733] [DEBUG] [VALIDATOR ] CNH: Primary data validation passed. [2025-11-04T18:07:30.650569] [INFO ] [MAIN ] CNH: Fetched primary data is valid. [2025-11-04T18:07:30.650630] [DEBUG] [LOCAL ] CNH: Saving primary data. [2025-11-04T18:07:30.652015] [DEBUG] [LOCAL ] CNH: Trimmed 77707 records to 53537 (keeping 200 days of data) [2025-11-04T18:07:30.965091] [DEBUG] [LOCAL ] CNH: Primary data saved (53537 records). [2025-11-04T18:07:30.965227] [INFO ] [MAIN ] CNH: Full history saved. [2025-11-04T18:07:30.965305] [INFO ] [MAIN ] Update Progress: 492/2367 symbols completed. CNH took 33.7s. Estimated remaining: 4h 17m 6s. Rate: 7.3 symbols/min [2025-11-04T18:07:31.134781] [INFO ] [MAIN ] CNMD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:07:31.163890] [DEBUG] [LOCAL ] CNMD: Loaded 31493 existing records from primary storage. [2025-11-04T18:07:31.164149] [INFO ] [MAIN ] CNMD: Loaded 31493 existing records. [2025-11-04T18:07:31.349408] [WARN ] [MAIN ] CNMD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:07:31.349858] [INFO ] [MAIN ] CNMD: Wiping existing primary data before full refresh. [2025-11-04T18:07:31.349935] [INFO ] [LOCAL ] CNMD: Starting deletion of primary data. [2025-11-04T18:07:31.350101] [DEBUG] [LOCAL ] CNMD: Deleted primary data file. [2025-11-04T18:07:31.350174] [INFO ] [MAIN ] CNMD: Starting full primary refresh. [2025-11-04T18:07:31.372205] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-02. [2025-11-04T18:07:31.740249] [INFO ] [REMOTE ] CNMD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:07:31.859948] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-01 (2.09 seconds). [2025-11-04T18:07:32.167050] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-01. [2025-11-04T18:07:32.208044] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-11-04T18:07:32.967031] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-02. [2025-11-04T18:07:33.265913] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-11-04T18:07:33.297913] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T18:07:33.767104] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-03. [2025-11-04T18:07:34.099092] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T18:07:34.567060] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-02. [2025-11-04T18:07:35.314020] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T18:07:35.367012] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-04. [2025-11-04T18:07:35.557015] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T18:07:36.167081] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-03. [2025-11-04T18:07:36.683110] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T18:07:36.967223] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-03. [2025-11-04T18:07:37.507801] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T18:07:37.767054] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-05. [2025-11-04T18:07:38.071970] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T18:07:38.567039] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-04. [2025-11-04T18:07:38.802307] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-11-04T18:07:39.367045] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-04. [2025-11-04T18:07:39.803175] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T18:07:40.167067] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-06. [2025-11-04T18:07:40.556422] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T18:07:40.967077] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-09. [2025-11-04T18:07:41.556474] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T18:07:41.767068] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-07. [2025-11-04T18:07:42.567024] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-05. [2025-11-04T18:07:42.650538] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-11-04T18:07:43.367034] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-05. [2025-11-04T18:07:43.916878] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-07 (2.15 seconds). [2025-11-04T18:07:44.167019] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-10. [2025-11-04T18:07:44.273019] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-11-04T18:07:44.785241] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-11-04T18:07:44.967053] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-08. [2025-11-04T18:07:45.528022] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-11-04T18:07:45.767029] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-11. [2025-11-04T18:07:46.258558] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T18:07:46.567021] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-09. [2025-11-04T18:07:46.821217] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T18:07:46.823255] [INFO ] [REMOTE ] CNI: Full fetch complete (74834 records). [2025-11-04T18:07:46.823396] [DEBUG] [VALIDATOR ] CNI: Validating primary data. [2025-11-04T18:07:47.221772] [DEBUG] [VALIDATOR ] CNI: All expected trading days are present. [2025-11-04T18:07:47.221880] [DEBUG] [VALIDATOR ] CNI: Primary data validation passed. [2025-11-04T18:07:47.222947] [INFO ] [MAIN ] CNI: Fetched primary data is valid. [2025-11-04T18:07:47.223009] [DEBUG] [LOCAL ] CNI: Saving primary data. [2025-11-04T18:07:47.224579] [DEBUG] [LOCAL ] CNI: Trimmed 74834 records to 51555 (keeping 200 days of data) [2025-11-04T18:07:47.406205] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-06. [2025-11-04T18:07:47.544853] [DEBUG] [LOCAL ] CNI: Primary data saved (51555 records). [2025-11-04T18:07:47.544976] [INFO ] [MAIN ] CNI: Full history saved. [2025-11-04T18:07:47.545051] [INFO ] [MAIN ] Update Progress: 493/2367 symbols completed. CNI took 48.3s. Estimated remaining: 4h 17m 29s. Rate: 7.3 symbols/min [2025-11-04T18:07:47.729195] [INFO ] [MAIN ] CNNE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:07:47.767661] [DEBUG] [LOCAL ] CNNE: Loaded 35087 existing records from primary storage. [2025-11-04T18:07:47.796267] [INFO ] [MAIN ] CNNE: Loaded 35087 existing records. [2025-11-04T18:07:47.881283] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T18:07:47.997121] [WARN ] [MAIN ] CNNE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:07:47.997347] [INFO ] [MAIN ] CNNE: Wiping existing primary data before full refresh. [2025-11-04T18:07:47.997396] [INFO ] [LOCAL ] CNNE: Starting deletion of primary data. [2025-11-04T18:07:47.997571] [DEBUG] [LOCAL ] CNNE: Deleted primary data file. [2025-11-04T18:07:47.997609] [INFO ] [MAIN ] CNNE: Starting full primary refresh. [2025-11-04T18:07:48.176800] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-10. [2025-11-04T18:07:48.406806] [INFO ] [REMOTE ] CNNE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:07:48.424395] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T18:07:48.967045] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-01. [2025-11-04T18:07:49.718762] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-11-04T18:07:49.745242] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-11-04T18:07:49.766994] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-02. [2025-11-04T18:07:50.567049] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-07. [2025-11-04T18:07:50.625350] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-11-04T18:07:51.367110] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-03. [2025-11-04T18:07:51.635283] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T18:07:52.167039] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-11. [2025-11-04T18:07:52.222369] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-11-04T18:07:52.967050] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-08. [2025-11-04T18:07:53.342417] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T18:07:53.344414] [INFO ] [REMOTE ] CNK: Full fetch complete (76460 records). [2025-11-04T18:07:53.344527] [DEBUG] [VALIDATOR ] CNK: Validating primary data. [2025-11-04T18:07:53.733733] [DEBUG] [VALIDATOR ] CNK: All expected trading days are present. [2025-11-04T18:07:53.733864] [DEBUG] [VALIDATOR ] CNK: Primary data validation passed. [2025-11-04T18:07:53.734709] [INFO ] [MAIN ] CNK: Fetched primary data is valid. [2025-11-04T18:07:53.734782] [DEBUG] [LOCAL ] CNK: Saving primary data. [2025-11-04T18:07:53.736267] [DEBUG] [LOCAL ] CNK: Trimmed 76460 records to 52866 (keeping 200 days of data) [2025-11-04T18:07:53.788766] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-06. [2025-11-04T18:07:54.062793] [DEBUG] [LOCAL ] CNK: Primary data saved (52866 records). [2025-11-04T18:07:54.062938] [INFO ] [MAIN ] CNK: Full history saved. [2025-11-04T18:07:54.063018] [INFO ] [MAIN ] Update Progress: 494/2367 symbols completed. CNK took 31.5s. Estimated remaining: 4h 17m 14s. Rate: 7.3 symbols/min [2025-11-04T18:07:54.271292] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T18:07:54.304265] [INFO ] [MAIN ] CNO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:07:54.346039] [DEBUG] [LOCAL ] CNO: Loaded 39810 existing records from primary storage. [2025-11-04T18:07:54.346341] [INFO ] [MAIN ] CNO: Loaded 39810 existing records. [2025-11-04T18:07:54.539961] [WARN ] [MAIN ] CNO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:07:54.540183] [INFO ] [MAIN ] CNO: Wiping existing primary data before full refresh. [2025-11-04T18:07:54.540228] [INFO ] [LOCAL ] CNO: Starting deletion of primary data. [2025-11-04T18:07:54.540413] [DEBUG] [LOCAL ] CNO: Deleted primary data file. [2025-11-04T18:07:54.540450] [INFO ] [MAIN ] CNO: Starting full primary refresh. [2025-11-04T18:07:54.572187] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-09. [2025-11-04T18:07:54.885685] [INFO ] [REMOTE ] CNO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:07:55.367026] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-01. [2025-11-04T18:07:55.521633] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-11-04T18:07:55.601167] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-11-04T18:07:56.167109] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-10. [2025-11-04T18:07:56.798677] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T18:07:56.967071] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-04. [2025-11-04T18:07:56.999026] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-11-04T18:07:57.767063] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-07. [2025-11-04T18:07:58.117904] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T18:07:58.567035] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-11. [2025-11-04T18:07:59.012769] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:07:59.367065] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-08. [2025-11-04T18:07:59.717487] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T18:07:59.719179] [INFO ] [REMOTE ] CNMD: Full fetch complete (47478 records). [2025-11-04T18:07:59.719323] [DEBUG] [VALIDATOR ] CNMD: Validating primary data. [2025-11-04T18:08:00.152815] [DEBUG] [VALIDATOR ] CNMD: All expected trading days are present. [2025-11-04T18:08:00.152901] [DEBUG] [VALIDATOR ] CNMD: Primary data validation passed. [2025-11-04T18:08:00.153476] [INFO ] [MAIN ] CNMD: Fetched primary data is valid. [2025-11-04T18:08:00.153515] [DEBUG] [LOCAL ] CNMD: Saving primary data. [2025-11-04T18:08:00.154818] [DEBUG] [LOCAL ] CNMD: Trimmed 47478 records to 31482 (keeping 200 days of data) [2025-11-04T18:08:00.207801] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-05. [2025-11-04T18:08:00.347159] [DEBUG] [LOCAL ] CNMD: Primary data saved (31482 records). [2025-11-04T18:08:00.347315] [INFO ] [MAIN ] CNMD: Full history saved. [2025-11-04T18:08:00.347398] [INFO ] [MAIN ] Update Progress: 495/2367 symbols completed. CNMD took 29.4s. Estimated remaining: 4h 16m 59s. Rate: 7.3 symbols/min [2025-11-04T18:08:00.519775] [INFO ] [MAIN ] CNP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:08:00.519969] [DEBUG] [LOCAL ] CNP: No existing data file found in primary storage. [2025-11-04T18:08:00.520024] [WARN ] [MAIN ] CNP: No existing data found, performing full refresh instead. [2025-11-04T18:08:00.520129] [INFO ] [MAIN ] CNP: Wiping existing primary data before full refresh. [2025-11-04T18:08:00.520170] [INFO ] [LOCAL ] CNP: Starting deletion of primary data. [2025-11-04T18:08:00.520210] [DEBUG] [LOCAL ] CNP: No primary data file found to delete. [2025-11-04T18:08:00.520241] [INFO ] [MAIN ] CNP: Starting full primary refresh. [2025-11-04T18:08:00.886385] [INFO ] [REMOTE ] CNP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:08:00.921296] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-11-04T18:08:00.967056] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-01. [2025-11-04T18:08:01.191787] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-11-04T18:08:01.767089] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-02. [2025-11-04T18:08:02.338938] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-11-04T18:08:02.567071] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-06. [2025-11-04T18:08:03.256397] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-11-04T18:08:03.367065] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-09. [2025-11-04T18:08:03.553321] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-11-04T18:08:04.167064] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-07. [2025-11-04T18:08:04.764070] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T18:08:04.967031] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-03. [2025-11-04T18:08:05.193461] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T18:08:05.767049] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-10. [2025-11-04T18:08:06.193145] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T18:08:06.567091] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-02. [2025-11-04T18:08:07.166207] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T18:08:07.367046] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-11. [2025-11-04T18:08:07.589424] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T18:08:08.167095] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-04. [2025-11-04T18:08:08.523271] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T18:08:08.525314] [INFO ] [REMOTE ] CNM: Full fetch complete (73686 records). [2025-11-04T18:08:08.525422] [DEBUG] [VALIDATOR ] CNM: Validating primary data. [2025-11-04T18:08:08.921779] [DEBUG] [VALIDATOR ] CNM: All expected trading days are present. [2025-11-04T18:08:08.921881] [DEBUG] [VALIDATOR ] CNM: Primary data validation passed. [2025-11-04T18:08:08.922703] [INFO ] [MAIN ] CNM: Fetched primary data is valid. [2025-11-04T18:08:08.922988] [DEBUG] [LOCAL ] CNM: Saving primary data. [2025-11-04T18:08:08.924600] [DEBUG] [LOCAL ] CNM: Trimmed 73686 records to 50867 (keeping 200 days of data) [2025-11-04T18:08:08.972274] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-08. [2025-11-04T18:08:09.227671] [DEBUG] [LOCAL ] CNM: Primary data saved (50867 records). [2025-11-04T18:08:09.227833] [INFO ] [MAIN ] CNM: Full history saved. [2025-11-04T18:08:09.228493] [INFO ] [MAIN ] Update Progress: 496/2367 symbols completed. CNM took 40.8s. Estimated remaining: 4h 16m 53s. Rate: 7.3 symbols/min [2025-11-04T18:08:09.414716] [INFO ] [MAIN ] CNQ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:08:09.441554] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T18:08:09.494589] [DEBUG] [LOCAL ] CNQ: Loaded 53570 existing records from primary storage. [2025-11-04T18:08:09.494953] [INFO ] [MAIN ] CNQ: Loaded 53570 existing records. [2025-11-04T18:08:09.676473] [WARN ] [MAIN ] CNQ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:08:09.676755] [INFO ] [MAIN ] CNQ: Wiping existing primary data before full refresh. [2025-11-04T18:08:09.676842] [INFO ] [LOCAL ] CNQ: Starting deletion of primary data. [2025-11-04T18:08:09.677056] [DEBUG] [LOCAL ] CNQ: Deleted primary data file. [2025-11-04T18:08:09.677112] [INFO ] [MAIN ] CNQ: Starting full primary refresh. [2025-11-04T18:08:09.774569] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-05. [2025-11-04T18:08:10.041428] [INFO ] [REMOTE ] CNQ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:08:10.071108] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-11-04T18:08:10.567079] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-09. [2025-11-04T18:08:11.066443] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T18:08:11.367079] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-03. [2025-11-04T18:08:11.789706] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T18:08:12.167092] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-10. [2025-11-04T18:08:12.811140] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T18:08:12.967031] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-04. [2025-11-04T18:08:13.523139] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-11-04T18:08:13.767034] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-06. [2025-11-04T18:08:14.290554] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T18:08:14.567089] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-05. [2025-11-04T18:08:14.894583] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T18:08:15.367012] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-11. [2025-11-04T18:08:15.864516] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T18:08:16.167073] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-07. [2025-11-04T18:08:16.266626] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T18:08:16.268143] [INFO ] [REMOTE ] CNNE: Full fetch complete (50911 records). [2025-11-04T18:08:16.268242] [DEBUG] [VALIDATOR ] CNNE: Validating primary data. [2025-11-04T18:08:16.634876] [DEBUG] [VALIDATOR ] CNNE: All expected trading days are present. [2025-11-04T18:08:16.634951] [DEBUG] [VALIDATOR ] CNNE: Primary data validation passed. [2025-11-04T18:08:16.635547] [INFO ] [MAIN ] CNNE: Fetched primary data is valid. [2025-11-04T18:08:16.635585] [DEBUG] [LOCAL ] CNNE: Saving primary data. [2025-11-04T18:08:16.636559] [DEBUG] [LOCAL ] CNNE: Trimmed 50911 records to 35032 (keeping 200 days of data) [2025-11-04T18:08:16.840769] [DEBUG] [LOCAL ] CNNE: Primary data saved (35032 records). [2025-11-04T18:08:16.840898] [INFO ] [MAIN ] CNNE: Full history saved. [2025-11-04T18:08:16.840974] [INFO ] [MAIN ] Update Progress: 497/2367 symbols completed. CNNE took 29.3s. Estimated remaining: 4h 16m 42s. Rate: 7.3 symbols/min [2025-11-04T18:08:16.978244] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-06. [2025-11-04T18:08:17.010998] [INFO ] [MAIN ] CNR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:08:17.051828] [DEBUG] [LOCAL ] CNR: Loaded 43624 existing records from primary storage. [2025-11-04T18:08:17.052129] [INFO ] [MAIN ] CNR: Loaded 43624 existing records. [2025-11-04T18:08:17.224588] [WARN ] [MAIN ] CNR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:08:17.224864] [INFO ] [MAIN ] CNR: Wiping existing primary data before full refresh. [2025-11-04T18:08:17.224913] [INFO ] [LOCAL ] CNR: Starting deletion of primary data. [2025-11-04T18:08:17.225112] [DEBUG] [LOCAL ] CNR: Deleted primary data file. [2025-11-04T18:08:17.225167] [INFO ] [MAIN ] CNR: Starting full primary refresh. [2025-11-04T18:08:17.609517] [INFO ] [REMOTE ] CNR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:08:17.640788] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-11-04T18:08:17.767067] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-08. [2025-11-04T18:08:18.321479] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T18:08:18.567054] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-01. [2025-11-04T18:08:18.894286] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T18:08:19.367068] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-09. [2025-11-04T18:08:19.482908] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-11-04T18:08:20.167281] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-01. [2025-11-04T18:08:20.504085] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T18:08:20.967103] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-02. [2025-11-04T18:08:21.747885] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-11-04T18:08:21.766993] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-02. [2025-11-04T18:08:22.233013] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T18:08:22.567126] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-10. [2025-11-04T18:08:23.008339] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T18:08:23.367043] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-03. [2025-11-04T18:08:23.516210] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T18:08:24.167088] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-03. [2025-11-04T18:08:24.739404] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T18:08:24.967041] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-04. [2025-11-04T18:08:25.182937] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-11-04T18:08:25.767021] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-07. [2025-11-04T18:08:26.397625] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T18:08:26.567019] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-05. [2025-11-04T18:08:27.215579] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T18:08:27.367039] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-08. [2025-11-04T18:08:28.134221] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-11-04T18:08:28.167058] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-06. [2025-11-04T18:08:28.967017] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-04. [2025-11-04T18:08:29.000739] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-11-04T18:08:29.619704] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T18:08:29.767126] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-11. [2025-11-04T18:08:30.403578] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T18:08:30.567030] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-05. [2025-11-04T18:08:30.911442] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T18:08:30.913144] [INFO ] [REMOTE ] CNO: Full fetch complete (60504 records). [2025-11-04T18:08:30.913220] [DEBUG] [VALIDATOR ] CNO: Validating primary data. [2025-11-04T18:08:31.323819] [DEBUG] [VALIDATOR ] CNO: All expected trading days are present. [2025-11-04T18:08:31.323907] [DEBUG] [VALIDATOR ] CNO: Primary data validation passed. [2025-11-04T18:08:31.324603] [INFO ] [MAIN ] CNO: Fetched primary data is valid. [2025-11-04T18:08:31.324666] [DEBUG] [LOCAL ] CNO: Saving primary data. [2025-11-04T18:08:31.325939] [DEBUG] [LOCAL ] CNO: Trimmed 60504 records to 39836 (keeping 200 days of data) [2025-11-04T18:08:31.378677] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-07. [2025-11-04T18:08:31.567545] [DEBUG] [LOCAL ] CNO: Primary data saved (39836 records). [2025-11-04T18:08:31.567703] [INFO ] [MAIN ] CNO: Full history saved. [2025-11-04T18:08:31.567794] [INFO ] [MAIN ] Update Progress: 498/2367 symbols completed. CNO took 37.5s. Estimated remaining: 4h 16m 58s. Rate: 7.3 symbols/min [2025-11-04T18:08:31.812371] [INFO ] [MAIN ] CNS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:08:31.812590] [DEBUG] [LOCAL ] CNS: No existing data file found in primary storage. [2025-11-04T18:08:31.812667] [WARN ] [MAIN ] CNS: No existing data found, performing full refresh instead. [2025-11-04T18:08:31.812786] [INFO ] [MAIN ] CNS: Wiping existing primary data before full refresh. [2025-11-04T18:08:31.812828] [INFO ] [LOCAL ] CNS: Starting deletion of primary data. [2025-11-04T18:08:31.812868] [DEBUG] [LOCAL ] CNS: No primary data file found to delete. [2025-11-04T18:08:31.812899] [INFO ] [MAIN ] CNS: Starting full primary refresh. [2025-11-04T18:08:32.019478] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T18:08:32.175483] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-09. [2025-11-04T18:08:32.215876] [INFO ] [REMOTE ] CNS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:08:32.826052] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T18:08:32.967034] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-01. [2025-11-04T18:08:33.550531] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T18:08:33.767084] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-08. [2025-11-04T18:08:33.820968] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-11-04T18:08:34.567046] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-02. [2025-11-04T18:08:35.096854] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-11-04T18:08:35.277988] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T18:08:35.367043] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-06. [2025-11-04T18:08:36.167040] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-10. [2025-11-04T18:08:36.429247] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-11-04T18:08:36.967088] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-09. [2025-11-04T18:08:37.560193] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T18:08:37.767037] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-07. [2025-11-04T18:08:38.137504] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T18:08:38.567043] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-03. [2025-11-04T18:08:38.959183] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-11-04T18:08:39.367081] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-08. [2025-11-04T18:08:39.444339] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-11-04T18:08:40.167044] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-04. [2025-11-04T18:08:40.481295] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T18:08:40.944160] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-11-04T18:08:40.966996] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-05. [2025-11-04T18:08:41.661266] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-11-04T18:08:41.767077] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-09. [2025-11-04T18:08:42.567082] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-06. [2025-11-04T18:08:42.791893] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T18:08:43.262890] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T18:08:43.367038] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-10. [2025-11-04T18:08:44.167089] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-07. [2025-11-04T18:08:44.669623] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-11-04T18:08:44.967030] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-11. [2025-11-04T18:08:45.080504] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-11-04T18:08:45.234543] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-11 (0.27 seconds). [2025-11-04T18:08:45.236070] [INFO ] [REMOTE ] CNP: Full fetch complete (77174 records). [2025-11-04T18:08:45.236169] [DEBUG] [VALIDATOR ] CNP: Validating primary data. [2025-11-04T18:08:45.237033] [FATAL] [VALIDATOR ] CNP: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:08:45.237129] [FATAL] [MAIN ] CNP: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:08:45.237214] [WARN ] [MAIN ] CNP: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:08:45.237304] [INFO ] [LOCAL ] CNP: Starting deletion of primary data. [2025-11-04T18:08:45.237512] [DEBUG] [LOCAL ] CNP: No primary data file found to delete. [2025-11-04T18:08:45.237566] [INFO ] [LOCAL ] CNP: Starting deletion of derivative data. [2025-11-04T18:08:45.237610] [DEBUG] [LOCAL ] CNP: No derivative data file found to delete. [2025-11-04T18:08:45.237693] [INFO ] [MAIN ] Update Progress: 499/2367 symbols completed. CNP took 44.9s. Estimated remaining: 4h 17m 10s. Rate: 7.3 symbols/min [2025-11-04T18:08:45.406335] [INFO ] [MAIN ] CNX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:08:45.453537] [DEBUG] [LOCAL ] CNX: Loaded 51287 existing records from primary storage. [2025-11-04T18:08:45.453858] [INFO ] [MAIN ] CNX: Loaded 51287 existing records. [2025-11-04T18:08:45.641268] [WARN ] [MAIN ] CNX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:08:45.641483] [INFO ] [MAIN ] CNX: Wiping existing primary data before full refresh. [2025-11-04T18:08:45.641537] [INFO ] [LOCAL ] CNX: Starting deletion of primary data. [2025-11-04T18:08:45.641753] [DEBUG] [LOCAL ] CNX: Deleted primary data file. [2025-11-04T18:08:45.641816] [INFO ] [MAIN ] CNX: Starting full primary refresh. [2025-11-04T18:08:45.779151] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-11. [2025-11-04T18:08:45.993893] [INFO ] [REMOTE ] CNX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:08:46.567102] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-08. [2025-11-04T18:08:46.913700] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T18:08:46.915551] [INFO ] [REMOTE ] CNR: Full fetch complete (64578 records). [2025-11-04T18:08:46.915843] [DEBUG] [VALIDATOR ] CNR: Validating primary data. [2025-11-04T18:08:47.361593] [DEBUG] [VALIDATOR ] CNR: All expected trading days are present. [2025-11-04T18:08:47.361730] [DEBUG] [VALIDATOR ] CNR: Primary data validation passed. [2025-11-04T18:08:47.362574] [INFO ] [MAIN ] CNR: Fetched primary data is valid. [2025-11-04T18:08:47.362678] [DEBUG] [LOCAL ] CNR: Saving primary data. [2025-11-04T18:08:47.369100] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-01. [2025-11-04T18:08:47.378313] [DEBUG] [LOCAL ] CNR: Trimmed 64578 records to 43662 (keeping 200 days of data) [2025-11-04T18:08:47.378013] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-11-04T18:08:47.663125] [DEBUG] [LOCAL ] CNR: Primary data saved (43662 records). [2025-11-04T18:08:47.663311] [INFO ] [MAIN ] CNR: Full history saved. [2025-11-04T18:08:47.663421] [INFO ] [MAIN ] Update Progress: 500/2367 symbols completed. CNR took 30.8s. Estimated remaining: 4h 16m 40s. Rate: 7.3 symbols/min [2025-11-04T18:08:47.905464] [INFO ] [MAIN ] CODI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:08:47.964922] [DEBUG] [LOCAL ] CODI: Loaded 42545 existing records from primary storage. [2025-11-04T18:08:47.965394] [INFO ] [MAIN ] CODI: Loaded 42545 existing records. [2025-11-04T18:08:48.172744] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-09. [2025-11-04T18:08:48.264378] [WARN ] [MAIN ] CODI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:08:48.264712] [INFO ] [MAIN ] CODI: Wiping existing primary data before full refresh. [2025-11-04T18:08:48.264825] [INFO ] [LOCAL ] CODI: Starting deletion of primary data. [2025-11-04T18:08:48.265076] [DEBUG] [LOCAL ] CODI: Deleted primary data file. [2025-11-04T18:08:48.265149] [INFO ] [MAIN ] CODI: Starting full primary refresh. [2025-11-04T18:08:48.762817] [INFO ] [REMOTE ] CODI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:08:48.846220] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-11-04T18:08:48.959274] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T18:08:48.967008] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-10. [2025-11-04T18:08:49.767098] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-10. [2025-11-04T18:08:50.567782] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-02. [2025-11-04T18:08:50.577547] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-11-04T18:08:50.640879] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-11-04T18:08:51.367143] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-11. [2025-11-04T18:08:51.756715] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T18:08:52.167062] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-01. [2025-11-04T18:08:52.245580] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-11 (0.88 seconds). [2025-11-04T18:08:52.247748] [INFO ] [REMOTE ] CNS: Full fetch complete (31800 records). [2025-11-04T18:08:52.247878] [DEBUG] [VALIDATOR ] CNS: Validating primary data. [2025-11-04T18:08:52.648594] [DEBUG] [VALIDATOR ] CNS: All expected trading days are present. [2025-11-04T18:08:52.648717] [DEBUG] [VALIDATOR ] CNS: Primary data validation passed. [2025-11-04T18:08:52.649204] [INFO ] [MAIN ] CNS: Fetched primary data is valid. [2025-11-04T18:08:52.649245] [DEBUG] [LOCAL ] CNS: Saving primary data. [2025-11-04T18:08:52.649972] [DEBUG] [LOCAL ] CNS: Trimmed 31800 records to 21258 (keeping 200 days of data) [2025-11-04T18:08:52.776525] [DEBUG] [LOCAL ] CNS: Primary data saved (21258 records). [2025-11-04T18:08:52.776670] [INFO ] [MAIN ] CNS: Full history saved. [2025-11-04T18:08:52.776756] [INFO ] [MAIN ] Update Progress: 501/2367 symbols completed. CNS took 21.2s. Estimated remaining: 4h 16m 20s. Rate: 7.3 symbols/min [2025-11-04T18:08:52.947316] [INFO ] [MAIN ] COF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:08:52.975159] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-11. [2025-11-04T18:08:53.004204] [DEBUG] [LOCAL ] COF: Loaded 53629 existing records from primary storage. [2025-11-04T18:08:53.004577] [INFO ] [MAIN ] COF: Loaded 53629 existing records. [2025-11-04T18:08:53.173826] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T18:08:53.205458] [WARN ] [MAIN ] COF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:08:53.205706] [INFO ] [MAIN ] COF: Wiping existing primary data before full refresh. [2025-11-04T18:08:53.205762] [INFO ] [LOCAL ] COF: Starting deletion of primary data. [2025-11-04T18:08:53.206007] [DEBUG] [LOCAL ] COF: Deleted primary data file. [2025-11-04T18:08:53.206063] [INFO ] [MAIN ] COF: Starting full primary refresh. [2025-11-04T18:08:53.555516] [INFO ] [REMOTE ] COF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:08:53.767055] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-02. [2025-11-04T18:08:54.199792] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T18:08:54.202615] [INFO ] [REMOTE ] CNQ: Full fetch complete (77731 records). [2025-11-04T18:08:54.202789] [DEBUG] [VALIDATOR ] CNQ: Validating primary data. [2025-11-04T18:08:54.574291] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-01. [2025-11-04T18:08:54.629389] [DEBUG] [VALIDATOR ] CNQ: All expected trading days are present. [2025-11-04T18:08:54.629606] [DEBUG] [VALIDATOR ] CNQ: Primary data validation passed. [2025-11-04T18:08:54.630998] [INFO ] [MAIN ] CNQ: Fetched primary data is valid. [2025-11-04T18:08:54.631995] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-11-04T18:08:54.631421] [DEBUG] [LOCAL ] CNQ: Saving primary data. [2025-11-04T18:08:54.638591] [DEBUG] [LOCAL ] CNQ: Trimmed 77731 records to 53570 (keeping 200 days of data) [2025-11-04T18:08:54.988360] [DEBUG] [LOCAL ] CNQ: Primary data saved (53570 records). [2025-11-04T18:08:54.988487] [INFO ] [MAIN ] CNQ: Full history saved. [2025-11-04T18:08:54.989106] [INFO ] [MAIN ] Update Progress: 502/2367 symbols completed. CNQ took 45.8s. Estimated remaining: 4h 15m 50s. Rate: 7.3 symbols/min [2025-11-04T18:08:55.161161] [INFO ] [MAIN ] COHR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:08:55.215575] [DEBUG] [LOCAL ] COHR: Loaded 53379 existing records from primary storage. [2025-11-04T18:08:55.216096] [INFO ] [MAIN ] COHR: Loaded 53379 existing records. [2025-11-04T18:08:55.376480] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-03. [2025-11-04T18:08:55.398091] [WARN ] [MAIN ] COHR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:08:55.399199] [INFO ] [MAIN ] COHR: Wiping existing primary data before full refresh. [2025-11-04T18:08:55.399252] [INFO ] [LOCAL ] COHR: Starting deletion of primary data. [2025-11-04T18:08:55.399497] [DEBUG] [LOCAL ] COHR: Deleted primary data file. [2025-11-04T18:08:55.399554] [INFO ] [MAIN ] COHR: Starting full primary refresh. [2025-11-04T18:08:55.758698] [INFO ] [REMOTE ] COHR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:08:56.167069] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-03. [2025-11-04T18:08:56.311393] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-11-04T18:08:56.748551] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T18:08:56.964688] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-11-04T18:08:56.967227] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-01. [2025-11-04T18:08:57.767221] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-04. [2025-11-04T18:08:58.457162] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-11-04T18:08:58.567200] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-02. [2025-11-04T18:08:59.162965] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T18:08:59.367174] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-05. [2025-11-04T18:09:00.045285] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-11-04T18:09:00.167240] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-04. [2025-11-04T18:09:00.898904] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-11-04T18:09:00.967067] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-03. [2025-11-04T18:09:01.043154] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-11-04T18:09:01.767061] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-02. [2025-11-04T18:09:02.467486] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-11-04T18:09:02.567063] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-06. [2025-11-04T18:09:02.928326] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T18:09:03.367068] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-03. [2025-11-04T18:09:04.054978] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-11-04T18:09:04.167127] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-07. [2025-11-04T18:09:04.863189] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-11-04T18:09:04.967044] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-04. [2025-11-04T18:09:05.364266] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T18:09:05.767053] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-08. [2025-11-04T18:09:06.567045] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-04. [2025-11-04T18:09:06.835915] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-11-04T18:09:06.993171] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T18:09:07.367087] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-09. [2025-11-04T18:09:08.175381] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-05. [2025-11-04T18:09:08.182882] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-11-04T18:09:08.655197] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T18:09:08.967095] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-05. [2025-11-04T18:09:09.535503] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T18:09:09.767019] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-10. [2025-11-04T18:09:10.526222] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-11-04T18:09:10.566999] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-06. [2025-11-04T18:09:11.367033] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-05. [2025-11-04T18:09:11.482095] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-11-04T18:09:12.085130] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-11-04T18:09:12.167002] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-07. [2025-11-04T18:09:12.691850] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:09:12.967087] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-06. [2025-11-04T18:09:13.668922] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-11-04T18:09:13.767019] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-08. [2025-11-04T18:09:14.217082] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-11-04T18:09:14.567056] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-06. [2025-11-04T18:09:15.374281] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-11. [2025-11-04T18:09:15.393915] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-11-04T18:09:15.791127] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-11-04T18:09:16.167088] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-09. [2025-11-04T18:09:16.419998] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T18:09:16.422755] [INFO ] [REMOTE ] CNX: Full fetch complete (74880 records). [2025-11-04T18:09:16.422953] [DEBUG] [VALIDATOR ] CNX: Validating primary data. [2025-11-04T18:09:16.827758] [DEBUG] [VALIDATOR ] CNX: All expected trading days are present. [2025-11-04T18:09:16.828062] [DEBUG] [VALIDATOR ] CNX: Primary data validation passed. [2025-11-04T18:09:16.828896] [INFO ] [MAIN ] CNX: Fetched primary data is valid. [2025-11-04T18:09:16.828957] [DEBUG] [LOCAL ] CNX: Saving primary data. [2025-11-04T18:09:16.830292] [DEBUG] [LOCAL ] CNX: Trimmed 74880 records to 51286 (keeping 200 days of data) [2025-11-04T18:09:16.992762] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-07. [2025-11-04T18:09:17.125837] [DEBUG] [LOCAL ] CNX: Primary data saved (51286 records). [2025-11-04T18:09:17.125970] [INFO ] [MAIN ] CNX: Full history saved. [2025-11-04T18:09:17.126051] [INFO ] [MAIN ] Update Progress: 503/2367 symbols completed. CNX took 31.9s. Estimated remaining: 4h 16m 33s. Rate: 7.3 symbols/min [2025-11-04T18:09:17.297168] [INFO ] [MAIN ] COLD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:09:17.345961] [DEBUG] [LOCAL ] COLD: Loaded 52554 existing records from primary storage. [2025-11-04T18:09:17.346263] [INFO ] [MAIN ] COLD: Loaded 52554 existing records. [2025-11-04T18:09:17.563497] [WARN ] [MAIN ] COLD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:09:17.563957] [INFO ] [MAIN ] COLD: Wiping existing primary data before full refresh. [2025-11-04T18:09:17.564017] [INFO ] [LOCAL ] COLD: Starting deletion of primary data. [2025-11-04T18:09:17.564347] [DEBUG] [LOCAL ] COLD: Deleted primary data file. [2025-11-04T18:09:17.564405] [INFO ] [MAIN ] COLD: Starting full primary refresh. [2025-11-04T18:09:17.769250] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-07. [2025-11-04T18:09:17.953369] [INFO ] [REMOTE ] COLD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:09:17.985508] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-11-04T18:09:18.567085] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-01. [2025-11-04T18:09:18.751794] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-11-04T18:09:18.994631] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T18:09:19.367076] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-08. [2025-11-04T18:09:19.816314] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T18:09:20.167038] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-10. [2025-11-04T18:09:20.563627] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T18:09:20.967067] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-09. [2025-11-04T18:09:21.767030] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-08. [2025-11-04T18:09:21.831871] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-11-04T18:09:22.036017] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-11-04T18:09:22.567028] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-10. [2025-11-04T18:09:23.245945] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T18:09:23.367005] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-09. [2025-11-04T18:09:23.844673] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-11-04T18:09:24.167035] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-11. [2025-11-04T18:09:24.945714] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-11-04T18:09:24.967044] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-11. [2025-11-04T18:09:25.418669] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T18:09:25.420550] [INFO ] [REMOTE ] CODI: Full fetch complete (55258 records). [2025-11-04T18:09:25.420698] [DEBUG] [VALIDATOR ] CODI: Validating primary data. [2025-11-04T18:09:25.779047] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-10. [2025-11-04T18:09:25.813175] [DEBUG] [VALIDATOR ] CODI: All expected trading days are present. [2025-11-04T18:09:25.813283] [DEBUG] [VALIDATOR ] CODI: Primary data validation passed. [2025-11-04T18:09:25.813951] [INFO ] [MAIN ] CODI: Fetched primary data is valid. [2025-11-04T18:09:25.814014] [DEBUG] [LOCAL ] CODI: Saving primary data. [2025-11-04T18:09:25.815501] [DEBUG] [LOCAL ] CODI: Trimmed 55258 records to 42645 (keeping 200 days of data) [2025-11-04T18:09:26.046879] [DEBUG] [LOCAL ] CODI: Primary data saved (42645 records). [2025-11-04T18:09:26.047007] [INFO ] [MAIN ] CODI: Full history saved. [2025-11-04T18:09:26.047084] [INFO ] [MAIN ] Update Progress: 504/2367 symbols completed. CODI took 38.4s. Estimated remaining: 4h 16m 27s. Rate: 7.3 symbols/min [2025-11-04T18:09:26.216261] [INFO ] [MAIN ] COMP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:09:26.263582] [DEBUG] [LOCAL ] COMP: Loaded 53390 existing records from primary storage. [2025-11-04T18:09:26.263952] [INFO ] [MAIN ] COMP: Loaded 53390 existing records. [2025-11-04T18:09:26.439101] [WARN ] [MAIN ] COMP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:09:26.439322] [INFO ] [MAIN ] COMP: Wiping existing primary data before full refresh. [2025-11-04T18:09:26.439369] [INFO ] [LOCAL ] COMP: Starting deletion of primary data. [2025-11-04T18:09:26.439587] [DEBUG] [LOCAL ] COMP: Deleted primary data file. [2025-11-04T18:09:26.439675] [INFO ] [MAIN ] COMP: Starting full primary refresh. [2025-11-04T18:09:26.593572] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-02. [2025-11-04T18:09:26.641183] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-11 (1.67 seconds). [2025-11-04T18:09:26.677181] [INFO ] [REMOTE ] COHR: Full fetch complete (77471 records). [2025-11-04T18:09:26.689543] [DEBUG] [VALIDATOR ] COHR: Validating primary data. [2025-11-04T18:09:26.995529] [INFO ] [REMOTE ] COMP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:09:27.240154] [DEBUG] [VALIDATOR ] COHR: All expected trading days are present. [2025-11-04T18:09:27.240232] [DEBUG] [VALIDATOR ] COHR: Primary data validation passed. [2025-11-04T18:09:27.241064] [INFO ] [MAIN ] COHR: Fetched primary data is valid. [2025-11-04T18:09:27.241126] [DEBUG] [LOCAL ] COHR: Saving primary data. [2025-11-04T18:09:27.242509] [DEBUG] [LOCAL ] COHR: Trimmed 77471 records to 53383 (keeping 200 days of data) [2025-11-04T18:09:27.411379] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-01. [2025-11-04T18:09:27.597449] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-11-04T18:09:27.597547] [DEBUG] [LOCAL ] COHR: Primary data saved (53383 records). [2025-11-04T18:09:27.609605] [INFO ] [MAIN ] COHR: Full history saved. [2025-11-04T18:09:27.610299] [INFO ] [MAIN ] Update Progress: 505/2367 symbols completed. COHR took 32.6s. Estimated remaining: 4h 15m 54s. Rate: 7.3 symbols/min [2025-11-04T18:09:27.781747] [INFO ] [MAIN ] CON: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:09:27.814770] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T18:09:27.858548] [DEBUG] [LOCAL ] CON: Loaded 38454 existing records from primary storage. [2025-11-04T18:09:27.858843] [INFO ] [MAIN ] CON: Loaded 38454 existing records. [2025-11-04T18:09:28.031313] [WARN ] [MAIN ] CON: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:09:28.031533] [INFO ] [MAIN ] CON: Wiping existing primary data before full refresh. [2025-11-04T18:09:28.031579] [INFO ] [LOCAL ] CON: Starting deletion of primary data. [2025-11-04T18:09:28.031777] [DEBUG] [LOCAL ] CON: Deleted primary data file. [2025-11-04T18:09:28.031831] [INFO ] [MAIN ] CON: Starting full primary refresh. [2025-11-04T18:09:28.172294] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-11. [2025-11-04T18:09:28.408059] [INFO ] [REMOTE ] CON: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:09:28.791428] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T18:09:28.967039] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-01. [2025-11-04T18:09:29.767009] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-03. [2025-11-04T18:09:29.819456] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-11 (1.65 seconds). [2025-11-04T18:09:29.821471] [INFO ] [REMOTE ] COF: Full fetch complete (77721 records). [2025-11-04T18:09:29.821568] [DEBUG] [VALIDATOR ] COF: Validating primary data. [2025-11-04T18:09:30.252264] [DEBUG] [VALIDATOR ] COF: All expected trading days are present. [2025-11-04T18:09:30.252523] [DEBUG] [VALIDATOR ] COF: Primary data validation passed. [2025-11-04T18:09:30.254820] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T18:09:30.253966] [INFO ] [MAIN ] COF: Fetched primary data is valid. [2025-11-04T18:09:30.254956] [DEBUG] [LOCAL ] COF: Saving primary data. [2025-11-04T18:09:30.264272] [DEBUG] [LOCAL ] COF: Trimmed 77721 records to 53630 (keeping 200 days of data) [2025-11-04T18:09:30.573429] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-02. [2025-11-04T18:09:30.599735] [DEBUG] [LOCAL ] COF: Primary data saved (53630 records). [2025-11-04T18:09:30.599852] [INFO ] [MAIN ] COF: Full history saved. [2025-11-04T18:09:30.599940] [INFO ] [MAIN ] Update Progress: 506/2367 symbols completed. COF took 37.8s. Estimated remaining: 4h 15m 27s. Rate: 7.3 symbols/min [2025-11-04T18:09:30.770385] [INFO ] [MAIN ] COOK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:09:30.793757] [DEBUG] [LOCAL ] COOK: Loaded 24873 existing records from primary storage. [2025-11-04T18:09:30.794031] [INFO ] [MAIN ] COOK: Loaded 24873 existing records. [2025-11-04T18:09:30.979056] [WARN ] [MAIN ] COOK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:09:30.979374] [INFO ] [MAIN ] COOK: Wiping existing primary data before full refresh. [2025-11-04T18:09:30.979447] [INFO ] [LOCAL ] COOK: Starting deletion of primary data. [2025-11-04T18:09:30.979994] [DEBUG] [LOCAL ] COOK: Deleted primary data file. [2025-11-04T18:09:30.980045] [INFO ] [MAIN ] COOK: Starting full primary refresh. [2025-11-04T18:09:31.353101] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-11-04T18:09:31.398014] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-02. [2025-11-04T18:09:31.403103] [INFO ] [REMOTE ] COOK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:09:31.568849] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T18:09:32.167066] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-04. [2025-11-04T18:09:32.583140] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T18:09:32.967067] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-03. [2025-11-04T18:09:33.382676] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T18:09:33.767041] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-03. [2025-11-04T18:09:34.003180] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T18:09:34.567041] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-04. [2025-11-04T18:09:35.086485] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T18:09:35.367118] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-04. [2025-11-04T18:09:36.167052] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-05. [2025-11-04T18:09:36.604353] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T18:09:36.619797] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-11-04T18:09:36.967094] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-05. [2025-11-04T18:09:37.544566] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-11-04T18:09:37.767053] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-01. [2025-11-04T18:09:38.342987] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-11-04T18:09:38.567027] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-06. [2025-11-04T18:09:38.585724] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-11-04T18:09:39.367092] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-02. [2025-11-04T18:09:40.027363] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-11-04T18:09:40.032942] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T18:09:40.167042] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-07. [2025-11-04T18:09:40.967094] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-06. [2025-11-04T18:09:41.767083] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-03. [2025-11-04T18:09:41.853049] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-11-04T18:09:42.402126] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T18:09:42.567049] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-08. [2025-11-04T18:09:43.367031] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-05. [2025-11-04T18:09:43.479700] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-06 (2.51 seconds). [2025-11-04T18:09:43.856915] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T18:09:44.167251] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-07. [2025-11-04T18:09:44.356264] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T18:09:44.967041] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-09. [2025-11-04T18:09:45.655016] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-11-04T18:09:45.767040] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-04. [2025-11-04T18:09:46.131248] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T18:09:46.409847] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-11-04T18:09:46.567031] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-10. [2025-11-04T18:09:47.367037] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-06. [2025-11-04T18:09:47.968060] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T18:09:48.167052] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-05. [2025-11-04T18:09:48.391967] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T18:09:48.967044] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-07. [2025-11-04T18:09:49.155779] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T18:09:49.767019] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-06. [2025-11-04T18:09:49.927054] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T18:09:50.555265] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T18:09:50.567005] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-08. [2025-11-04T18:09:51.367036] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-07. [2025-11-04T18:09:51.636950] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T18:09:52.167021] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-09. [2025-11-04T18:09:52.197511] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T18:09:52.967096] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-11. [2025-11-04T18:09:53.339913] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T18:09:53.767049] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-08. [2025-11-04T18:09:54.392299] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-11 (1.43 seconds). [2025-11-04T18:09:54.395620] [INFO ] [REMOTE ] COLD: Full fetch complete (75180 records). [2025-11-04T18:09:54.395853] [DEBUG] [VALIDATOR ] COLD: Validating primary data. [2025-11-04T18:09:54.567441] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-08. [2025-11-04T18:09:54.777797] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-11-04T18:09:54.892197] [DEBUG] [VALIDATOR ] COLD: All expected trading days are present. [2025-11-04T18:09:54.892320] [DEBUG] [VALIDATOR ] COLD: Primary data validation passed. [2025-11-04T18:09:54.893787] [INFO ] [MAIN ] COLD: Fetched primary data is valid. [2025-11-04T18:09:54.893860] [DEBUG] [LOCAL ] COLD: Saving primary data. [2025-11-04T18:09:54.896324] [DEBUG] [LOCAL ] COLD: Trimmed 75180 records to 52591 (keeping 200 days of data) [2025-11-04T18:09:55.269799] [DEBUG] [LOCAL ] COLD: Primary data saved (52591 records). [2025-11-04T18:09:55.270045] [INFO ] [MAIN ] COLD: Full history saved. [2025-11-04T18:09:55.270176] [INFO ] [MAIN ] Update Progress: 507/2367 symbols completed. COLD took 38.1s. Estimated remaining: 4h 16m 19s. Rate: 7.3 symbols/min [2025-11-04T18:09:55.372314] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-09. [2025-11-04T18:09:55.483016] [INFO ] [MAIN ] COP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:09:55.541076] [DEBUG] [LOCAL ] COP: Loaded 53727 existing records from primary storage. [2025-11-04T18:09:55.541581] [INFO ] [MAIN ] COP: Loaded 53727 existing records. [2025-11-04T18:09:55.745631] [WARN ] [MAIN ] COP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:09:55.745969] [INFO ] [MAIN ] COP: Wiping existing primary data before full refresh. [2025-11-04T18:09:55.746021] [INFO ] [LOCAL ] COP: Starting deletion of primary data. [2025-11-04T18:09:55.746275] [DEBUG] [LOCAL ] COP: Deleted primary data file. [2025-11-04T18:09:55.746333] [INFO ] [MAIN ] COP: Starting full primary refresh. [2025-11-04T18:09:56.152757] [INFO ] [REMOTE ] COP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:09:56.167023] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-01. [2025-11-04T18:09:56.248442] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-11-04T18:09:56.388783] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T18:09:56.967038] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-10. [2025-11-04T18:09:57.767017] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-10. [2025-11-04T18:09:57.798508] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-11-04T18:09:58.347281] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-01 (2.18 seconds). [2025-11-04T18:09:58.567034] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-09. [2025-11-04T18:09:58.878101] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-11-04T18:09:59.367087] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-02. [2025-11-04T18:09:59.798731] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T18:10:00.167106] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-11. [2025-11-04T18:10:00.783952] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-11-04T18:10:00.967102] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-11. [2025-11-04T18:10:01.292523] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T18:10:01.294868] [INFO ] [REMOTE ] CON: Full fetch complete (56631 records). [2025-11-04T18:10:01.295134] [DEBUG] [VALIDATOR ] CON: Validating primary data. [2025-11-04T18:10:01.723060] [DEBUG] [VALIDATOR ] CON: All expected trading days are present. [2025-11-04T18:10:01.723220] [DEBUG] [VALIDATOR ] CON: Primary data validation passed. [2025-11-04T18:10:01.724686] [INFO ] [MAIN ] CON: Fetched primary data is valid. [2025-11-04T18:10:01.724763] [DEBUG] [LOCAL ] CON: Saving primary data. [2025-11-04T18:10:01.726512] [DEBUG] [LOCAL ] CON: Trimmed 56631 records to 38507 (keeping 200 days of data) [2025-11-04T18:10:01.784309] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-10. [2025-11-04T18:10:01.989893] [DEBUG] [LOCAL ] CON: Primary data saved (38507 records). [2025-11-04T18:10:01.990094] [INFO ] [MAIN ] CON: Full history saved. [2025-11-04T18:10:01.990793] [INFO ] [MAIN ] Update Progress: 508/2367 symbols completed. CON took 34.4s. Estimated remaining: 4h 16m 5s. Rate: 7.3 symbols/min [2025-11-04T18:10:02.181981] [INFO ] [MAIN ] COR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:10:02.182273] [DEBUG] [LOCAL ] COR: No existing data file found in primary storage. [2025-11-04T18:10:02.182341] [WARN ] [MAIN ] COR: No existing data found, performing full refresh instead. [2025-11-04T18:10:02.182516] [INFO ] [MAIN ] COR: Wiping existing primary data before full refresh. [2025-11-04T18:10:02.182576] [INFO ] [LOCAL ] COR: Starting deletion of primary data. [2025-11-04T18:10:02.182626] [DEBUG] [LOCAL ] COR: No primary data file found to delete. [2025-11-04T18:10:02.182701] [INFO ] [MAIN ] COR: Starting full primary refresh. [2025-11-04T18:10:02.295186] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-11 (1.33 seconds). [2025-11-04T18:10:02.300962] [INFO ] [REMOTE ] COOK: Full fetch complete (33183 records). [2025-11-04T18:10:02.311959] [DEBUG] [VALIDATOR ] COOK: Validating primary data. [2025-11-04T18:10:02.348575] [FATAL] [VALIDATOR ] COOK: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:10:02.373037] [FATAL] [MAIN ] COOK: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:10:02.391909] [WARN ] [MAIN ] COOK: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:10:02.442771] [INFO ] [LOCAL ] COOK: Starting deletion of primary data. [2025-11-04T18:10:02.480493] [DEBUG] [LOCAL ] COOK: No primary data file found to delete. [2025-11-04T18:10:02.504956] [INFO ] [LOCAL ] COOK: Starting deletion of derivative data. [2025-11-04T18:10:02.517078] [DEBUG] [LOCAL ] COOK: Deleted derivative data file. [2025-11-04T18:10:02.528448] [INFO ] [MAIN ] Update Progress: 509/2367 symbols completed. COOK took 31.9s. Estimated remaining: 4h 15m 28s. Rate: 7.3 symbols/min [2025-11-04T18:10:02.575016] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-03. [2025-11-04T18:10:02.646198] [INFO ] [REMOTE ] COR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:10:02.815427] [INFO ] [MAIN ] COTY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:10:02.866502] [DEBUG] [LOCAL ] COTY: Loaded 53237 existing records from primary storage. [2025-11-04T18:10:02.866868] [INFO ] [MAIN ] COTY: Loaded 53237 existing records. [2025-11-04T18:10:03.063395] [WARN ] [MAIN ] COTY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:10:03.063710] [INFO ] [MAIN ] COTY: Wiping existing primary data before full refresh. [2025-11-04T18:10:03.063771] [INFO ] [LOCAL ] COTY: Starting deletion of primary data. [2025-11-04T18:10:03.063963] [DEBUG] [LOCAL ] COTY: Deleted primary data file. [2025-11-04T18:10:03.064000] [INFO ] [MAIN ] COTY: Starting full primary refresh. [2025-11-04T18:10:03.378590] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-01. [2025-11-04T18:10:03.438214] [INFO ] [REMOTE ] COTY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:10:03.471956] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T18:10:04.167006] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-01. [2025-11-04T18:10:04.276857] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-11-04T18:10:04.838610] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-11-04T18:10:04.967057] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-02. [2025-11-04T18:10:05.497126] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T18:10:05.767031] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-11. [2025-11-04T18:10:06.177018] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T18:10:06.567044] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-02. [2025-11-04T18:10:07.111418] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-11 (1.34 seconds). [2025-11-04T18:10:07.113438] [INFO ] [REMOTE ] COMP: Full fetch complete (77416 records). [2025-11-04T18:10:07.113534] [DEBUG] [VALIDATOR ] COMP: Validating primary data. [2025-11-04T18:10:07.377735] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-03. [2025-11-04T18:10:07.521383] [DEBUG] [VALIDATOR ] COMP: All expected trading days are present. [2025-11-04T18:10:07.521487] [DEBUG] [VALIDATOR ] COMP: Primary data validation passed. [2025-11-04T18:10:07.522335] [INFO ] [MAIN ] COMP: Fetched primary data is valid. [2025-11-04T18:10:07.522406] [DEBUG] [LOCAL ] COMP: Saving primary data. [2025-11-04T18:10:07.523893] [DEBUG] [LOCAL ] COMP: Trimmed 77416 records to 53390 (keeping 200 days of data) [2025-11-04T18:10:07.826388] [DEBUG] [LOCAL ] COMP: Primary data saved (53390 records). [2025-11-04T18:10:07.826560] [INFO ] [MAIN ] COMP: Full history saved. [2025-11-04T18:10:07.826678] [INFO ] [MAIN ] Update Progress: 510/2367 symbols completed. COMP took 41.8s. Estimated remaining: 4h 15m 9s. Rate: 7.3 symbols/min [2025-11-04T18:10:08.004270] [INFO ] [MAIN ] COUR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:10:08.055191] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-11-04T18:10:08.081548] [DEBUG] [LOCAL ] COUR: Loaded 49571 existing records from primary storage. [2025-11-04T18:10:08.081879] [INFO ] [MAIN ] COUR: Loaded 49571 existing records. [2025-11-04T18:10:08.172269] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-04. [2025-11-04T18:10:08.268454] [WARN ] [MAIN ] COUR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:10:08.268720] [INFO ] [MAIN ] COUR: Wiping existing primary data before full refresh. [2025-11-04T18:10:08.268776] [INFO ] [LOCAL ] COUR: Starting deletion of primary data. [2025-11-04T18:10:08.269190] [DEBUG] [LOCAL ] COUR: Deleted primary data file. [2025-11-04T18:10:08.269231] [INFO ] [MAIN ] COUR: Starting full primary refresh. [2025-11-04T18:10:08.624407] [INFO ] [REMOTE ] COUR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:10:08.897936] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T18:10:08.967050] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-04. [2025-11-04T18:10:09.767006] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-01. [2025-11-04T18:10:09.895496] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-11-04T18:10:10.371252] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T18:10:10.567052] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-05. [2025-11-04T18:10:11.287284] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-11-04T18:10:11.367014] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-02. [2025-11-04T18:10:12.026813] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T18:10:12.167055] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-06. [2025-11-04T18:10:12.506419] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T18:10:12.967045] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-03. [2025-11-04T18:10:13.733160] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-11-04T18:10:13.767005] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-05. [2025-11-04T18:10:14.349708] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T18:10:14.567011] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-03. [2025-11-04T18:10:15.228346] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T18:10:15.367044] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-04. [2025-11-04T18:10:16.095290] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T18:10:16.167068] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-04. [2025-11-04T18:10:16.967108] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-07. [2025-11-04T18:10:17.087664] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-11-04T18:10:17.774680] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-06. [2025-11-04T18:10:17.780523] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-11-04T18:10:18.567154] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-05. [2025-11-04T18:10:18.887274] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-11-04T18:10:19.367018] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-08. [2025-11-04T18:10:19.404591] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-11-04T18:10:19.733846] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T18:10:20.167075] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-05. [2025-11-04T18:10:20.967010] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-06. [2025-11-04T18:10:21.023872] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-11-04T18:10:21.491455] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:10:21.767091] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-06. [2025-11-04T18:10:22.567019] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-09. [2025-11-04T18:10:22.708458] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-11-04T18:10:23.081209] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T18:10:23.367113] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-07. [2025-11-04T18:10:23.985298] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T18:10:24.167029] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-07. [2025-11-04T18:10:24.792004] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-11-04T18:10:24.967078] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-10. [2025-11-04T18:10:25.445257] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T18:10:25.767231] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-08. [2025-11-04T18:10:26.417635] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T18:10:26.567007] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-11. [2025-11-04T18:10:26.908782] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T18:10:26.911040] [INFO ] [REMOTE ] COP: Full fetch complete (77906 records). [2025-11-04T18:10:26.911187] [DEBUG] [VALIDATOR ] COP: Validating primary data. [2025-11-04T18:10:27.315502] [DEBUG] [VALIDATOR ] COP: All expected trading days are present. [2025-11-04T18:10:27.315627] [DEBUG] [VALIDATOR ] COP: Primary data validation passed. [2025-11-04T18:10:27.316591] [INFO ] [MAIN ] COP: Fetched primary data is valid. [2025-11-04T18:10:27.316708] [DEBUG] [LOCAL ] COP: Saving primary data. [2025-11-04T18:10:27.318593] [DEBUG] [LOCAL ] COP: Trimmed 77906 records to 53727 (keeping 200 days of data) [2025-11-04T18:10:27.382235] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-08. [2025-11-04T18:10:27.440610] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-11-04T18:10:27.683148] [DEBUG] [LOCAL ] COP: Primary data saved (53727 records). [2025-11-04T18:10:27.683288] [INFO ] [MAIN ] COP: Full history saved. [2025-11-04T18:10:27.683366] [INFO ] [MAIN ] Update Progress: 511/2367 symbols completed. COP took 32.4s. Estimated remaining: 4h 15m 43s. Rate: 7.3 symbols/min [2025-11-04T18:10:27.852929] [INFO ] [MAIN ] CP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:10:27.902043] [DEBUG] [LOCAL ] CP: Loaded 53208 existing records from primary storage. [2025-11-04T18:10:27.902351] [INFO ] [MAIN ] CP: Loaded 53208 existing records. [2025-11-04T18:10:28.081179] [WARN ] [MAIN ] CP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:10:28.081452] [INFO ] [MAIN ] CP: Wiping existing primary data before full refresh. [2025-11-04T18:10:28.081501] [INFO ] [LOCAL ] CP: Starting deletion of primary data. [2025-11-04T18:10:28.081841] [DEBUG] [LOCAL ] CP: Deleted primary data file. [2025-11-04T18:10:28.081899] [INFO ] [MAIN ] CP: Starting full primary refresh. [2025-11-04T18:10:28.172330] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-07. [2025-11-04T18:10:28.483223] [INFO ] [REMOTE ] CP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:10:28.967102] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-09. [2025-11-04T18:10:29.044535] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-11-04T18:10:29.605347] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-11-04T18:10:29.767061] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-08. [2025-11-04T18:10:30.494980] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-11-04T18:10:30.567067] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-10. [2025-11-04T18:10:30.910458] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T18:10:31.367021] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-09. [2025-11-04T18:10:31.633888] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T18:10:32.167099] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-09. [2025-11-04T18:10:32.560031] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T18:10:32.967094] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-10. [2025-11-04T18:10:33.343587] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:10:33.767024] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-11. [2025-11-04T18:10:34.412796] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T18:10:34.567045] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-01. [2025-11-04T18:10:34.805397] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T18:10:34.807863] [INFO ] [REMOTE ] COR: Full fetch complete (68269 records). [2025-11-04T18:10:34.808166] [DEBUG] [VALIDATOR ] COR: Validating primary data. [2025-11-04T18:10:35.207993] [DEBUG] [VALIDATOR ] COR: All expected trading days are present. [2025-11-04T18:10:35.208083] [DEBUG] [VALIDATOR ] COR: Primary data validation passed. [2025-11-04T18:10:35.209112] [INFO ] [MAIN ] COR: Fetched primary data is valid. [2025-11-04T18:10:35.209179] [DEBUG] [LOCAL ] COR: Saving primary data. [2025-11-04T18:10:35.210590] [DEBUG] [LOCAL ] COR: Trimmed 68269 records to 45875 (keeping 200 days of data) [2025-11-04T18:10:35.387776] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-11. [2025-11-04T18:10:35.508819] [DEBUG] [LOCAL ] COR: Primary data saved (45875 records). [2025-11-04T18:10:35.508963] [INFO ] [MAIN ] COR: Full history saved. [2025-11-04T18:10:35.509036] [INFO ] [MAIN ] Update Progress: 512/2367 symbols completed. COR took 33.5s. Estimated remaining: 4h 15m 34s. Rate: 7.3 symbols/min [2025-11-04T18:10:35.681023] [INFO ] [MAIN ] CPA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:10:35.708085] [DEBUG] [LOCAL ] CPA: Loaded 27006 existing records from primary storage. [2025-11-04T18:10:35.708342] [INFO ] [MAIN ] CPA: Loaded 27006 existing records. [2025-11-04T18:10:35.890776] [WARN ] [MAIN ] CPA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:10:35.891022] [INFO ] [MAIN ] CPA: Wiping existing primary data before full refresh. [2025-11-04T18:10:35.891106] [INFO ] [LOCAL ] CPA: Starting deletion of primary data. [2025-11-04T18:10:35.891282] [DEBUG] [LOCAL ] CPA: Deleted primary data file. [2025-11-04T18:10:35.891321] [INFO ] [MAIN ] CPA: Starting full primary refresh. [2025-11-04T18:10:36.172745] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-10. [2025-11-04T18:10:36.253112] [INFO ] [REMOTE ] CPA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:10:36.284984] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-11-04T18:10:36.967057] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-02. [2025-11-04T18:10:37.346437] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-11 (1.96 seconds). [2025-11-04T18:10:37.348446] [INFO ] [REMOTE ] COUR: Full fetch complete (71556 records). [2025-11-04T18:10:37.348562] [DEBUG] [VALIDATOR ] COUR: Validating primary data. [2025-11-04T18:10:37.781966] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-01. [2025-11-04T18:10:37.782056] [DEBUG] [VALIDATOR ] COUR: All expected trading days are present. [2025-11-04T18:10:37.782316] [DEBUG] [VALIDATOR ] COUR: Primary data validation passed. [2025-11-04T18:10:37.784158] [INFO ] [MAIN ] COUR: Fetched primary data is valid. [2025-11-04T18:10:37.784465] [DEBUG] [LOCAL ] COUR: Saving primary data. [2025-11-04T18:10:37.816561] [DEBUG] [LOCAL ] COUR: Trimmed 71556 records to 49601 (keeping 200 days of data) [2025-11-04T18:10:37.818788] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-11-04T18:10:38.135899] [DEBUG] [LOCAL ] COUR: Primary data saved (49601 records). [2025-11-04T18:10:38.136033] [INFO ] [MAIN ] COUR: Full history saved. [2025-11-04T18:10:38.136131] [INFO ] [MAIN ] Update Progress: 513/2367 symbols completed. COUR took 30.3s. Estimated remaining: 4h 15m 5s. Rate: 7.3 symbols/min [2025-11-04T18:10:38.311184] [INFO ] [MAIN ] CPAC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:10:38.311741] [DEBUG] [LOCAL ] CPAC: No existing data file found in primary storage. [2025-11-04T18:10:38.311799] [WARN ] [MAIN ] CPAC: No existing data found, performing full refresh instead. [2025-11-04T18:10:38.312009] [INFO ] [MAIN ] CPAC: Wiping existing primary data before full refresh. [2025-11-04T18:10:38.312066] [INFO ] [LOCAL ] CPAC: Starting deletion of primary data. [2025-11-04T18:10:38.312110] [DEBUG] [LOCAL ] CPAC: No primary data file found to delete. [2025-11-04T18:10:38.312142] [INFO ] [MAIN ] CPAC: Starting full primary refresh. [2025-11-04T18:10:38.494096] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-11-04T18:10:38.570412] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-11. [2025-11-04T18:10:38.729075] [INFO ] [REMOTE ] CPAC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:10:38.798708] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T18:10:39.367044] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-03. [2025-11-04T18:10:39.952475] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-11 (1.38 seconds). [2025-11-04T18:10:39.954617] [INFO ] [REMOTE ] COTY: Full fetch complete (77285 records). [2025-11-04T18:10:39.954747] [DEBUG] [VALIDATOR ] COTY: Validating primary data. [2025-11-04T18:10:40.172174] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-01. [2025-11-04T18:10:40.349500] [DEBUG] [VALIDATOR ] COTY: All expected trading days are present. [2025-11-04T18:10:40.349589] [DEBUG] [VALIDATOR ] COTY: Primary data validation passed. [2025-11-04T18:10:40.350436] [INFO ] [MAIN ] COTY: Fetched primary data is valid. [2025-11-04T18:10:40.350503] [DEBUG] [LOCAL ] COTY: Saving primary data. [2025-11-04T18:10:40.352046] [DEBUG] [LOCAL ] COTY: Trimmed 77285 records to 53235 (keeping 200 days of data) [2025-11-04T18:10:40.633083] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-11-04T18:10:40.655385] [DEBUG] [LOCAL ] COTY: Primary data saved (53235 records). [2025-11-04T18:10:40.655470] [INFO ] [MAIN ] COTY: Full history saved. [2025-11-04T18:10:40.655541] [INFO ] [MAIN ] Update Progress: 514/2367 symbols completed. COTY took 38.1s. Estimated remaining: 4h 14m 36s. Rate: 7.3 symbols/min [2025-11-04T18:10:40.829109] [INFO ] [MAIN ] CPAY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:10:40.859135] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-11-04T18:10:40.895410] [DEBUG] [LOCAL ] CPAY: Loaded 29899 existing records from primary storage. [2025-11-04T18:10:40.895681] [INFO ] [MAIN ] CPAY: Loaded 29899 existing records. [2025-11-04T18:10:40.972108] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-04. [2025-11-04T18:10:41.074460] [WARN ] [MAIN ] CPAY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:10:41.074686] [INFO ] [MAIN ] CPAY: Wiping existing primary data before full refresh. [2025-11-04T18:10:41.074745] [INFO ] [LOCAL ] CPAY: Starting deletion of primary data. [2025-11-04T18:10:41.074926] [DEBUG] [LOCAL ] CPAY: Deleted primary data file. [2025-11-04T18:10:41.074982] [INFO ] [MAIN ] CPAY: Starting full primary refresh. [2025-11-04T18:10:41.427403] [INFO ] [REMOTE ] CPAY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:10:41.767011] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-02. [2025-11-04T18:10:42.017678] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-02 (0.25 seconds). [2025-11-04T18:10:42.495620] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-11-04T18:10:42.567009] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-01. [2025-11-04T18:10:43.367099] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-02. [2025-11-04T18:10:43.542489] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T18:10:44.167052] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-05. [2025-11-04T18:10:44.240342] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T18:10:44.967079] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-02. [2025-11-04T18:10:45.581102] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T18:10:45.767003] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-06. [2025-11-04T18:10:45.933026] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T18:10:46.567044] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-03. [2025-11-04T18:10:47.059282] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-11-04T18:10:47.367035] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-07. [2025-11-04T18:10:47.554877] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T18:10:48.167050] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-03. [2025-11-04T18:10:48.402292] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-03 (0.24 seconds). [2025-11-04T18:10:48.721314] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T18:10:48.967051] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-03. [2025-11-04T18:10:49.767048] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-04. [2025-11-04T18:10:49.908254] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-11-04T18:10:50.567100] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-04. [2025-11-04T18:10:50.762516] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-11-04T18:10:50.988313] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-11-04T18:10:51.367042] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-05. [2025-11-04T18:10:51.612774] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-05 (0.25 seconds). [2025-11-04T18:10:52.167035] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-06. [2025-11-04T18:10:52.438300] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-11-04T18:10:52.967032] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-07. [2025-11-04T18:10:53.263033] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-07 (0.30 seconds). [2025-11-04T18:10:53.767071] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-04. [2025-11-04T18:10:54.572230] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-08. [2025-11-04T18:10:54.581365] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-11-04T18:10:55.367063] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-05. [2025-11-04T18:10:56.036944] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-11-04T18:10:56.167035] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-09. [2025-11-04T18:10:56.497638] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T18:10:56.967024] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-05. [2025-11-04T18:10:57.553492] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T18:10:57.767003] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-06. [2025-11-04T18:10:57.884098] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T18:10:58.564878] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-11-04T18:10:58.571362] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-08. [2025-11-04T18:10:58.822063] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-08 (0.25 seconds). [2025-11-04T18:10:59.367089] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-06. [2025-11-04T18:11:00.167078] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-07. [2025-11-04T18:11:00.521682] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T18:11:00.967042] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-09. [2025-11-04T18:11:01.057452] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-11-04T18:11:01.408817] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-11-04T18:11:01.767095] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-08. [2025-11-04T18:11:02.567040] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-07. [2025-11-04T18:11:02.698793] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-11-04T18:11:03.372531] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-10. [2025-11-04T18:11:03.381888] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-11-04T18:11:03.676312] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-10 (0.30 seconds). [2025-11-04T18:11:04.167089] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-11. [2025-11-04T18:11:04.581253] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T18:11:04.582263] [INFO ] [REMOTE ] CPAC: Full fetch complete (1875 records). [2025-11-04T18:11:04.582339] [DEBUG] [VALIDATOR ] CPAC: Validating primary data. [2025-11-04T18:11:04.765616] [WARN ] [VALIDATOR ] CPAC: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-11-04T18:11:04.765872] [FATAL] [MAIN ] CPAC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:11:04.765958] [WARN ] [MAIN ] CPAC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:11:04.766052] [INFO ] [LOCAL ] CPAC: Starting deletion of primary data. [2025-11-04T18:11:04.766128] [DEBUG] [LOCAL ] CPAC: No primary data file found to delete. [2025-11-04T18:11:04.766164] [INFO ] [LOCAL ] CPAC: Starting deletion of derivative data. [2025-11-04T18:11:04.766204] [DEBUG] [LOCAL ] CPAC: No derivative data file found to delete. [2025-11-04T18:11:04.766260] [INFO ] [MAIN ] Update Progress: 515/2367 symbols completed. CPAC took 26.6s. Estimated remaining: 4h 15m 25s. Rate: 7.3 symbols/min [2025-11-04T18:11:04.945158] [INFO ] [MAIN ] CPF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:11:04.967127] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-08. [2025-11-04T18:11:04.974417] [DEBUG] [LOCAL ] CPF: Loaded 14820 existing records from primary storage. [2025-11-04T18:11:04.974745] [INFO ] [MAIN ] CPF: Loaded 14820 existing records. [2025-11-04T18:11:05.163366] [WARN ] [MAIN ] CPF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:11:05.163700] [INFO ] [MAIN ] CPF: Wiping existing primary data before full refresh. [2025-11-04T18:11:05.163760] [INFO ] [LOCAL ] CPF: Starting deletion of primary data. [2025-11-04T18:11:05.164322] [DEBUG] [LOCAL ] CPF: Deleted primary data file. [2025-11-04T18:11:05.164381] [INFO ] [MAIN ] CPF: Starting full primary refresh. [2025-11-04T18:11:05.548429] [INFO ] [REMOTE ] CPF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:11:05.767051] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-01. [2025-11-04T18:11:05.919157] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T18:11:06.553227] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-11-04T18:11:06.567002] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-02. [2025-11-04T18:11:07.271125] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T18:11:07.367070] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-03. [2025-11-04T18:11:08.056721] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-11-04T18:11:08.167035] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-04. [2025-11-04T18:11:08.845634] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T18:11:08.967010] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-09. [2025-11-04T18:11:09.767090] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-10. [2025-11-04T18:11:09.882979] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T18:11:10.567051] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-10. [2025-11-04T18:11:11.358336] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T18:11:11.370034] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-09. [2025-11-04T18:11:11.390954] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T18:11:12.167091] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-05. [2025-11-04T18:11:12.283931] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-11-04T18:11:12.632728] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-11-04T18:11:12.967050] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-06. [2025-11-04T18:11:13.610121] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-11-04T18:11:13.767066] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-11. [2025-11-04T18:11:14.567026] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-07. [2025-11-04T18:11:14.800085] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T18:11:14.801990] [INFO ] [REMOTE ] CPAY: Full fetch complete (44937 records). [2025-11-04T18:11:14.802068] [DEBUG] [VALIDATOR ] CPAY: Validating primary data. [2025-11-04T18:11:15.208016] [DEBUG] [VALIDATOR ] CPAY: All expected trading days are present. [2025-11-04T18:11:15.208187] [DEBUG] [VALIDATOR ] CPAY: Primary data validation passed. [2025-11-04T18:11:15.209035] [INFO ] [MAIN ] CPAY: Fetched primary data is valid. [2025-11-04T18:11:15.209125] [DEBUG] [LOCAL ] CPAY: Saving primary data. [2025-11-04T18:11:15.210385] [DEBUG] [LOCAL ] CPAY: Trimmed 44937 records to 30084 (keeping 200 days of data) [2025-11-04T18:11:15.312905] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-11-04T18:11:15.378695] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-11. [2025-11-04T18:11:15.417091] [DEBUG] [LOCAL ] CPAY: Primary data saved (30084 records). [2025-11-04T18:11:15.417232] [INFO ] [MAIN ] CPAY: Full history saved. [2025-11-04T18:11:15.417314] [INFO ] [MAIN ] Update Progress: 516/2367 symbols completed. CPAY took 34.8s. Estimated remaining: 4h 15m 25s. Rate: 7.2 symbols/min [2025-11-04T18:11:15.591014] [INFO ] [MAIN ] CPK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:11:15.591177] [DEBUG] [LOCAL ] CPK: No existing data file found in primary storage. [2025-11-04T18:11:15.591213] [WARN ] [MAIN ] CPK: No existing data found, performing full refresh instead. [2025-11-04T18:11:15.591321] [INFO ] [MAIN ] CPK: Wiping existing primary data before full refresh. [2025-11-04T18:11:15.591361] [INFO ] [LOCAL ] CPK: Starting deletion of primary data. [2025-11-04T18:11:15.591401] [DEBUG] [LOCAL ] CPK: No primary data file found to delete. [2025-11-04T18:11:15.591431] [INFO ] [MAIN ] CPK: Starting full primary refresh. [2025-11-04T18:11:15.965752] [INFO ] [REMOTE ] CPK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:11:16.167065] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-08. [2025-11-04T18:11:16.562508] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T18:11:16.564543] [INFO ] [REMOTE ] CP: Full fetch complete (77291 records). [2025-11-04T18:11:16.564666] [DEBUG] [VALIDATOR ] CP: Validating primary data. [2025-11-04T18:11:16.972981] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-01. [2025-11-04T18:11:16.952064] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T18:11:16.977387] [DEBUG] [VALIDATOR ] CP: All expected trading days are present. [2025-11-04T18:11:16.980927] [DEBUG] [VALIDATOR ] CP: Primary data validation passed. [2025-11-04T18:11:16.982700] [INFO ] [MAIN ] CP: Fetched primary data is valid. [2025-11-04T18:11:16.983506] [DEBUG] [LOCAL ] CP: Saving primary data. [2025-11-04T18:11:16.985017] [DEBUG] [LOCAL ] CP: Trimmed 77291 records to 53208 (keeping 200 days of data) [2025-11-04T18:11:17.320607] [DEBUG] [LOCAL ] CP: Primary data saved (53208 records). [2025-11-04T18:11:17.320777] [INFO ] [MAIN ] CP: Full history saved. [2025-11-04T18:11:17.320877] [INFO ] [MAIN ] Update Progress: 517/2367 symbols completed. CP took 49.6s. Estimated remaining: 4h 14m 54s. Rate: 7.3 symbols/min [2025-11-04T18:11:17.767054] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-10. [2025-11-04T18:11:17.811701] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-11-04T18:11:18.567086] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-02. [2025-11-04T18:11:18.716262] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T18:11:19.046943] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T18:11:19.367097] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-09. [2025-11-04T18:11:19.974721] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-11-04T18:11:20.167052] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-11. [2025-11-04T18:11:20.865969] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-11 (0.70 seconds). [2025-11-04T18:11:20.867435] [INFO ] [REMOTE ] CPA: Full fetch complete (40434 records). [2025-11-04T18:11:20.867506] [DEBUG] [VALIDATOR ] CPA: Validating primary data. [2025-11-04T18:11:20.972342] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-03. [2025-11-04T18:11:21.257534] [DEBUG] [VALIDATOR ] CPA: All expected trading days are present. [2025-11-04T18:11:21.257608] [DEBUG] [VALIDATOR ] CPA: Primary data validation passed. [2025-11-04T18:11:21.258146] [INFO ] [MAIN ] CPA: Fetched primary data is valid. [2025-11-04T18:11:21.258206] [DEBUG] [LOCAL ] CPA: Saving primary data. [2025-11-04T18:11:21.259085] [DEBUG] [LOCAL ] CPA: Trimmed 40434 records to 27029 (keeping 200 days of data) [2025-11-04T18:11:21.422622] [DEBUG] [LOCAL ] CPA: Primary data saved (27029 records). [2025-11-04T18:11:21.422788] [INFO ] [MAIN ] CPA: Full history saved. [2025-11-04T18:11:21.422883] [INFO ] [MAIN ] Update Progress: 518/2367 symbols completed. CPA took 45.9s. Estimated remaining: 4h 14m 31s. Rate: 7.3 symbols/min [2025-11-04T18:11:21.615862] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T18:11:21.767065] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-10. [2025-11-04T18:11:22.410210] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-11-04T18:11:22.567076] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-11. [2025-11-04T18:11:23.137191] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T18:11:23.138093] [INFO ] [REMOTE ] CPF: Full fetch complete (22942 records). [2025-11-04T18:11:23.138173] [DEBUG] [VALIDATOR ] CPF: Validating primary data. [2025-11-04T18:11:23.138611] [FATAL] [VALIDATOR ] CPF: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:11:23.138741] [FATAL] [MAIN ] CPF: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:11:23.138807] [WARN ] [MAIN ] CPF: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:11:23.138876] [INFO ] [LOCAL ] CPF: Starting deletion of primary data. [2025-11-04T18:11:23.138939] [DEBUG] [LOCAL ] CPF: No primary data file found to delete. [2025-11-04T18:11:23.138973] [INFO ] [LOCAL ] CPF: Starting deletion of derivative data. [2025-11-04T18:11:23.139237] [DEBUG] [LOCAL ] CPF: Deleted derivative data file. [2025-11-04T18:11:23.139344] [INFO ] [MAIN ] Update Progress: 519/2367 symbols completed. CPF took 18.4s. Estimated remaining: 4h 13m 59s. Rate: 7.3 symbols/min [2025-11-04T18:11:23.367081] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-04. [2025-11-04T18:11:24.166291] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-11-04T18:11:24.171394] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-05. [2025-11-04T18:11:24.762767] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T18:11:24.967152] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-06. [2025-11-04T18:11:25.583088] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-11-04T18:11:25.767158] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-07. [2025-11-04T18:11:26.436321] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T18:11:26.567074] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-08. [2025-11-04T18:11:27.069944] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-11-04T18:11:27.367107] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-09. [2025-11-04T18:11:27.997702] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T18:11:28.167054] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-10. [2025-11-04T18:11:28.709555] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-11-04T18:11:28.967050] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-11. [2025-11-04T18:11:29.603776] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T18:11:29.604779] [INFO ] [REMOTE ] CPK: Full fetch complete (18985 records). [2025-11-04T18:11:29.604853] [DEBUG] [VALIDATOR ] CPK: Validating primary data. [2025-11-04T18:11:30.037342] [DEBUG] [VALIDATOR ] CPK: All expected trading days are present. [2025-11-04T18:11:30.037421] [DEBUG] [VALIDATOR ] CPK: Primary data validation passed. [2025-11-04T18:11:30.037854] [INFO ] [MAIN ] CPK: Fetched primary data is valid. [2025-11-04T18:11:30.037916] [DEBUG] [LOCAL ] CPK: Saving primary data. [2025-11-04T18:11:30.038532] [DEBUG] [LOCAL ] CPK: Trimmed 18985 records to 13203 (keeping 200 days of data) [2025-11-04T18:11:30.126432] [DEBUG] [LOCAL ] CPK: Primary data saved (13203 records). [2025-11-04T18:11:30.126547] [INFO ] [MAIN ] CPK: Full history saved. [2025-11-04T18:11:30.126619] [INFO ] [MAIN ] Update Progress: 520/2367 symbols completed. CPK took 14.7s. Estimated remaining: 4h 13m 46s. Rate: 7.3 symbols/min [2025-11-04T18:11:30.717691] [INFO ] [MAIN ] CPS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:11:30.887809] [INFO ] [MAIN ] CPNG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:11:30.893241] [INFO ] [MAIN ] CPRI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:11:30.896255] [INFO ] [MAIN ] CPT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:11:30.933831] [DEBUG] [LOCAL ] CPS: Loaded 24216 existing records from primary storage. [2025-11-04T18:11:30.934462] [INFO ] [MAIN ] CPS: Loaded 24216 existing records. [2025-11-04T18:11:31.177317] [DEBUG] [LOCAL ] CPT: Loaded 44384 existing records from primary storage. [2025-11-04T18:11:31.202145] [INFO ] [MAIN ] CPT: Loaded 44384 existing records. [2025-11-04T18:11:31.199971] [DEBUG] [LOCAL ] CPRI: Loaded 52308 existing records from primary storage. [2025-11-04T18:11:31.248195] [INFO ] [MAIN ] CPRI: Loaded 52308 existing records. [2025-11-04T18:11:31.338293] [DEBUG] [LOCAL ] CPNG: Loaded 53680 existing records from primary storage. [2025-11-04T18:11:31.428722] [INFO ] [MAIN ] CPNG: Loaded 53680 existing records. [2025-11-04T18:11:31.644809] [WARN ] [MAIN ] CPS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:11:31.736338] [INFO ] [MAIN ] CPS: Wiping existing primary data before full refresh. [2025-11-04T18:11:31.790300] [INFO ] [LOCAL ] CPS: Starting deletion of primary data. [2025-11-04T18:11:31.881455] [DEBUG] [LOCAL ] CPS: Deleted primary data file. [2025-11-04T18:11:31.918819] [WARN ] [MAIN ] CPT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:11:31.930076] [INFO ] [MAIN ] CPS: Starting full primary refresh. [2025-11-04T18:11:31.957547] [INFO ] [MAIN ] CPT: Wiping existing primary data before full refresh. [2025-11-04T18:11:32.029947] [INFO ] [LOCAL ] CPT: Starting deletion of primary data. [2025-11-04T18:11:32.092425] [DEBUG] [LOCAL ] CPT: Deleted primary data file. [2025-11-04T18:11:32.131105] [INFO ] [MAIN ] CPT: Starting full primary refresh. [2025-11-04T18:11:32.157580] [WARN ] [MAIN ] CPNG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:11:32.204376] [INFO ] [MAIN ] CPNG: Wiping existing primary data before full refresh. [2025-11-04T18:11:32.204282] [WARN ] [MAIN ] CPRI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:11:32.223527] [INFO ] [MAIN ] CPRI: Wiping existing primary data before full refresh. [2025-11-04T18:11:32.223633] [INFO ] [LOCAL ] CPRI: Starting deletion of primary data. [2025-11-04T18:11:32.212698] [INFO ] [LOCAL ] CPNG: Starting deletion of primary data. [2025-11-04T18:11:32.229144] [DEBUG] [LOCAL ] CPRI: Deleted primary data file. [2025-11-04T18:11:32.234317] [DEBUG] [LOCAL ] CPNG: Deleted primary data file. [2025-11-04T18:11:32.307565] [INFO ] [MAIN ] CPRI: Starting full primary refresh. [2025-11-04T18:11:32.281765] [INFO ] [MAIN ] CPNG: Starting full primary refresh. [2025-11-04T18:11:33.305254] [INFO ] [REMOTE ] CPT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:11:33.319760] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-01. [2025-11-04T18:11:33.314193] [INFO ] [REMOTE ] CPS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:11:33.347698] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-01. [2025-11-04T18:11:33.558047] [INFO ] [REMOTE ] CPRI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:11:33.579981] [INFO ] [REMOTE ] CPNG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:11:33.580218] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-01. [2025-11-04T18:11:33.580718] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-01. [2025-11-04T18:11:34.246736] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T18:11:34.251535] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-02. [2025-11-04T18:11:34.597380] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T18:11:34.605993] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-02. [2025-11-04T18:11:35.042509] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T18:11:35.048304] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-03. [2025-11-04T18:11:35.129756] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-11-04T18:11:35.169367] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-11-04T18:11:35.367066] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-02. [2025-11-04T18:11:35.717130] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T18:11:35.979058] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T18:11:36.167011] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-03. [2025-11-04T18:11:36.550202] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T18:11:36.967105] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-02. [2025-11-04T18:11:37.353525] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T18:11:37.767061] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-03. [2025-11-04T18:11:38.438351] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-11-04T18:11:38.567047] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-03. [2025-11-04T18:11:39.230009] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T18:11:39.367025] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-04. [2025-11-04T18:11:40.136132] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-11-04T18:11:40.167006] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-04. [2025-11-04T18:11:40.560552] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T18:11:40.967104] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-04. [2025-11-04T18:11:41.665520] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-11-04T18:11:41.767015] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-05. [2025-11-04T18:11:41.876946] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-11-04T18:11:42.567070] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-05. [2025-11-04T18:11:43.289209] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-11-04T18:11:43.367012] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-06. [2025-11-04T18:11:44.103680] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T18:11:44.167036] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-06. [2025-11-04T18:11:44.643463] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T18:11:44.967073] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-05. [2025-11-04T18:11:45.767082] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-04. [2025-11-04T18:11:45.956759] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-11-04T18:11:46.091827] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-11-04T18:11:46.567062] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-07. [2025-11-04T18:11:47.080344] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T18:11:47.367040] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-05. [2025-11-04T18:11:48.167025] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-07. [2025-11-04T18:11:48.305173] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-11-04T18:11:48.531498] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T18:11:48.967042] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-06. [2025-11-04T18:11:49.547500] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-11-04T18:11:49.767039] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-08. [2025-11-04T18:11:50.166888] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T18:11:50.567027] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-08. [2025-11-04T18:11:51.161342] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T18:11:51.367073] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-06. [2025-11-04T18:11:52.167022] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-07. [2025-11-04T18:11:52.192893] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T18:11:52.544217] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-11-04T18:11:52.967030] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-07. [2025-11-04T18:11:53.421730] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:11:53.691176] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-11-04T18:11:53.767193] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-08. [2025-11-04T18:11:54.567015] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-08. [2025-11-04T18:11:54.701095] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-11-04T18:11:55.367052] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-09. [2025-11-04T18:11:55.693013] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T18:11:56.167086] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-09. [2025-11-04T18:11:56.967110] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-09. [2025-11-04T18:11:57.048774] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-11-04T18:11:57.542942] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T18:11:57.767009] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-10. [2025-11-04T18:11:57.851880] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T18:11:58.567042] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-09. [2025-11-04T18:11:59.091305] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T18:11:59.367067] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-10. [2025-11-04T18:11:59.766435] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T18:12:00.167052] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-11. [2025-11-04T18:12:00.967075] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-10. [2025-11-04T18:12:01.293824] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-11-04T18:12:01.447859] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-11 (1.28 seconds). [2025-11-04T18:12:01.449699] [INFO ] [REMOTE ] CPRI: Full fetch complete (75772 records). [2025-11-04T18:12:01.449928] [DEBUG] [VALIDATOR ] CPRI: Validating primary data. [2025-11-04T18:12:01.450683] [FATAL] [VALIDATOR ] CPRI: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:12:01.450767] [FATAL] [MAIN ] CPRI: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:12:01.450810] [WARN ] [MAIN ] CPRI: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:12:01.450859] [INFO ] [LOCAL ] CPRI: Starting deletion of primary data. [2025-11-04T18:12:01.450938] [DEBUG] [LOCAL ] CPRI: No primary data file found to delete. [2025-11-04T18:12:01.451012] [INFO ] [LOCAL ] CPRI: Starting deletion of derivative data. [2025-11-04T18:12:01.451144] [DEBUG] [LOCAL ] CPRI: Deleted derivative data file. [2025-11-04T18:12:01.451852] [INFO ] [MAIN ] Update Progress: 521/2367 symbols completed. CPRI took 31.3s. Estimated remaining: 4h 15m 0s. Rate: 7.2 symbols/min [2025-11-04T18:12:01.627750] [INFO ] [MAIN ] CQP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:01.650240] [DEBUG] [LOCAL ] CQP: Loaded 18840 existing records from primary storage. [2025-11-04T18:12:01.650513] [INFO ] [MAIN ] CQP: Loaded 18840 existing records. [2025-11-04T18:12:01.769751] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-11. [2025-11-04T18:12:01.846186] [WARN ] [MAIN ] CQP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:12:01.846448] [INFO ] [MAIN ] CQP: Wiping existing primary data before full refresh. [2025-11-04T18:12:01.846576] [INFO ] [LOCAL ] CQP: Starting deletion of primary data. [2025-11-04T18:12:01.846919] [DEBUG] [LOCAL ] CQP: Deleted primary data file. [2025-11-04T18:12:01.847042] [INFO ] [MAIN ] CQP: Starting full primary refresh. [2025-11-04T18:12:01.975170] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-11-04T18:12:02.239385] [INFO ] [REMOTE ] CQP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:02.567025] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-10. [2025-11-04T18:12:03.367018] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-01. [2025-11-04T18:12:03.372798] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-11 (1.60 seconds). [2025-11-04T18:12:03.374978] [INFO ] [REMOTE ] CPNG: Full fetch complete (77812 records). [2025-11-04T18:12:03.375079] [DEBUG] [VALIDATOR ] CPNG: Validating primary data. [2025-11-04T18:12:03.774664] [DEBUG] [VALIDATOR ] CPNG: All expected trading days are present. [2025-11-04T18:12:03.774767] [DEBUG] [VALIDATOR ] CPNG: Primary data validation passed. [2025-11-04T18:12:03.775770] [INFO ] [MAIN ] CPNG: Fetched primary data is valid. [2025-11-04T18:12:03.775919] [DEBUG] [LOCAL ] CPNG: Saving primary data. [2025-11-04T18:12:03.777606] [DEBUG] [LOCAL ] CPNG: Trimmed 77812 records to 53682 (keeping 200 days of data) [2025-11-04T18:12:03.895440] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-11-04T18:12:04.124142] [DEBUG] [LOCAL ] CPNG: Primary data saved (53682 records). [2025-11-04T18:12:04.124265] [INFO ] [MAIN ] CPNG: Full history saved. [2025-11-04T18:12:04.124339] [INFO ] [MAIN ] Update Progress: 522/2367 symbols completed. CPNG took 34.0s. Estimated remaining: 4h 14m 32s. Rate: 7.2 symbols/min [2025-11-04T18:12:04.167716] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-11. [2025-11-04T18:12:04.302938] [INFO ] [MAIN ] CR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:04.320459] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T18:12:04.345997] [DEBUG] [LOCAL ] CR: Loaded 23938 existing records from primary storage. [2025-11-04T18:12:04.346236] [INFO ] [MAIN ] CR: Loaded 23938 existing records. [2025-11-04T18:12:04.525559] [WARN ] [MAIN ] CR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:12:04.525825] [INFO ] [MAIN ] CR: Wiping existing primary data before full refresh. [2025-11-04T18:12:04.525884] [INFO ] [LOCAL ] CR: Starting deletion of primary data. [2025-11-04T18:12:04.526052] [DEBUG] [LOCAL ] CR: Deleted primary data file. [2025-11-04T18:12:04.526089] [INFO ] [MAIN ] CR: Starting full primary refresh. [2025-11-04T18:12:04.878472] [INFO ] [REMOTE ] CR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:04.928522] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-11 (0.76 seconds). [2025-11-04T18:12:04.929588] [INFO ] [REMOTE ] CPS: Full fetch complete (36579 records). [2025-11-04T18:12:04.929679] [DEBUG] [VALIDATOR ] CPS: Validating primary data. [2025-11-04T18:12:04.930107] [FATAL] [VALIDATOR ] CPS: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:12:04.930187] [FATAL] [MAIN ] CPS: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:12:04.930237] [WARN ] [MAIN ] CPS: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:12:04.930300] [INFO ] [LOCAL ] CPS: Starting deletion of primary data. [2025-11-04T18:12:04.930360] [DEBUG] [LOCAL ] CPS: No primary data file found to delete. [2025-11-04T18:12:04.930393] [INFO ] [LOCAL ] CPS: Starting deletion of derivative data. [2025-11-04T18:12:04.930511] [DEBUG] [LOCAL ] CPS: Deleted derivative data file. [2025-11-04T18:12:04.930573] [INFO ] [MAIN ] Update Progress: 523/2367 symbols completed. CPS took 34.8s. Estimated remaining: 4h 13m 57s. Rate: 7.3 symbols/min [2025-11-04T18:12:04.972122] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-11. [2025-11-04T18:12:05.104549] [INFO ] [MAIN ] CRBD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:05.115756] [DEBUG] [LOCAL ] CRBD: Loaded 8530 existing records from primary storage. [2025-11-04T18:12:05.115971] [INFO ] [MAIN ] CRBD: Loaded 8530 existing records. [2025-11-04T18:12:05.292479] [WARN ] [MAIN ] CRBD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:12:05.292694] [INFO ] [MAIN ] CRBD: Wiping existing primary data before full refresh. [2025-11-04T18:12:05.292750] [INFO ] [LOCAL ] CRBD: Starting deletion of primary data. [2025-11-04T18:12:05.292889] [DEBUG] [LOCAL ] CRBD: Deleted primary data file. [2025-11-04T18:12:05.292940] [INFO ] [MAIN ] CRBD: Starting full primary refresh. [2025-11-04T18:12:05.657946] [INFO ] [REMOTE ] CRBD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:05.767038] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-01. [2025-11-04T18:12:06.111942] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T18:12:06.113896] [INFO ] [REMOTE ] CPT: Full fetch complete (64992 records). [2025-11-04T18:12:06.114015] [DEBUG] [VALIDATOR ] CPT: Validating primary data. [2025-11-04T18:12:06.444537] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-11-04T18:12:06.540387] [DEBUG] [VALIDATOR ] CPT: All expected trading days are present. [2025-11-04T18:12:06.540503] [DEBUG] [VALIDATOR ] CPT: Primary data validation passed. [2025-11-04T18:12:06.541341] [INFO ] [MAIN ] CPT: Fetched primary data is valid. [2025-11-04T18:12:06.541407] [DEBUG] [LOCAL ] CPT: Saving primary data. [2025-11-04T18:12:06.542806] [DEBUG] [LOCAL ] CPT: Trimmed 64992 records to 44453 (keeping 200 days of data) [2025-11-04T18:12:06.596940] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-02. [2025-11-04T18:12:06.817414] [DEBUG] [LOCAL ] CPT: Primary data saved (44453 records). [2025-11-04T18:12:06.817539] [INFO ] [MAIN ] CPT: Full history saved. [2025-11-04T18:12:06.817614] [INFO ] [MAIN ] Update Progress: 524/2367 symbols completed. CPT took 36.7s. Estimated remaining: 4h 13m 27s. Rate: 7.3 symbols/min [2025-11-04T18:12:06.984615] [INFO ] [MAIN ] CRBG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:07.031052] [DEBUG] [LOCAL ] CRBG: Loaded 52552 existing records from primary storage. [2025-11-04T18:12:07.031434] [INFO ] [MAIN ] CRBG: Loaded 52552 existing records. [2025-11-04T18:12:07.214085] [WARN ] [MAIN ] CRBG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:12:07.214329] [INFO ] [MAIN ] CRBG: Wiping existing primary data before full refresh. [2025-11-04T18:12:07.214409] [INFO ] [LOCAL ] CRBG: Starting deletion of primary data. [2025-11-04T18:12:07.214893] [DEBUG] [LOCAL ] CRBG: Deleted primary data file. [2025-11-04T18:12:07.214951] [INFO ] [MAIN ] CRBG: Starting full primary refresh. [2025-11-04T18:12:07.372172] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-02. [2025-11-04T18:12:07.621742] [INFO ] [REMOTE ] CRBG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:07.636160] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-11-04T18:12:07.900622] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-11-04T18:12:08.167075] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-01. [2025-11-04T18:12:08.967093] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-03. [2025-11-04T18:12:09.199493] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-11-04T18:12:09.767019] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-01. [2025-11-04T18:12:09.847076] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-11-04T18:12:10.567074] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-02. [2025-11-04T18:12:11.072245] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T18:12:11.357748] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T18:12:11.367008] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-04. [2025-11-04T18:12:12.062132] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T18:12:12.167036] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-03. [2025-11-04T18:12:12.917272] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-11-04T18:12:12.967094] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-05. [2025-11-04T18:12:13.739022] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-11-04T18:12:13.767055] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-02. [2025-11-04T18:12:14.567089] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-03. [2025-11-04T18:12:14.914925] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T18:12:15.060993] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-11-04T18:12:15.367052] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-04. [2025-11-04T18:12:16.167044] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-04. [2025-11-04T18:12:16.413865] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-11-04T18:12:16.759838] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-11-04T18:12:16.967066] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-05. [2025-11-04T18:12:17.772166] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-06. [2025-11-04T18:12:17.784954] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T18:12:18.540936] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T18:12:18.567030] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-07. [2025-11-04T18:12:19.281135] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-11-04T18:12:19.367096] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-03. [2025-11-04T18:12:20.167074] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-06. [2025-11-04T18:12:20.719568] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-11-04T18:12:20.970947] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-08. [2025-11-04T18:12:20.975299] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T18:12:21.677335] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T18:12:21.767058] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-05. [2025-11-04T18:12:22.293456] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-11-04T18:12:22.567081] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-06. [2025-11-04T18:12:23.097280] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-11-04T18:12:23.367856] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-07. [2025-11-04T18:12:23.811717] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-11-04T18:12:24.167071] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-08. [2025-11-04T18:12:24.696517] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-11-04T18:12:24.967111] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-07. [2025-11-04T18:12:25.770004] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-09. [2025-11-04T18:12:25.776714] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-11-04T18:12:26.332449] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T18:12:26.567059] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-10. [2025-11-04T18:12:27.138005] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-11-04T18:12:27.367052] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-04. [2025-11-04T18:12:28.167059] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-08. [2025-11-04T18:12:28.961566] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-11-04T18:12:28.972254] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-09. [2025-11-04T18:12:29.023896] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-11-04T18:12:29.484830] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-11-04T18:12:29.767058] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-05. [2025-11-04T18:12:30.567032] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-10. [2025-11-04T18:12:31.204979] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-11-04T18:12:31.237397] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-11-04T18:12:31.367048] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-06. [2025-11-04T18:12:32.167082] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-11. [2025-11-04T18:12:32.443762] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T18:12:32.781448] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T18:12:32.782282] [INFO ] [REMOTE ] CRBD: Full fetch complete (12559 records). [2025-11-04T18:12:32.782351] [DEBUG] [VALIDATOR ] CRBD: Validating primary data. [2025-11-04T18:12:32.967481] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-09. [2025-11-04T18:12:33.215620] [DEBUG] [VALIDATOR ] CRBD: All expected trading days are present. [2025-11-04T18:12:33.215729] [DEBUG] [VALIDATOR ] CRBD: Primary data validation passed. [2025-11-04T18:12:33.216055] [INFO ] [MAIN ] CRBD: Fetched primary data is valid. [2025-11-04T18:12:33.216096] [DEBUG] [LOCAL ] CRBD: Saving primary data. [2025-11-04T18:12:33.216595] [DEBUG] [LOCAL ] CRBD: Trimmed 12559 records to 8542 (keeping 200 days of data) [2025-11-04T18:12:33.274354] [DEBUG] [LOCAL ] CRBD: Primary data saved (8542 records). [2025-11-04T18:12:33.274508] [INFO ] [MAIN ] CRBD: Full history saved. [2025-11-04T18:12:33.274691] [INFO ] [MAIN ] Update Progress: 525/2367 symbols completed. CRBD took 28.3s. Estimated remaining: 4h 14m 22s. Rate: 7.2 symbols/min [2025-11-04T18:12:33.480439] [INFO ] [MAIN ] CRC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:33.521842] [DEBUG] [LOCAL ] CRC: Loaded 43527 existing records from primary storage. [2025-11-04T18:12:33.522196] [INFO ] [MAIN ] CRC: Loaded 43527 existing records. [2025-11-04T18:12:33.723580] [WARN ] [MAIN ] CRC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:12:33.723843] [INFO ] [MAIN ] CRC: Wiping existing primary data before full refresh. [2025-11-04T18:12:33.723892] [INFO ] [LOCAL ] CRC: Starting deletion of primary data. [2025-11-04T18:12:33.724092] [DEBUG] [LOCAL ] CRC: Deleted primary data file. [2025-11-04T18:12:33.724147] [INFO ] [MAIN ] CRC: Starting full primary refresh. [2025-11-04T18:12:33.779225] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-11. [2025-11-04T18:12:34.112893] [INFO ] [REMOTE ] CRC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:34.127419] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T18:12:34.489411] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T18:12:34.490597] [INFO ] [REMOTE ] CQP: Full fetch complete (29194 records). [2025-11-04T18:12:34.490708] [DEBUG] [VALIDATOR ] CQP: Validating primary data. [2025-11-04T18:12:34.572228] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-01. [2025-11-04T18:12:34.856712] [DEBUG] [VALIDATOR ] CQP: All expected trading days are present. [2025-11-04T18:12:34.856830] [DEBUG] [VALIDATOR ] CQP: Primary data validation passed. [2025-11-04T18:12:34.857363] [INFO ] [MAIN ] CQP: Fetched primary data is valid. [2025-11-04T18:12:34.857406] [DEBUG] [LOCAL ] CQP: Saving primary data. [2025-11-04T18:12:34.858211] [DEBUG] [LOCAL ] CQP: Trimmed 29194 records to 18905 (keeping 200 days of data) [2025-11-04T18:12:34.968969] [DEBUG] [LOCAL ] CQP: Primary data saved (18905 records). [2025-11-04T18:12:34.969100] [INFO ] [MAIN ] CQP: Full history saved. [2025-11-04T18:12:34.969178] [INFO ] [MAIN ] Update Progress: 526/2367 symbols completed. CQP took 33.5s. Estimated remaining: 4h 13m 51s. Rate: 7.3 symbols/min [2025-11-04T18:12:35.140330] [INFO ] [MAIN ] CRCL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:35.140506] [DEBUG] [LOCAL ] CRCL: No existing data file found in primary storage. [2025-11-04T18:12:35.140543] [WARN ] [MAIN ] CRCL: No existing data found, performing full refresh instead. [2025-11-04T18:12:35.140671] [INFO ] [MAIN ] CRCL: Wiping existing primary data before full refresh. [2025-11-04T18:12:35.140726] [INFO ] [LOCAL ] CRCL: Starting deletion of primary data. [2025-11-04T18:12:35.140771] [DEBUG] [LOCAL ] CRCL: No primary data file found to delete. [2025-11-04T18:12:35.140802] [INFO ] [MAIN ] CRCL: Starting full primary refresh. [2025-11-04T18:12:35.375109] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-10. [2025-11-04T18:12:35.519973] [INFO ] [REMOTE ] CRCL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:35.682751] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-11-04T18:12:36.167047] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-02. [2025-11-04T18:12:36.378347] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-11-04T18:12:36.967040] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-01. [2025-11-04T18:12:37.134512] [DEBUG] [REMOTE ] CRCL: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:12:37.143617] [WARN ] [REMOTE ] CRCL: No data returned for month 2025-01 [2025-11-04T18:12:37.144817] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-11-04T18:12:37.144903] [FATAL] [REMOTE ] CRCL: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:12:37.152915] [FATAL] [MAIN ] CRCL: Failed to fetch primary data. [2025-11-04T18:12:37.152993] [WARN ] [MAIN ] CRCL: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:12:37.153078] [INFO ] [LOCAL ] CRCL: Starting deletion of primary data. [2025-11-04T18:12:37.153160] [DEBUG] [LOCAL ] CRCL: No primary data file found to delete. [2025-11-04T18:12:37.153224] [INFO ] [LOCAL ] CRCL: Starting deletion of derivative data. [2025-11-04T18:12:37.153273] [DEBUG] [LOCAL ] CRCL: No derivative data file found to delete. [2025-11-04T18:12:37.153319] [INFO ] [MAIN ] Update Progress: 527/2367 symbols completed. CRCL took 2.2s. Estimated remaining: 4h 13m 21s. Rate: 7.3 symbols/min [2025-11-04T18:12:37.319263] [INFO ] [MAIN ] CRGY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:37.367245] [DEBUG] [LOCAL ] CRGY: Loaded 52482 existing records from primary storage. [2025-11-04T18:12:37.367577] [INFO ] [MAIN ] CRGY: Loaded 52482 existing records. [2025-11-04T18:12:37.589203] [WARN ] [MAIN ] CRGY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:12:37.589402] [INFO ] [MAIN ] CRGY: Wiping existing primary data before full refresh. [2025-11-04T18:12:37.589451] [INFO ] [LOCAL ] CRGY: Starting deletion of primary data. [2025-11-04T18:12:37.589778] [DEBUG] [LOCAL ] CRGY: Deleted primary data file. [2025-11-04T18:12:37.589835] [INFO ] [MAIN ] CRGY: Starting full primary refresh. [2025-11-04T18:12:37.773584] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-07. [2025-11-04T18:12:38.020155] [INFO ] [REMOTE ] CRGY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:38.567079] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-11. [2025-11-04T18:12:39.234440] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T18:12:39.346542] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T18:12:39.347959] [INFO ] [REMOTE ] CR: Full fetch complete (35325 records). [2025-11-04T18:12:39.348055] [DEBUG] [VALIDATOR ] CR: Validating primary data. [2025-11-04T18:12:39.376281] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-03. [2025-11-04T18:12:39.782364] [DEBUG] [VALIDATOR ] CR: All expected trading days are present. [2025-11-04T18:12:39.782438] [DEBUG] [VALIDATOR ] CR: Primary data validation passed. [2025-11-04T18:12:39.782948] [INFO ] [MAIN ] CR: Fetched primary data is valid. [2025-11-04T18:12:39.783008] [DEBUG] [LOCAL ] CR: Saving primary data. [2025-11-04T18:12:39.783790] [DEBUG] [LOCAL ] CR: Trimmed 35325 records to 23963 (keeping 200 days of data) [2025-11-04T18:12:39.928232] [DEBUG] [LOCAL ] CR: Primary data saved (23963 records). [2025-11-04T18:12:39.928357] [INFO ] [MAIN ] CR: Full history saved. [2025-11-04T18:12:39.928435] [INFO ] [MAIN ] Update Progress: 528/2367 symbols completed. CR took 35.8s. Estimated remaining: 4h 12m 54s. Rate: 7.3 symbols/min [2025-11-04T18:12:40.091101] [INFO ] [MAIN ] CRH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:40.138469] [DEBUG] [LOCAL ] CRH: Loaded 53378 existing records from primary storage. [2025-11-04T18:12:40.138794] [INFO ] [MAIN ] CRH: Loaded 53378 existing records. [2025-11-04T18:12:40.172251] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-01. [2025-11-04T18:12:40.313473] [WARN ] [MAIN ] CRH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:12:40.313721] [INFO ] [MAIN ] CRH: Wiping existing primary data before full refresh. [2025-11-04T18:12:40.313774] [INFO ] [LOCAL ] CRH: Starting deletion of primary data. [2025-11-04T18:12:40.313950] [DEBUG] [LOCAL ] CRH: Deleted primary data file. [2025-11-04T18:12:40.313987] [INFO ] [MAIN ] CRH: Starting full primary refresh. [2025-11-04T18:12:40.670786] [INFO ] [REMOTE ] CRH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:40.701060] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T18:12:40.967035] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-08. [2025-11-04T18:12:41.604495] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T18:12:41.767104] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-04. [2025-11-04T18:12:42.363237] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T18:12:42.567043] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-02. [2025-11-04T18:12:43.006760] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T18:12:43.367037] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-09. [2025-11-04T18:12:43.879797] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T18:12:44.167050] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-05. [2025-11-04T18:12:44.723698] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T18:12:44.967079] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-10. [2025-11-04T18:12:45.267582] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-11-04T18:12:45.767066] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-06. [2025-11-04T18:12:46.438591] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T18:12:46.567048] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-03. [2025-11-04T18:12:46.831355] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-11-04T18:12:47.367008] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-11. [2025-11-04T18:12:47.787673] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T18:12:48.167045] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-01. [2025-11-04T18:12:48.321155] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-11 (0.95 seconds). [2025-11-04T18:12:48.323627] [INFO ] [REMOTE ] CRBG: Full fetch complete (76036 records). [2025-11-04T18:12:48.323810] [DEBUG] [VALIDATOR ] CRBG: Validating primary data. [2025-11-04T18:12:48.749870] [DEBUG] [VALIDATOR ] CRBG: All expected trading days are present. [2025-11-04T18:12:48.749969] [DEBUG] [VALIDATOR ] CRBG: Primary data validation passed. [2025-11-04T18:12:48.750844] [INFO ] [MAIN ] CRBG: Fetched primary data is valid. [2025-11-04T18:12:48.750905] [DEBUG] [LOCAL ] CRBG: Saving primary data. [2025-11-04T18:12:48.752318] [DEBUG] [LOCAL ] CRBG: Trimmed 76036 records to 52556 (keeping 200 days of data) [2025-11-04T18:12:48.967148] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-04. [2025-11-04T18:12:49.064774] [DEBUG] [LOCAL ] CRBG: Primary data saved (52556 records). [2025-11-04T18:12:49.064909] [INFO ] [MAIN ] CRBG: Full history saved. [2025-11-04T18:12:49.064987] [INFO ] [MAIN ] Update Progress: 529/2367 symbols completed. CRBG took 42.2s. Estimated remaining: 4h 12m 49s. Rate: 7.3 symbols/min [2025-11-04T18:12:49.231165] [INFO ] [MAIN ] CRI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:49.273096] [DEBUG] [LOCAL ] CRI: Loaded 49372 existing records from primary storage. [2025-11-04T18:12:49.273415] [INFO ] [MAIN ] CRI: Loaded 49372 existing records. [2025-11-04T18:12:49.462458] [WARN ] [MAIN ] CRI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:12:49.462756] [INFO ] [MAIN ] CRI: Wiping existing primary data before full refresh. [2025-11-04T18:12:49.462808] [INFO ] [LOCAL ] CRI: Starting deletion of primary data. [2025-11-04T18:12:49.463022] [DEBUG] [LOCAL ] CRI: Deleted primary data file. [2025-11-04T18:12:49.463076] [INFO ] [MAIN ] CRI: Starting full primary refresh. [2025-11-04T18:12:49.791290] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-07. [2025-11-04T18:12:49.821638] [INFO ] [REMOTE ] CRI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:49.853375] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-11-04T18:12:50.351982] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T18:12:50.567055] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-05. [2025-11-04T18:12:50.808891] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T18:12:51.367040] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-08. [2025-11-04T18:12:51.848850] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T18:12:52.167039] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-01. [2025-11-04T18:12:52.313041] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T18:12:52.967044] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-09. [2025-11-04T18:12:53.421746] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T18:12:53.767070] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-06. [2025-11-04T18:12:54.023089] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T18:12:54.567044] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-02. [2025-11-04T18:12:54.944056] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T18:12:55.367012] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-07. [2025-11-04T18:12:55.743871] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T18:12:56.167079] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-10. [2025-11-04T18:12:56.640064] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T18:12:56.967088] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-08. [2025-11-04T18:12:57.346024] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T18:12:57.767022] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-11. [2025-11-04T18:12:58.493832] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T18:12:58.567028] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-03. [2025-11-04T18:12:58.820138] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T18:12:58.821439] [INFO ] [REMOTE ] CRC: Full fetch complete (62283 records). [2025-11-04T18:12:58.821531] [DEBUG] [VALIDATOR ] CRC: Validating primary data. [2025-11-04T18:12:58.822244] [FATAL] [VALIDATOR ] CRC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:12:58.822330] [FATAL] [MAIN ] CRC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:12:58.822375] [WARN ] [MAIN ] CRC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:12:58.822420] [INFO ] [LOCAL ] CRC: Starting deletion of primary data. [2025-11-04T18:12:58.822474] [DEBUG] [LOCAL ] CRC: No primary data file found to delete. [2025-11-04T18:12:58.822513] [INFO ] [LOCAL ] CRC: Starting deletion of derivative data. [2025-11-04T18:12:58.822751] [DEBUG] [LOCAL ] CRC: Deleted derivative data file. [2025-11-04T18:12:58.822855] [INFO ] [MAIN ] Update Progress: 530/2367 symbols completed. CRC took 25.5s. Estimated remaining: 4h 12m 46s. Rate: 7.3 symbols/min [2025-11-04T18:12:58.997322] [INFO ] [MAIN ] CRK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:12:59.045528] [DEBUG] [LOCAL ] CRK: Loaded 52155 existing records from primary storage. [2025-11-04T18:12:59.046095] [INFO ] [MAIN ] CRK: Loaded 52155 existing records. [2025-11-04T18:12:59.226709] [WARN ] [MAIN ] CRK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:12:59.227000] [INFO ] [MAIN ] CRK: Wiping existing primary data before full refresh. [2025-11-04T18:12:59.227068] [INFO ] [LOCAL ] CRK: Starting deletion of primary data. [2025-11-04T18:12:59.227315] [DEBUG] [LOCAL ] CRK: Deleted primary data file. [2025-11-04T18:12:59.227391] [INFO ] [MAIN ] CRK: Starting full primary refresh. [2025-11-04T18:12:59.372198] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-09. [2025-11-04T18:12:59.593485] [INFO ] [REMOTE ] CRK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:12:59.961460] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T18:13:00.167059] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-01. [2025-11-04T18:13:00.967006] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-02. [2025-11-04T18:13:01.166808] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-11-04T18:13:01.767088] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-10. [2025-11-04T18:13:02.038720] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-11-04T18:13:02.567042] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-04. [2025-11-04T18:13:03.265852] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-02 (2.30 seconds). [2025-11-04T18:13:03.367089] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-02. [2025-11-04T18:13:03.814698] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-11-04T18:13:04.021357] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T18:13:04.167063] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-11. [2025-11-04T18:13:04.639685] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T18:13:04.967046] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-03. [2025-11-04T18:13:05.661956] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-11 (1.49 seconds). [2025-11-04T18:13:05.663989] [INFO ] [REMOTE ] CRGY: Full fetch complete (76176 records). [2025-11-04T18:13:05.664183] [DEBUG] [VALIDATOR ] CRGY: Validating primary data. [2025-11-04T18:13:05.772349] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-03. [2025-11-04T18:13:06.112555] [DEBUG] [VALIDATOR ] CRGY: All expected trading days are present. [2025-11-04T18:13:06.112707] [DEBUG] [VALIDATOR ] CRGY: Primary data validation passed. [2025-11-04T18:13:06.113548] [INFO ] [MAIN ] CRGY: Fetched primary data is valid. [2025-11-04T18:13:06.113594] [DEBUG] [LOCAL ] CRGY: Saving primary data. [2025-11-04T18:13:06.115072] [DEBUG] [LOCAL ] CRGY: Trimmed 76176 records to 52486 (keeping 200 days of data) [2025-11-04T18:13:06.384886] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T18:13:06.475814] [DEBUG] [LOCAL ] CRGY: Primary data saved (52486 records). [2025-11-04T18:13:06.475959] [INFO ] [MAIN ] CRGY: Full history saved. [2025-11-04T18:13:06.476746] [INFO ] [MAIN ] Update Progress: 531/2367 symbols completed. CRGY took 29.3s. Estimated remaining: 4h 12m 35s. Rate: 7.3 symbols/min [2025-11-04T18:13:06.574151] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-04. [2025-11-04T18:13:06.668159] [INFO ] [MAIN ] CRL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:13:06.718013] [DEBUG] [LOCAL ] CRL: Loaded 42190 existing records from primary storage. [2025-11-04T18:13:06.718346] [INFO ] [MAIN ] CRL: Loaded 42190 existing records. [2025-11-04T18:13:06.946072] [WARN ] [MAIN ] CRL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:13:06.946307] [INFO ] [MAIN ] CRL: Wiping existing primary data before full refresh. [2025-11-04T18:13:06.946354] [INFO ] [LOCAL ] CRL: Starting deletion of primary data. [2025-11-04T18:13:06.946718] [DEBUG] [LOCAL ] CRL: Deleted primary data file. [2025-11-04T18:13:06.946941] [INFO ] [MAIN ] CRL: Starting full primary refresh. [2025-11-04T18:13:07.301828] [INFO ] [REMOTE ] CRL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:13:07.338759] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-11-04T18:13:07.367010] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-01. [2025-11-04T18:13:07.854514] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T18:13:08.167125] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-04. [2025-11-04T18:13:08.637248] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T18:13:08.967118] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-05. [2025-11-04T18:13:09.581729] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-11-04T18:13:09.767064] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-05. [2025-11-04T18:13:10.276275] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T18:13:10.567122] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-02. [2025-11-04T18:13:11.367031] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-06. [2025-11-04T18:13:11.430489] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-11-04T18:13:11.590149] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T18:13:12.167058] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-06. [2025-11-04T18:13:12.577855] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T18:13:12.967022] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-07. [2025-11-04T18:13:13.761433] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-11-04T18:13:13.771718] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-03. [2025-11-04T18:13:14.132902] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T18:13:14.567064] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-05. [2025-11-04T18:13:14.998399] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T18:13:15.367109] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-08. [2025-11-04T18:13:15.780402] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T18:13:16.167059] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-04. [2025-11-04T18:13:16.634782] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T18:13:16.967262] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-06. [2025-11-04T18:13:17.355238] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T18:13:17.767038] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-07. [2025-11-04T18:13:18.223992] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-11-04T18:13:18.567057] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-09. [2025-11-04T18:13:19.129590] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T18:13:19.367095] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-05. [2025-11-04T18:13:19.768310] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T18:13:20.167083] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-08. [2025-11-04T18:13:20.796540] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T18:13:20.967051] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-06. [2025-11-04T18:13:21.616740] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T18:13:21.767056] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-09. [2025-11-04T18:13:21.995316] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T18:13:22.567062] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-07. [2025-11-04T18:13:23.212774] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-11-04T18:13:23.367040] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-10. [2025-11-04T18:13:23.715434] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-11-04T18:13:24.167071] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-07. [2025-11-04T18:13:24.953421] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T18:13:24.967134] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-08. [2025-11-04T18:13:25.305387] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T18:13:25.767086] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-08. [2025-11-04T18:13:25.951753] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-11-04T18:13:26.567105] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-09. [2025-11-04T18:13:27.069559] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T18:13:27.367076] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-11. [2025-11-04T18:13:27.693767] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-11-04T18:13:28.167090] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-09. [2025-11-04T18:13:28.482150] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T18:13:28.484156] [INFO ] [REMOTE ] CRH: Full fetch complete (77513 records). [2025-11-04T18:13:28.484271] [DEBUG] [VALIDATOR ] CRH: Validating primary data. [2025-11-04T18:13:28.877737] [DEBUG] [VALIDATOR ] CRH: All expected trading days are present. [2025-11-04T18:13:28.877829] [DEBUG] [VALIDATOR ] CRH: Primary data validation passed. [2025-11-04T18:13:28.878686] [INFO ] [MAIN ] CRH: Fetched primary data is valid. [2025-11-04T18:13:28.878750] [DEBUG] [LOCAL ] CRH: Saving primary data. [2025-11-04T18:13:28.880174] [DEBUG] [LOCAL ] CRH: Trimmed 77513 records to 53377 (keeping 200 days of data) [2025-11-04T18:13:28.968878] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-10. [2025-11-04T18:13:29.203785] [DEBUG] [LOCAL ] CRH: Primary data saved (53377 records). [2025-11-04T18:13:29.203916] [INFO ] [MAIN ] CRH: Full history saved. [2025-11-04T18:13:29.203993] [INFO ] [MAIN ] Update Progress: 532/2367 symbols completed. CRH took 49.3s. Estimated remaining: 4h 13m 17s. Rate: 7.2 symbols/min [2025-11-04T18:13:29.381638] [INFO ] [MAIN ] CRM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:13:29.429300] [DEBUG] [LOCAL ] CRM: Loaded 53770 existing records from primary storage. [2025-11-04T18:13:29.429602] [INFO ] [MAIN ] CRM: Loaded 53770 existing records. [2025-11-04T18:13:29.633356] [WARN ] [MAIN ] CRM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:13:29.648415] [INFO ] [MAIN ] CRM: Wiping existing primary data before full refresh. [2025-11-04T18:13:29.648588] [INFO ] [LOCAL ] CRM: Starting deletion of primary data. [2025-11-04T18:13:29.650896] [DEBUG] [LOCAL ] CRM: Deleted primary data file. [2025-11-04T18:13:29.650823] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-11-04T18:13:29.651127] [INFO ] [MAIN ] CRM: Starting full primary refresh. [2025-11-04T18:13:29.776352] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-10. [2025-11-04T18:13:30.031585] [INFO ] [REMOTE ] CRM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:13:30.205980] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-11-04T18:13:30.567056] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-10. [2025-11-04T18:13:31.114692] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-11-04T18:13:31.367088] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-11. [2025-11-04T18:13:32.152189] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T18:13:32.167036] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-11. [2025-11-04T18:13:32.511691] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T18:13:32.514355] [INFO ] [REMOTE ] CRK: Full fetch complete (75735 records). [2025-11-04T18:13:32.514489] [DEBUG] [VALIDATOR ] CRK: Validating primary data. [2025-11-04T18:13:32.929723] [DEBUG] [VALIDATOR ] CRK: All expected trading days are present. [2025-11-04T18:13:32.929833] [DEBUG] [VALIDATOR ] CRK: Primary data validation passed. [2025-11-04T18:13:32.930704] [INFO ] [MAIN ] CRK: Fetched primary data is valid. [2025-11-04T18:13:32.930767] [DEBUG] [LOCAL ] CRK: Saving primary data. [2025-11-04T18:13:32.932205] [DEBUG] [LOCAL ] CRK: Trimmed 75735 records to 52162 (keeping 200 days of data) [2025-11-04T18:13:32.979334] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-11. [2025-11-04T18:13:33.166468] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T18:13:33.199907] [INFO ] [REMOTE ] CRI: Full fetch complete (71013 records). [2025-11-04T18:13:33.205472] [DEBUG] [VALIDATOR ] CRI: Validating primary data. [2025-11-04T18:13:33.554092] [DEBUG] [LOCAL ] CRK: Primary data saved (52162 records). [2025-11-04T18:13:33.564480] [INFO ] [MAIN ] CRK: Full history saved. [2025-11-04T18:13:33.574882] [INFO ] [MAIN ] Update Progress: 533/2367 symbols completed. CRK took 34.8s. Estimated remaining: 4h 12m 55s. Rate: 7.3 symbols/min [2025-11-04T18:13:33.754769] [DEBUG] [VALIDATOR ] CRI: All expected trading days are present. [2025-11-04T18:13:33.765741] [DEBUG] [VALIDATOR ] CRI: Primary data validation passed. [2025-11-04T18:13:33.772170] [INFO ] [MAIN ] CRI: Fetched primary data is valid. [2025-11-04T18:13:33.770926] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-01. [2025-11-04T18:13:33.783391] [DEBUG] [LOCAL ] CRI: Saving primary data. [2025-11-04T18:13:33.837307] [INFO ] [MAIN ] CRS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:13:33.837458] [DEBUG] [LOCAL ] CRI: Trimmed 71013 records to 49304 (keeping 200 days of data) [2025-11-04T18:13:34.053264] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T18:13:34.075924] [INFO ] [REMOTE ] CRL: Full fetch complete (63678 records). [2025-11-04T18:13:34.079537] [DEBUG] [VALIDATOR ] CRL: Validating primary data. [2025-11-04T18:13:34.214055] [DEBUG] [LOCAL ] CRS: Loaded 42922 existing records from primary storage. [2025-11-04T18:13:34.334733] [INFO ] [MAIN ] CRS: Loaded 42922 existing records. [2025-11-04T18:13:34.546559] [DEBUG] [LOCAL ] CRI: Primary data saved (49304 records). [2025-11-04T18:13:34.583204] [INFO ] [MAIN ] CRI: Full history saved. [2025-11-04T18:13:34.588939] [INFO ] [MAIN ] Update Progress: 534/2367 symbols completed. CRI took 45.5s. Estimated remaining: 4h 12m 22s. Rate: 7.3 symbols/min [2025-11-04T18:13:34.830491] [DEBUG] [VALIDATOR ] CRL: All expected trading days are present. [2025-11-04T18:13:34.838694] [WARN ] [MAIN ] CRS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:13:34.844018] [DEBUG] [VALIDATOR ] CRL: Primary data validation passed. [2025-11-04T18:13:34.851356] [INFO ] [MAIN ] CRS: Wiping existing primary data before full refresh. [2025-11-04T18:13:34.864914] [INFO ] [LOCAL ] CRS: Starting deletion of primary data. [2025-11-04T18:13:34.864735] [INFO ] [MAIN ] CRL: Fetched primary data is valid. [2025-11-04T18:13:34.865102] [DEBUG] [LOCAL ] CRL: Saving primary data. [2025-11-04T18:13:34.865500] [DEBUG] [LOCAL ] CRS: Deleted primary data file. [2025-11-04T18:13:34.865727] [INFO ] [MAIN ] CRS: Starting full primary refresh. [2025-11-04T18:13:34.967790] [INFO ] [MAIN ] CRT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:13:35.040315] [DEBUG] [LOCAL ] CRL: Trimmed 63678 records to 42146 (keeping 200 days of data) [2025-11-04T18:13:35.138869] [DEBUG] [LOCAL ] CRT: Loaded 4854 existing records from primary storage. [2025-11-04T18:13:35.216038] [INFO ] [MAIN ] CRT: Loaded 4854 existing records. [2025-11-04T18:13:35.591198] [WARN ] [MAIN ] CRT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:13:35.622668] [INFO ] [MAIN ] CRT: Wiping existing primary data before full refresh. [2025-11-04T18:13:35.622389] [INFO ] [REMOTE ] CRS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:13:35.623803] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-01. [2025-11-04T18:13:35.623172] [INFO ] [LOCAL ] CRT: Starting deletion of primary data. [2025-11-04T18:13:35.627255] [DEBUG] [LOCAL ] CRT: Deleted primary data file. [2025-11-04T18:13:35.627611] [INFO ] [MAIN ] CRT: Starting full primary refresh. [2025-11-04T18:13:36.003130] [DEBUG] [LOCAL ] CRL: Primary data saved (42146 records). [2025-11-04T18:13:36.053878] [INFO ] [MAIN ] CRL: Full history saved. [2025-11-04T18:13:36.067422] [INFO ] [MAIN ] Update Progress: 535/2367 symbols completed. CRL took 29.6s. Estimated remaining: 4h 11m 51s. Rate: 7.3 symbols/min [2025-11-04T18:13:36.265218] [INFO ] [REMOTE ] CRT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:13:36.291337] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-01. [2025-11-04T18:13:36.339258] [INFO ] [MAIN ] CSAN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:13:36.369762] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-01 (2.60 seconds). [2025-11-04T18:13:36.380523] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-02. [2025-11-04T18:13:36.412961] [DEBUG] [LOCAL ] CSAN: Loaded 41990 existing records from primary storage. [2025-11-04T18:13:36.413246] [INFO ] [MAIN ] CSAN: Loaded 41990 existing records. [2025-11-04T18:13:36.599741] [WARN ] [MAIN ] CSAN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:13:36.599967] [INFO ] [MAIN ] CSAN: Wiping existing primary data before full refresh. [2025-11-04T18:13:36.600014] [INFO ] [LOCAL ] CSAN: Starting deletion of primary data. [2025-11-04T18:13:36.600209] [DEBUG] [LOCAL ] CSAN: Deleted primary data file. [2025-11-04T18:13:36.600265] [INFO ] [MAIN ] CSAN: Starting full primary refresh. [2025-11-04T18:13:36.948593] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-11-04T18:13:36.967502] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-02. [2025-11-04T18:13:36.995320] [INFO ] [REMOTE ] CSAN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:13:37.021619] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T18:13:37.468842] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-11-04T18:13:37.767042] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-01. [2025-11-04T18:13:38.401600] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-11-04T18:13:38.567036] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-03. [2025-11-04T18:13:38.976070] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T18:13:39.367031] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-02. [2025-11-04T18:13:40.167058] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-03. [2025-11-04T18:13:40.277215] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-11-04T18:13:40.360110] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T18:13:40.561376] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-11-04T18:13:40.967048] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-03. [2025-11-04T18:13:41.767021] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-04. [2025-11-04T18:13:41.873550] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T18:13:42.567087] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-04. [2025-11-04T18:13:43.011402] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-11-04T18:13:43.367002] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-02. [2025-11-04T18:13:43.466808] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-11-04T18:13:44.167034] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-05. [2025-11-04T18:13:44.493758] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T18:13:44.616468] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-11-04T18:13:44.967037] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-06. [2025-11-04T18:13:45.375303] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-11-04T18:13:45.767044] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-05. [2025-11-04T18:13:46.567048] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-07. [2025-11-04T18:13:47.071303] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-11-04T18:13:47.367065] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-08. [2025-11-04T18:13:47.658744] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-11-04T18:13:47.774964] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-11-04T18:13:48.167076] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-03. [2025-11-04T18:13:48.967040] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-06. [2025-11-04T18:13:49.409126] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T18:13:49.767068] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-04. [2025-11-04T18:13:50.567072] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-09. [2025-11-04T18:13:50.790592] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-11-04T18:13:51.025357] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T18:13:51.046631] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-11-04T18:13:51.367080] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-04. [2025-11-04T18:13:52.167018] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-07. [2025-11-04T18:13:52.601192] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-11-04T18:13:52.967090] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-05. [2025-11-04T18:13:53.767090] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-05. [2025-11-04T18:13:54.108899] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-11-04T18:13:54.236827] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-11-04T18:13:54.567042] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-08. [2025-11-04T18:13:54.915943] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T18:13:55.367034] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-06. [2025-11-04T18:13:56.167081] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-06. [2025-11-04T18:13:56.579053] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T18:13:56.620383] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-11-04T18:13:56.967063] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-07. [2025-11-04T18:13:57.332961] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T18:13:57.767073] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-09. [2025-11-04T18:13:58.161854] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-11-04T18:13:58.567054] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-08. [2025-11-04T18:13:59.367039] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-10. [2025-11-04T18:13:59.820561] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T18:13:59.870540] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-11-04T18:13:59.888377] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-11-04T18:14:00.167052] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-09. [2025-11-04T18:14:00.967071] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-11. [2025-11-04T18:14:01.347783] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:14:01.532876] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T18:14:01.533695] [INFO ] [REMOTE ] CRT: Full fetch complete (7476 records). [2025-11-04T18:14:01.533766] [DEBUG] [VALIDATOR ] CRT: Validating primary data. [2025-11-04T18:14:01.772280] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-10. [2025-11-04T18:14:01.961561] [DEBUG] [VALIDATOR ] CRT: All expected trading days are present. [2025-11-04T18:14:01.961637] [DEBUG] [VALIDATOR ] CRT: Primary data validation passed. [2025-11-04T18:14:01.961902] [INFO ] [MAIN ] CRT: Fetched primary data is valid. [2025-11-04T18:14:01.961947] [DEBUG] [LOCAL ] CRT: Saving primary data. [2025-11-04T18:14:01.962347] [DEBUG] [LOCAL ] CRT: Trimmed 7476 records to 4875 (keeping 200 days of data) [2025-11-04T18:14:01.991065] [DEBUG] [LOCAL ] CRT: Primary data saved (4875 records). [2025-11-04T18:14:01.991185] [INFO ] [MAIN ] CRT: Full history saved. [2025-11-04T18:14:01.991268] [INFO ] [MAIN ] Update Progress: 536/2367 symbols completed. CRT took 27.4s. Estimated remaining: 4h 12m 43s. Rate: 7.2 symbols/min [2025-11-04T18:14:02.166166] [INFO ] [MAIN ] CSL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:14:02.197007] [DEBUG] [LOCAL ] CSL: Loaded 30961 existing records from primary storage. [2025-11-04T18:14:02.197274] [INFO ] [MAIN ] CSL: Loaded 30961 existing records. [2025-11-04T18:14:02.376607] [WARN ] [MAIN ] CSL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:14:02.376897] [INFO ] [MAIN ] CSL: Wiping existing primary data before full refresh. [2025-11-04T18:14:02.376950] [INFO ] [LOCAL ] CSL: Starting deletion of primary data. [2025-11-04T18:14:02.377165] [DEBUG] [LOCAL ] CSL: Deleted primary data file. [2025-11-04T18:14:02.377221] [INFO ] [MAIN ] CSL: Starting full primary refresh. [2025-11-04T18:14:02.572211] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-07. [2025-11-04T18:14:02.740500] [INFO ] [REMOTE ] CSL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:14:03.367038] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-01. [2025-11-04T18:14:04.077531] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T18:14:04.167020] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-08. [2025-11-04T18:14:04.274864] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-10 (2.50 seconds). [2025-11-04T18:14:04.476572] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-11-04T18:14:04.967067] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-11. [2025-11-04T18:14:05.091697] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-11-04T18:14:05.330890] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T18:14:05.332984] [INFO ] [REMOTE ] CRM: Full fetch complete (77947 records). [2025-11-04T18:14:05.333086] [DEBUG] [VALIDATOR ] CRM: Validating primary data. [2025-11-04T18:14:05.719314] [DEBUG] [VALIDATOR ] CRM: All expected trading days are present. [2025-11-04T18:14:05.719395] [DEBUG] [VALIDATOR ] CRM: Primary data validation passed. [2025-11-04T18:14:05.720234] [INFO ] [MAIN ] CRM: Fetched primary data is valid. [2025-11-04T18:14:05.720297] [DEBUG] [LOCAL ] CRM: Saving primary data. [2025-11-04T18:14:05.721705] [DEBUG] [LOCAL ] CRM: Trimmed 77947 records to 53770 (keeping 200 days of data) [2025-11-04T18:14:05.776439] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-10. [2025-11-04T18:14:06.059481] [DEBUG] [LOCAL ] CRM: Primary data saved (53770 records). [2025-11-04T18:14:06.059613] [INFO ] [MAIN ] CRM: Full history saved. [2025-11-04T18:14:06.059728] [INFO ] [MAIN ] Update Progress: 537/2367 symbols completed. CRM took 36.9s. Estimated remaining: 4h 12m 20s. Rate: 7.3 symbols/min [2025-11-04T18:14:06.225749] [INFO ] [MAIN ] CSR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:14:06.225925] [DEBUG] [LOCAL ] CSR: No existing data file found in primary storage. [2025-11-04T18:14:06.225963] [WARN ] [MAIN ] CSR: No existing data found, performing full refresh instead. [2025-11-04T18:14:06.226071] [INFO ] [MAIN ] CSR: Wiping existing primary data before full refresh. [2025-11-04T18:14:06.226111] [INFO ] [LOCAL ] CSR: Starting deletion of primary data. [2025-11-04T18:14:06.226151] [DEBUG] [LOCAL ] CSR: No primary data file found to delete. [2025-11-04T18:14:06.226181] [INFO ] [MAIN ] CSR: Starting full primary refresh. [2025-11-04T18:14:06.567074] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-02. [2025-11-04T18:14:06.570677] [INFO ] [REMOTE ] CSR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:14:07.125277] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T18:14:07.367023] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-09. [2025-11-04T18:14:07.627830] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T18:14:08.167058] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-01. [2025-11-04T18:14:08.460022] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-11-04T18:14:08.816742] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-11-04T18:14:08.967026] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-10. [2025-11-04T18:14:09.767073] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-02. [2025-11-04T18:14:10.131960] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T18:14:10.283305] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-11-04T18:14:10.567041] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-03. [2025-11-04T18:14:11.367068] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-03. [2025-11-04T18:14:11.560758] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T18:14:11.840606] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-11-04T18:14:12.167068] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-04. [2025-11-04T18:14:12.842886] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T18:14:12.967032] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-11. [2025-11-04T18:14:13.767076] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-04. [2025-11-04T18:14:14.013928] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T18:14:14.015516] [INFO ] [REMOTE ] CRS: Full fetch complete (62618 records). [2025-11-04T18:14:14.015604] [DEBUG] [VALIDATOR ] CRS: Validating primary data. [2025-11-04T18:14:14.390855] [DEBUG] [VALIDATOR ] CRS: All expected trading days are present. [2025-11-04T18:14:14.390929] [DEBUG] [VALIDATOR ] CRS: Primary data validation passed. [2025-11-04T18:14:14.391609] [INFO ] [MAIN ] CRS: Fetched primary data is valid. [2025-11-04T18:14:14.391670] [DEBUG] [LOCAL ] CRS: Saving primary data. [2025-11-04T18:14:14.392799] [DEBUG] [LOCAL ] CRS: Trimmed 62618 records to 42883 (keeping 200 days of data) [2025-11-04T18:14:14.572707] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-11. [2025-11-04T18:14:14.662970] [DEBUG] [LOCAL ] CRS: Primary data saved (42883 records). [2025-11-04T18:14:14.663106] [INFO ] [MAIN ] CRS: Full history saved. [2025-11-04T18:14:14.663189] [INFO ] [MAIN ] Update Progress: 538/2367 symbols completed. CRS took 41.1s. Estimated remaining: 4h 12m 13s. Rate: 7.3 symbols/min [2025-11-04T18:14:14.831866] [INFO ] [MAIN ] CSTM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:14:14.872226] [DEBUG] [LOCAL ] CSTM: Loaded 47279 existing records from primary storage. [2025-11-04T18:14:14.872499] [INFO ] [MAIN ] CSTM: Loaded 47279 existing records. [2025-11-04T18:14:15.052057] [WARN ] [MAIN ] CSTM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:14:15.052385] [INFO ] [MAIN ] CSTM: Wiping existing primary data before full refresh. [2025-11-04T18:14:15.052455] [INFO ] [LOCAL ] CSTM: Starting deletion of primary data. [2025-11-04T18:14:15.052917] [DEBUG] [LOCAL ] CSTM: Deleted primary data file. [2025-11-04T18:14:15.052981] [INFO ] [MAIN ] CSTM: Starting full primary refresh. [2025-11-04T18:14:15.349774] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-11-04T18:14:15.370865] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-05. [2025-11-04T18:14:15.440586] [INFO ] [REMOTE ] CSTM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:14:15.658230] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T18:14:15.659500] [INFO ] [REMOTE ] CSAN: Full fetch complete (60181 records). [2025-11-04T18:14:15.659577] [DEBUG] [VALIDATOR ] CSAN: Validating primary data. [2025-11-04T18:14:15.660132] [FATAL] [VALIDATOR ] CSAN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:14:15.660212] [FATAL] [MAIN ] CSAN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:14:15.660253] [WARN ] [MAIN ] CSAN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:14:15.660289] [INFO ] [LOCAL ] CSAN: Starting deletion of primary data. [2025-11-04T18:14:15.660378] [DEBUG] [LOCAL ] CSAN: No primary data file found to delete. [2025-11-04T18:14:15.660423] [INFO ] [LOCAL ] CSAN: Starting deletion of derivative data. [2025-11-04T18:14:15.660544] [DEBUG] [LOCAL ] CSAN: Deleted derivative data file. [2025-11-04T18:14:15.660608] [INFO ] [MAIN ] Update Progress: 539/2367 symbols completed. CSAN took 39.6s. Estimated remaining: 4h 11m 40s. Rate: 7.3 symbols/min [2025-11-04T18:14:15.836609] [INFO ] [MAIN ] CSV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:14:15.848712] [DEBUG] [LOCAL ] CSV: Loaded 11709 existing records from primary storage. [2025-11-04T18:14:15.848911] [INFO ] [MAIN ] CSV: Loaded 11709 existing records. [2025-11-04T18:14:16.042990] [WARN ] [MAIN ] CSV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:14:16.050700] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-11-04T18:14:16.050425] [INFO ] [MAIN ] CSV: Wiping existing primary data before full refresh. [2025-11-04T18:14:16.050831] [INFO ] [LOCAL ] CSV: Starting deletion of primary data. [2025-11-04T18:14:16.054199] [DEBUG] [LOCAL ] CSV: Deleted primary data file. [2025-11-04T18:14:16.054258] [INFO ] [MAIN ] CSV: Starting full primary refresh. [2025-11-04T18:14:16.172144] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-01. [2025-11-04T18:14:16.419829] [INFO ] [REMOTE ] CSV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:14:16.967064] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-01. [2025-11-04T18:14:17.427440] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T18:14:17.624598] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-11-04T18:14:17.767046] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-02. [2025-11-04T18:14:18.330867] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-11-04T18:14:18.567112] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-03. [2025-11-04T18:14:19.298782] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-11-04T18:14:19.367082] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-05. [2025-11-04T18:14:20.167052] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-02. [2025-11-04T18:14:20.318809] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-11-04T18:14:20.967049] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-06. [2025-11-04T18:14:21.208351] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-11-04T18:14:21.767029] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-06. [2025-11-04T18:14:21.863941] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-11-04T18:14:22.427496] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-11-04T18:14:22.567081] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-04. [2025-11-04T18:14:23.242143] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T18:14:23.367074] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-07. [2025-11-04T18:14:23.878884] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-11-04T18:14:24.167038] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-08. [2025-11-04T18:14:24.623167] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-11-04T18:14:24.967089] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-07. [2025-11-04T18:14:25.767067] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-09. [2025-11-04T18:14:25.864321] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T18:14:26.359922] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-11-04T18:14:26.567038] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-10. [2025-11-04T18:14:27.270343] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-11-04T18:14:27.367044] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-11. [2025-11-04T18:14:28.037412] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T18:14:28.038430] [INFO ] [REMOTE ] CSR: Full fetch complete (18055 records). [2025-11-04T18:14:28.038499] [DEBUG] [VALIDATOR ] CSR: Validating primary data. [2025-11-04T18:14:28.172197] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-03. [2025-11-04T18:14:28.444353] [DEBUG] [VALIDATOR ] CSR: All expected trading days are present. [2025-11-04T18:14:28.444429] [DEBUG] [VALIDATOR ] CSR: Primary data validation passed. [2025-11-04T18:14:28.444862] [INFO ] [MAIN ] CSR: Fetched primary data is valid. [2025-11-04T18:14:28.444928] [DEBUG] [LOCAL ] CSR: Saving primary data. [2025-11-04T18:14:28.445484] [DEBUG] [LOCAL ] CSR: Trimmed 18055 records to 13244 (keeping 200 days of data) [2025-11-04T18:14:28.526999] [DEBUG] [LOCAL ] CSR: Primary data saved (13244 records). [2025-11-04T18:14:28.527150] [INFO ] [MAIN ] CSR: Full history saved. [2025-11-04T18:14:28.527225] [INFO ] [MAIN ] Update Progress: 540/2367 symbols completed. CSR took 22.5s. Estimated remaining: 4h 11m 47s. Rate: 7.3 symbols/min [2025-11-04T18:14:28.707056] [INFO ] [MAIN ] CSW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:14:28.707253] [DEBUG] [LOCAL ] CSW: No existing data file found in primary storage. [2025-11-04T18:14:28.707289] [WARN ] [MAIN ] CSW: No existing data found, performing full refresh instead. [2025-11-04T18:14:28.707399] [INFO ] [MAIN ] CSW: Wiping existing primary data before full refresh. [2025-11-04T18:14:28.707439] [INFO ] [LOCAL ] CSW: Starting deletion of primary data. [2025-11-04T18:14:28.707500] [DEBUG] [LOCAL ] CSW: No primary data file found to delete. [2025-11-04T18:14:28.707532] [INFO ] [MAIN ] CSW: Starting full primary refresh. [2025-11-04T18:14:28.972257] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-08. [2025-11-04T18:14:29.075824] [INFO ] [REMOTE ] CSW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:14:29.590742] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T18:14:29.767050] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-01. [2025-11-04T18:14:29.942867] [DEBUG] [REMOTE ] CSW: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:14:29.943165] [WARN ] [REMOTE ] CSW: No data returned for month 2025-01 [2025-11-04T18:14:29.943252] [FATAL] [REMOTE ] CSW: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:14:29.943299] [FATAL] [MAIN ] CSW: Failed to fetch primary data. [2025-11-04T18:14:29.943359] [WARN ] [MAIN ] CSW: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:14:29.943426] [INFO ] [LOCAL ] CSW: Starting deletion of primary data. [2025-11-04T18:14:29.943575] [DEBUG] [LOCAL ] CSW: No primary data file found to delete. [2025-11-04T18:14:29.943763] [INFO ] [LOCAL ] CSW: Starting deletion of derivative data. [2025-11-04T18:14:29.943829] [DEBUG] [LOCAL ] CSW: No derivative data file found to delete. [2025-11-04T18:14:29.943929] [INFO ] [MAIN ] Update Progress: 541/2367 symbols completed. CSW took 1.4s. Estimated remaining: 4h 11m 16s. Rate: 7.3 symbols/min [2025-11-04T18:14:30.130210] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T18:14:30.131935] [INFO ] [MAIN ] CTBB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:14:30.146439] [DEBUG] [LOCAL ] CTBB: Loaded 6183 existing records from primary storage. [2025-11-04T18:14:30.146661] [INFO ] [MAIN ] CTBB: Loaded 6183 existing records. [2025-11-04T18:14:30.326377] [WARN ] [MAIN ] CTBB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:14:30.326625] [INFO ] [MAIN ] CTBB: Wiping existing primary data before full refresh. [2025-11-04T18:14:30.326723] [INFO ] [LOCAL ] CTBB: Starting deletion of primary data. [2025-11-04T18:14:30.326951] [DEBUG] [LOCAL ] CTBB: Deleted primary data file. [2025-11-04T18:14:30.327009] [INFO ] [MAIN ] CTBB: Starting full primary refresh. [2025-11-04T18:14:30.572211] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-09. [2025-11-04T18:14:30.765807] [INFO ] [REMOTE ] CTBB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:14:31.367043] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-01. [2025-11-04T18:14:31.751339] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:14:31.968509] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-11-04T18:14:32.167040] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-02. [2025-11-04T18:14:32.653162] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-11-04T18:14:32.967048] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-03. [2025-11-04T18:14:33.389297] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T18:14:33.767048] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-04. [2025-11-04T18:14:34.226753] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-11-04T18:14:34.567040] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-05. [2025-11-04T18:14:34.927438] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-11-04T18:14:35.367041] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-10. [2025-11-04T18:14:36.167061] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-04. [2025-11-04T18:14:36.664225] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-11-04T18:14:36.967057] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-11. [2025-11-04T18:14:37.378702] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T18:14:37.767065] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-06. [2025-11-04T18:14:37.862137] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T18:14:37.863803] [INFO ] [REMOTE ] CSL: Full fetch complete (46241 records). [2025-11-04T18:14:37.863877] [DEBUG] [VALIDATOR ] CSL: Validating primary data. [2025-11-04T18:14:38.248859] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-11-04T18:14:38.249136] [DEBUG] [VALIDATOR ] CSL: All expected trading days are present. [2025-11-04T18:14:38.251686] [DEBUG] [VALIDATOR ] CSL: Primary data validation passed. [2025-11-04T18:14:38.252348] [INFO ] [MAIN ] CSL: Fetched primary data is valid. [2025-11-04T18:14:38.252394] [DEBUG] [LOCAL ] CSL: Saving primary data. [2025-11-04T18:14:38.253387] [DEBUG] [LOCAL ] CSL: Trimmed 46241 records to 31053 (keeping 200 days of data) [2025-11-04T18:14:38.440966] [DEBUG] [LOCAL ] CSL: Primary data saved (31053 records). [2025-11-04T18:14:38.441095] [INFO ] [MAIN ] CSL: Full history saved. [2025-11-04T18:14:38.441172] [INFO ] [MAIN ] Update Progress: 542/2367 symbols completed. CSL took 36.4s. Estimated remaining: 4h 11m 9s. Rate: 7.3 symbols/min [2025-11-04T18:14:38.577094] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-07. [2025-11-04T18:14:38.607746] [INFO ] [MAIN ] CTDD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:14:38.614773] [DEBUG] [LOCAL ] CTDD: Loaded 4933 existing records from primary storage. [2025-11-04T18:14:38.614963] [INFO ] [MAIN ] CTDD: Loaded 4933 existing records. [2025-11-04T18:14:38.787414] [WARN ] [MAIN ] CTDD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:14:38.787620] [INFO ] [MAIN ] CTDD: Wiping existing primary data before full refresh. [2025-11-04T18:14:38.787697] [INFO ] [LOCAL ] CTDD: Starting deletion of primary data. [2025-11-04T18:14:38.787862] [DEBUG] [LOCAL ] CTDD: Deleted primary data file. [2025-11-04T18:14:38.787917] [INFO ] [MAIN ] CTDD: Starting full primary refresh. [2025-11-04T18:14:39.073921] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-11-04T18:14:39.139556] [INFO ] [REMOTE ] CTDD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:14:39.367016] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-01. [2025-11-04T18:14:39.840915] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-01 (0.47 seconds). [2025-11-04T18:14:40.167039] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-05. [2025-11-04T18:14:40.936183] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-11-04T18:14:40.967027] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-06. [2025-11-04T18:14:41.458702] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-11-04T18:14:41.767035] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-02. [2025-11-04T18:14:42.154163] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-11-04T18:14:42.567029] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-05. [2025-11-04T18:14:43.367036] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-08. [2025-11-04T18:14:43.936278] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T18:14:44.022281] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-11-04T18:14:44.167030] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-06. [2025-11-04T18:14:44.967033] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-09. [2025-11-04T18:14:45.248130] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T18:14:45.397214] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-11-04T18:14:45.767036] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-07. [2025-11-04T18:14:46.292668] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-11-04T18:14:46.567042] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-07. [2025-11-04T18:14:47.367079] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-08. [2025-11-04T18:14:47.817114] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:14:47.835907] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-11-04T18:14:48.167085] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-09. [2025-11-04T18:14:48.828225] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-11-04T18:14:48.967002] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-10. [2025-11-04T18:14:49.461593] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-11-04T18:14:49.767033] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-10. [2025-11-04T18:14:50.178515] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-11-04T18:14:50.567065] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-11. [2025-11-04T18:14:50.943001] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T18:14:50.943730] [INFO ] [REMOTE ] CTBB: Full fetch complete (9912 records). [2025-11-04T18:14:50.943798] [DEBUG] [VALIDATOR ] CTBB: Validating primary data. [2025-11-04T18:14:51.294127] [DEBUG] [VALIDATOR ] CTBB: All expected trading days are present. [2025-11-04T18:14:51.294392] [DEBUG] [VALIDATOR ] CTBB: Primary data validation passed. [2025-11-04T18:14:51.294626] [INFO ] [MAIN ] CTBB: Fetched primary data is valid. [2025-11-04T18:14:51.294697] [DEBUG] [LOCAL ] CTBB: Saving primary data. [2025-11-04T18:14:51.295104] [DEBUG] [LOCAL ] CTBB: Trimmed 9912 records to 6170 (keeping 200 days of data) [2025-11-04T18:14:51.333366] [DEBUG] [LOCAL ] CTBB: Primary data saved (6170 records). [2025-11-04T18:14:51.333456] [INFO ] [MAIN ] CTBB: Full history saved. [2025-11-04T18:14:51.333533] [INFO ] [MAIN ] Update Progress: 543/2367 symbols completed. CTBB took 21.4s. Estimated remaining: 4h 11m 16s. Rate: 7.3 symbols/min [2025-11-04T18:14:51.372135] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-03. [2025-11-04T18:14:51.507885] [INFO ] [MAIN ] CTEV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:14:51.508059] [DEBUG] [LOCAL ] CTEV: No existing data file found in primary storage. [2025-11-04T18:14:51.508115] [WARN ] [MAIN ] CTEV: No existing data found, performing full refresh instead. [2025-11-04T18:14:51.508219] [INFO ] [MAIN ] CTEV: Wiping existing primary data before full refresh. [2025-11-04T18:14:51.508259] [INFO ] [LOCAL ] CTEV: Starting deletion of primary data. [2025-11-04T18:14:51.508299] [DEBUG] [LOCAL ] CTEV: No primary data file found to delete. [2025-11-04T18:14:51.508329] [INFO ] [MAIN ] CTEV: Starting full primary refresh. [2025-11-04T18:14:51.867976] [INFO ] [REMOTE ] CTEV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:14:51.937203] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T18:14:52.167034] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-08. [2025-11-04T18:14:52.967096] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-11. [2025-11-04T18:14:53.244024] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-11-04T18:14:53.580101] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T18:14:53.581097] [INFO ] [REMOTE ] CSV: Full fetch complete (18717 records). [2025-11-04T18:14:53.581178] [DEBUG] [VALIDATOR ] CSV: Validating primary data. [2025-11-04T18:14:53.776351] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-04. [2025-11-04T18:14:53.953095] [DEBUG] [VALIDATOR ] CSV: All expected trading days are present. [2025-11-04T18:14:53.953173] [DEBUG] [VALIDATOR ] CSV: Primary data validation passed. [2025-11-04T18:14:53.953542] [INFO ] [MAIN ] CSV: Fetched primary data is valid. [2025-11-04T18:14:53.953581] [DEBUG] [LOCAL ] CSV: Saving primary data. [2025-11-04T18:14:53.954204] [DEBUG] [LOCAL ] CSV: Trimmed 18717 records to 11690 (keeping 200 days of data) [2025-11-04T18:14:54.024417] [DEBUG] [LOCAL ] CSV: Primary data saved (11690 records). [2025-11-04T18:14:54.024558] [INFO ] [MAIN ] CSV: Full history saved. [2025-11-04T18:14:54.024637] [INFO ] [MAIN ] Update Progress: 544/2367 symbols completed. CSV took 38.4s. Estimated remaining: 4h 10m 49s. Rate: 7.3 symbols/min [2025-11-04T18:14:54.192551] [INFO ] [MAIN ] CTO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:14:54.218025] [DEBUG] [LOCAL ] CTO: Loaded 27136 existing records from primary storage. [2025-11-04T18:14:54.218286] [INFO ] [MAIN ] CTO: Loaded 27136 existing records. [2025-11-04T18:14:54.402141] [WARN ] [MAIN ] CTO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:14:54.397778] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-11-04T18:14:54.405237] [INFO ] [MAIN ] CTO: Wiping existing primary data before full refresh. [2025-11-04T18:14:54.405300] [INFO ] [LOCAL ] CTO: Starting deletion of primary data. [2025-11-04T18:14:54.405461] [DEBUG] [LOCAL ] CTO: Deleted primary data file. [2025-11-04T18:14:54.405499] [INFO ] [MAIN ] CTO: Starting full primary refresh. [2025-11-04T18:14:54.577382] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-09. [2025-11-04T18:14:54.761179] [INFO ] [REMOTE ] CTO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:14:55.367038] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-01. [2025-11-04T18:14:55.881205] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T18:14:56.167017] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-10. [2025-11-04T18:14:56.241389] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T18:14:56.967045] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-05. [2025-11-04T18:14:57.279286] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-11-04T18:14:57.482182] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T18:14:57.767070] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-06. [2025-11-04T18:14:58.285177] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-11-04T18:14:58.567053] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-11. [2025-11-04T18:14:59.367042] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-07. [2025-11-04T18:14:59.674054] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-07 (0.31 seconds). [2025-11-04T18:14:59.695828] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T18:14:59.701343] [INFO ] [REMOTE ] CSTM: Full fetch complete (69507 records). [2025-11-04T18:14:59.701496] [DEBUG] [VALIDATOR ] CSTM: Validating primary data. [2025-11-04T18:15:00.076757] [DEBUG] [VALIDATOR ] CSTM: All expected trading days are present. [2025-11-04T18:15:00.076870] [DEBUG] [VALIDATOR ] CSTM: Primary data validation passed. [2025-11-04T18:15:00.077698] [INFO ] [MAIN ] CSTM: Fetched primary data is valid. [2025-11-04T18:15:00.077762] [DEBUG] [LOCAL ] CSTM: Saving primary data. [2025-11-04T18:15:00.079126] [DEBUG] [LOCAL ] CSTM: Trimmed 69507 records to 47248 (keeping 200 days of data) [2025-11-04T18:15:00.176546] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-02. [2025-11-04T18:15:00.349543] [DEBUG] [LOCAL ] CSTM: Primary data saved (47248 records). [2025-11-04T18:15:00.349706] [INFO ] [MAIN ] CSTM: Full history saved. [2025-11-04T18:15:00.349801] [INFO ] [MAIN ] Update Progress: 545/2367 symbols completed. CSTM took 45.7s. Estimated remaining: 4h 10m 34s. Rate: 7.3 symbols/min [2025-11-04T18:15:00.528473] [INFO ] [MAIN ] CTOS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:15:00.562183] [DEBUG] [LOCAL ] CTOS: Loaded 36943 existing records from primary storage. [2025-11-04T18:15:00.562468] [INFO ] [MAIN ] CTOS: Loaded 36943 existing records. [2025-11-04T18:15:00.747051] [WARN ] [MAIN ] CTOS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:15:00.747302] [INFO ] [MAIN ] CTOS: Wiping existing primary data before full refresh. [2025-11-04T18:15:00.747362] [INFO ] [LOCAL ] CTOS: Starting deletion of primary data. [2025-11-04T18:15:00.747547] [DEBUG] [LOCAL ] CTOS: Deleted primary data file. [2025-11-04T18:15:00.747588] [INFO ] [MAIN ] CTOS: Starting full primary refresh. [2025-11-04T18:15:00.979157] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-08. [2025-11-04T18:15:01.106220] [INFO ] [REMOTE ] CTOS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:15:01.142448] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T18:15:01.398572] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-11-04T18:15:01.767096] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-01. [2025-11-04T18:15:01.968890] [DEBUG] [REMOTE ] CTEV: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:15:01.969284] [WARN ] [REMOTE ] CTEV: No data returned for month 2025-01 [2025-11-04T18:15:01.969398] [FATAL] [REMOTE ] CTEV: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:15:01.969458] [FATAL] [MAIN ] CTEV: Failed to fetch primary data. [2025-11-04T18:15:01.969532] [WARN ] [MAIN ] CTEV: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:15:01.969619] [INFO ] [LOCAL ] CTEV: Starting deletion of primary data. [2025-11-04T18:15:01.969750] [DEBUG] [LOCAL ] CTEV: No primary data file found to delete. [2025-11-04T18:15:01.969808] [INFO ] [LOCAL ] CTEV: Starting deletion of derivative data. [2025-11-04T18:15:01.969867] [DEBUG] [LOCAL ] CTEV: No derivative data file found to delete. [2025-11-04T18:15:01.969936] [INFO ] [MAIN ] Update Progress: 546/2367 symbols completed. CTEV took 10.6s. Estimated remaining: 4h 10m 4s. Rate: 7.3 symbols/min [2025-11-04T18:15:02.158217] [INFO ] [MAIN ] CTRA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:15:02.158414] [DEBUG] [LOCAL ] CTRA: No existing data file found in primary storage. [2025-11-04T18:15:02.158451] [WARN ] [MAIN ] CTRA: No existing data found, performing full refresh instead. [2025-11-04T18:15:02.158568] [INFO ] [MAIN ] CTRA: Wiping existing primary data before full refresh. [2025-11-04T18:15:02.158629] [INFO ] [LOCAL ] CTRA: Starting deletion of primary data. [2025-11-04T18:15:02.158710] [DEBUG] [LOCAL ] CTRA: No primary data file found to delete. [2025-11-04T18:15:02.158751] [INFO ] [MAIN ] CTRA: Starting full primary refresh. [2025-11-04T18:15:02.516428] [INFO ] [REMOTE ] CTRA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:15:02.567051] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-01. [2025-11-04T18:15:03.367044] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-03. [2025-11-04T18:15:04.089349] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-11-04T18:15:04.167012] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-09. [2025-11-04T18:15:04.405278] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T18:15:04.787953] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-11-04T18:15:04.967028] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-01. [2025-11-04T18:15:05.767105] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-02. [2025-11-04T18:15:06.285502] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T18:15:06.567074] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-02. [2025-11-04T18:15:06.938383] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T18:15:07.367050] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-03. [2025-11-04T18:15:07.440004] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T18:15:08.167106] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-03. [2025-11-04T18:15:08.691886] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T18:15:08.967081] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-04. [2025-11-04T18:15:09.092068] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-11-04T18:15:09.767043] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-04. [2025-11-04T18:15:10.266067] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T18:15:10.567023] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-05. [2025-11-04T18:15:10.914832] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T18:15:11.367062] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-04. [2025-11-04T18:15:11.936813] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T18:15:12.167044] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-06. [2025-11-04T18:15:12.386359] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T18:15:12.967034] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-05. [2025-11-04T18:15:13.415954] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-11-04T18:15:13.767015] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-07. [2025-11-04T18:15:13.829728] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-11-04T18:15:14.567043] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-06. [2025-11-04T18:15:15.330604] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-11-04T18:15:15.367002] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-08. [2025-11-04T18:15:15.421628] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T18:15:16.167035] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-10. [2025-11-04T18:15:16.599073] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-11-04T18:15:16.818488] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T18:15:16.967023] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-11. [2025-11-04T18:15:17.379087] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T18:15:17.379914] [INFO ] [REMOTE ] CTDD: Full fetch complete (7908 records). [2025-11-04T18:15:17.379985] [DEBUG] [VALIDATOR ] CTDD: Validating primary data. [2025-11-04T18:15:17.772279] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-07. [2025-11-04T18:15:17.779747] [DEBUG] [VALIDATOR ] CTDD: All expected trading days are present. [2025-11-04T18:15:17.780783] [DEBUG] [VALIDATOR ] CTDD: Primary data validation passed. [2025-11-04T18:15:17.781146] [INFO ] [MAIN ] CTDD: Fetched primary data is valid. [2025-11-04T18:15:17.781222] [DEBUG] [LOCAL ] CTDD: Saving primary data. [2025-11-04T18:15:17.781907] [DEBUG] [LOCAL ] CTDD: Trimmed 7908 records to 4941 (keeping 200 days of data) [2025-11-04T18:15:17.815823] [DEBUG] [LOCAL ] CTDD: Primary data saved (4941 records). [2025-11-04T18:15:17.815950] [INFO ] [MAIN ] CTDD: Full history saved. [2025-11-04T18:15:17.816029] [INFO ] [MAIN ] Update Progress: 547/2367 symbols completed. CTDD took 39.4s. Estimated remaining: 4h 10m 21s. Rate: 7.3 symbols/min [2025-11-04T18:15:18.014997] [INFO ] [MAIN ] CTRE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:15:18.056955] [DEBUG] [LOCAL ] CTRE: Loaded 49464 existing records from primary storage. [2025-11-04T18:15:18.057264] [INFO ] [MAIN ] CTRE: Loaded 49464 existing records. [2025-11-04T18:15:18.240217] [WARN ] [MAIN ] CTRE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:15:18.240506] [INFO ] [MAIN ] CTRE: Wiping existing primary data before full refresh. [2025-11-04T18:15:18.240565] [INFO ] [LOCAL ] CTRE: Starting deletion of primary data. [2025-11-04T18:15:18.241173] [DEBUG] [LOCAL ] CTRE: Deleted primary data file. [2025-11-04T18:15:18.241231] [INFO ] [MAIN ] CTRE: Starting full primary refresh. [2025-11-04T18:15:18.571224] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-05. [2025-11-04T18:15:18.603174] [INFO ] [REMOTE ] CTRE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:15:18.621976] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T18:15:19.367054] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-09. [2025-11-04T18:15:19.778442] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T18:15:20.167087] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-01. [2025-11-04T18:15:20.569005] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T18:15:20.967035] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-08. [2025-11-04T18:15:21.520700] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T18:15:21.683329] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-11-04T18:15:21.767074] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-09. [2025-11-04T18:15:22.567009] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-06. [2025-11-04T18:15:22.705570] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-11-04T18:15:23.370704] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-02. [2025-11-04T18:15:23.375056] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T18:15:24.167049] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-10. [2025-11-04T18:15:24.582264] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T18:15:24.967112] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-10. [2025-11-04T18:15:25.767022] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-03. [2025-11-04T18:15:25.862063] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-11-04T18:15:26.033670] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T18:15:26.567114] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-11. [2025-11-04T18:15:26.964474] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T18:15:27.367092] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-07. [2025-11-04T18:15:27.406523] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T18:15:27.408358] [INFO ] [REMOTE ] CTO: Full fetch complete (41586 records). [2025-11-04T18:15:27.408477] [DEBUG] [VALIDATOR ] CTO: Validating primary data. [2025-11-04T18:15:27.816091] [DEBUG] [VALIDATOR ] CTO: All expected trading days are present. [2025-11-04T18:15:27.816201] [DEBUG] [VALIDATOR ] CTO: Primary data validation passed. [2025-11-04T18:15:27.817045] [INFO ] [MAIN ] CTO: Fetched primary data is valid. [2025-11-04T18:15:27.817111] [DEBUG] [LOCAL ] CTO: Saving primary data. [2025-11-04T18:15:27.818327] [DEBUG] [LOCAL ] CTO: Trimmed 41586 records to 27200 (keeping 200 days of data) [2025-11-04T18:15:28.001142] [DEBUG] [LOCAL ] CTO: Primary data saved (27200 records). [2025-11-04T18:15:28.001366] [INFO ] [MAIN ] CTO: Full history saved. [2025-11-04T18:15:28.001499] [INFO ] [MAIN ] Update Progress: 548/2367 symbols completed. CTO took 34.0s. Estimated remaining: 4h 10m 19s. Rate: 7.3 symbols/min [2025-11-04T18:15:28.167259] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-04. [2025-11-04T18:15:28.192026] [INFO ] [MAIN ] CTRI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:15:28.192340] [DEBUG] [LOCAL ] CTRI: No existing data file found in primary storage. [2025-11-04T18:15:28.192387] [WARN ] [MAIN ] CTRI: No existing data found, performing full refresh instead. [2025-11-04T18:15:28.192506] [INFO ] [MAIN ] CTRI: Wiping existing primary data before full refresh. [2025-11-04T18:15:28.192549] [INFO ] [LOCAL ] CTRI: Starting deletion of primary data. [2025-11-04T18:15:28.192590] [DEBUG] [LOCAL ] CTRI: No primary data file found to delete. [2025-11-04T18:15:28.192624] [INFO ] [MAIN ] CTRI: Starting full primary refresh. [2025-11-04T18:15:28.543787] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T18:15:28.622187] [INFO ] [REMOTE ] CTRI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:15:28.967097] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-01. [2025-11-04T18:15:29.548953] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T18:15:29.767517] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-08. [2025-11-04T18:15:29.767450] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-11-04T18:15:30.567045] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-05. [2025-11-04T18:15:30.909471] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T18:15:31.367045] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-02. [2025-11-04T18:15:31.887983] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:15:32.167034] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-06. [2025-11-04T18:15:32.263024] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T18:15:32.967038] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-09. [2025-11-04T18:15:33.263815] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T18:15:33.767054] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-11. [2025-11-04T18:15:33.967469] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T18:15:34.567054] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-10. [2025-11-04T18:15:35.367058] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-03. [2025-11-04T18:15:35.538844] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T18:15:35.553134] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-11 (1.79 seconds). [2025-11-04T18:15:35.555135] [INFO ] [REMOTE ] CTRA: Full fetch complete (77820 records). [2025-11-04T18:15:35.555228] [DEBUG] [VALIDATOR ] CTRA: Validating primary data. [2025-11-04T18:15:35.951216] [DEBUG] [VALIDATOR ] CTRA: All expected trading days are present. [2025-11-04T18:15:35.951294] [DEBUG] [VALIDATOR ] CTRA: Primary data validation passed. [2025-11-04T18:15:35.952121] [INFO ] [MAIN ] CTRA: Fetched primary data is valid. [2025-11-04T18:15:35.952183] [DEBUG] [LOCAL ] CTRA: Saving primary data. [2025-11-04T18:15:35.953637] [DEBUG] [LOCAL ] CTRA: Trimmed 77820 records to 53646 (keeping 200 days of data) [2025-11-04T18:15:36.175739] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-07. [2025-11-04T18:15:36.216464] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-11-04T18:15:36.303959] [DEBUG] [LOCAL ] CTRA: Primary data saved (53646 records). [2025-11-04T18:15:36.304086] [INFO ] [MAIN ] CTRA: Full history saved. [2025-11-04T18:15:36.304163] [INFO ] [MAIN ] Update Progress: 549/2367 symbols completed. CTRA took 34.3s. Estimated remaining: 4h 10m 11s. Rate: 7.3 symbols/min [2025-11-04T18:15:36.479370] [INFO ] [MAIN ] CTS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:15:36.479558] [DEBUG] [LOCAL ] CTS: No existing data file found in primary storage. [2025-11-04T18:15:36.479613] [WARN ] [MAIN ] CTS: No existing data found, performing full refresh instead. [2025-11-04T18:15:36.479760] [INFO ] [MAIN ] CTS: Wiping existing primary data before full refresh. [2025-11-04T18:15:36.479815] [INFO ] [LOCAL ] CTS: Starting deletion of primary data. [2025-11-04T18:15:36.479867] [DEBUG] [LOCAL ] CTS: No primary data file found to delete. [2025-11-04T18:15:36.479901] [INFO ] [MAIN ] CTS: Starting full primary refresh. [2025-11-04T18:15:36.848299] [INFO ] [REMOTE ] CTS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:15:36.967004] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-01. [2025-11-04T18:15:37.378832] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T18:15:37.736567] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-11-04T18:15:37.767056] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-02. [2025-11-04T18:15:37.823217] [DEBUG] [REMOTE ] CTS: Error fetching primary data for 2025-02: 'API call returned an empty response.' [2025-11-04T18:15:37.823528] [WARN ] [REMOTE ] CTS: No data returned for month 2025-02 [2025-11-04T18:15:37.823624] [FATAL] [REMOTE ] CTS: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:15:37.823737] [FATAL] [MAIN ] CTS: Failed to fetch primary data. [2025-11-04T18:15:37.823784] [WARN ] [MAIN ] CTS: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:15:37.823853] [INFO ] [LOCAL ] CTS: Starting deletion of primary data. [2025-11-04T18:15:37.823917] [DEBUG] [LOCAL ] CTS: No primary data file found to delete. [2025-11-04T18:15:37.823952] [INFO ] [LOCAL ] CTS: Starting deletion of derivative data. [2025-11-04T18:15:37.823993] [DEBUG] [LOCAL ] CTS: No derivative data file found to delete. [2025-11-04T18:15:37.824039] [INFO ] [MAIN ] Update Progress: 550/2367 symbols completed. CTS took 1.5s. Estimated remaining: 4h 9m 40s. Rate: 7.3 symbols/min [2025-11-04T18:15:38.001983] [INFO ] [MAIN ] CTVA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:15:38.053181] [DEBUG] [LOCAL ] CTVA: Loaded 53317 existing records from primary storage. [2025-11-04T18:15:38.053536] [INFO ] [MAIN ] CTVA: Loaded 53317 existing records. [2025-11-04T18:15:38.235115] [WARN ] [MAIN ] CTVA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:15:38.235341] [INFO ] [MAIN ] CTVA: Wiping existing primary data before full refresh. [2025-11-04T18:15:38.235390] [INFO ] [LOCAL ] CTVA: Starting deletion of primary data. [2025-11-04T18:15:38.235594] [DEBUG] [LOCAL ] CTVA: Deleted primary data file. [2025-11-04T18:15:38.235682] [INFO ] [MAIN ] CTVA: Starting full primary refresh. [2025-11-04T18:15:38.572443] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-11. [2025-11-04T18:15:38.601558] [INFO ] [REMOTE ] CTVA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:15:39.367039] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-08. [2025-11-04T18:15:39.513161] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T18:15:39.515519] [INFO ] [REMOTE ] CTOS: Full fetch complete (54291 records). [2025-11-04T18:15:39.515790] [DEBUG] [VALIDATOR ] CTOS: Validating primary data. [2025-11-04T18:15:39.913356] [DEBUG] [VALIDATOR ] CTOS: All expected trading days are present. [2025-11-04T18:15:39.913449] [DEBUG] [VALIDATOR ] CTOS: Primary data validation passed. [2025-11-04T18:15:39.914438] [INFO ] [MAIN ] CTOS: Fetched primary data is valid. [2025-11-04T18:15:39.914516] [DEBUG] [LOCAL ] CTOS: Saving primary data. [2025-11-04T18:15:39.915934] [DEBUG] [LOCAL ] CTOS: Trimmed 54291 records to 36842 (keeping 200 days of data) [2025-11-04T18:15:40.116631] [DEBUG] [LOCAL ] CTOS: Primary data saved (36842 records). [2025-11-04T18:15:40.116813] [INFO ] [MAIN ] CTOS: Full history saved. [2025-11-04T18:15:40.116907] [INFO ] [MAIN ] Update Progress: 551/2367 symbols completed. CTOS took 39.8s. Estimated remaining: 4h 9m 13s. Rate: 7.3 symbols/min [2025-11-04T18:15:40.175231] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-04. [2025-11-04T18:15:40.294596] [INFO ] [MAIN ] CUBB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:15:40.294782] [DEBUG] [LOCAL ] CUBB: No existing data file found in primary storage. [2025-11-04T18:15:40.294824] [WARN ] [MAIN ] CUBB: No existing data found, performing full refresh instead. [2025-11-04T18:15:40.294933] [INFO ] [MAIN ] CUBB: Wiping existing primary data before full refresh. [2025-11-04T18:15:40.294974] [INFO ] [LOCAL ] CUBB: Starting deletion of primary data. [2025-11-04T18:15:40.295014] [DEBUG] [LOCAL ] CUBB: No primary data file found to delete. [2025-11-04T18:15:40.295045] [INFO ] [MAIN ] CUBB: Starting full primary refresh. [2025-11-04T18:15:40.646866] [INFO ] [REMOTE ] CUBB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:15:40.727885] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-11-04T18:15:40.967021] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-01. [2025-11-04T18:15:41.149623] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-11-04T18:15:41.295375] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-11-04T18:15:41.767073] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-02. [2025-11-04T18:15:42.005702] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-02 (0.24 seconds). [2025-11-04T18:15:42.567039] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-03. [2025-11-04T18:15:42.771737] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-03 (0.20 seconds). [2025-11-04T18:15:43.367040] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-04. [2025-11-04T18:15:43.593636] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-04 (0.23 seconds). [2025-11-04T18:15:44.167049] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-09. [2025-11-04T18:15:44.967039] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-05. [2025-11-04T18:15:45.282913] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-11-04T18:15:45.492413] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T18:15:45.767029] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-06. [2025-11-04T18:15:45.971685] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-06 (0.20 seconds). [2025-11-04T18:15:46.567051] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-07. [2025-11-04T18:15:46.864295] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-07 (0.30 seconds). [2025-11-04T18:15:47.367050] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-10. [2025-11-04T18:15:48.167094] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-08. [2025-11-04T18:15:48.434095] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-08 (0.27 seconds). [2025-11-04T18:15:48.791178] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-11-04T18:15:48.967026] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-11. [2025-11-04T18:15:49.767086] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-09. [2025-11-04T18:15:49.961135] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T18:15:49.963522] [INFO ] [REMOTE ] CTRE: Full fetch complete (71570 records). [2025-11-04T18:15:49.963691] [DEBUG] [VALIDATOR ] CTRE: Validating primary data. [2025-11-04T18:15:50.041120] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-09 (0.27 seconds). [2025-11-04T18:15:50.351344] [DEBUG] [VALIDATOR ] CTRE: All expected trading days are present. [2025-11-04T18:15:50.351442] [DEBUG] [VALIDATOR ] CTRE: Primary data validation passed. [2025-11-04T18:15:50.352232] [INFO ] [MAIN ] CTRE: Fetched primary data is valid. [2025-11-04T18:15:50.352292] [DEBUG] [LOCAL ] CTRE: Saving primary data. [2025-11-04T18:15:50.353584] [DEBUG] [LOCAL ] CTRE: Trimmed 71570 records to 49493 (keeping 200 days of data) [2025-11-04T18:15:50.567057] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-05. [2025-11-04T18:15:50.648141] [DEBUG] [LOCAL ] CTRE: Primary data saved (49493 records). [2025-11-04T18:15:50.648262] [INFO ] [MAIN ] CTRE: Full history saved. [2025-11-04T18:15:50.648336] [INFO ] [MAIN ] Update Progress: 552/2367 symbols completed. CTRE took 32.8s. Estimated remaining: 4h 9m 12s. Rate: 7.3 symbols/min [2025-11-04T18:15:50.813947] [INFO ] [MAIN ] CUBE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:15:50.859124] [DEBUG] [LOCAL ] CUBE: Loaded 50923 existing records from primary storage. [2025-11-04T18:15:50.859439] [INFO ] [MAIN ] CUBE: Loaded 50923 existing records. [2025-11-04T18:15:51.050705] [WARN ] [MAIN ] CUBE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:15:51.050947] [INFO ] [MAIN ] CUBE: Wiping existing primary data before full refresh. [2025-11-04T18:15:51.051004] [INFO ] [LOCAL ] CUBE: Starting deletion of primary data. [2025-11-04T18:15:51.051181] [DEBUG] [LOCAL ] CUBE: Deleted primary data file. [2025-11-04T18:15:51.051218] [INFO ] [MAIN ] CUBE: Starting full primary refresh. [2025-11-04T18:15:51.372213] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-10. [2025-11-04T18:15:51.400256] [INFO ] [REMOTE ] CUBE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:15:51.555187] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T18:15:51.737844] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-11-04T18:15:52.167092] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-01. [2025-11-04T18:15:52.967015] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-01. [2025-11-04T18:15:53.452227] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T18:15:53.767045] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-02. [2025-11-04T18:15:54.258119] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T18:15:54.567089] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-02. [2025-11-04T18:15:54.761151] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T18:15:55.367079] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-03. [2025-11-04T18:15:55.607395] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-11-04T18:15:56.167028] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-03. [2025-11-04T18:15:56.566809] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T18:15:56.967041] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-11. [2025-11-04T18:15:57.368638] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-11 (0.40 seconds). [2025-11-04T18:15:57.369426] [INFO ] [REMOTE ] CUBB: Full fetch complete (981 records). [2025-11-04T18:15:57.369505] [DEBUG] [VALIDATOR ] CUBB: Validating primary data. [2025-11-04T18:15:57.565300] [WARN ] [VALIDATOR ] CUBB: Validation failed - insufficient historical data. Have 185 unique days, need 200 days [2025-11-04T18:15:57.565554] [FATAL] [MAIN ] CUBB: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:15:57.565635] [WARN ] [MAIN ] CUBB: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:15:57.565743] [INFO ] [LOCAL ] CUBB: Starting deletion of primary data. [2025-11-04T18:15:57.565832] [DEBUG] [LOCAL ] CUBB: No primary data file found to delete. [2025-11-04T18:15:57.565969] [INFO ] [LOCAL ] CUBB: Starting deletion of derivative data. [2025-11-04T18:15:57.566066] [DEBUG] [LOCAL ] CUBB: No derivative data file found to delete. [2025-11-04T18:15:57.566196] [INFO ] [MAIN ] Update Progress: 553/2367 symbols completed. CUBB took 17.4s. Estimated remaining: 4h 8m 59s. Rate: 7.3 symbols/min [2025-11-04T18:15:57.720308] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T18:15:57.776673] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-04. [2025-11-04T18:15:57.815500] [INFO ] [MAIN ] CUBI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:15:57.845114] [DEBUG] [LOCAL ] CUBI: Loaded 26565 existing records from primary storage. [2025-11-04T18:15:57.845381] [INFO ] [MAIN ] CUBI: Loaded 26565 existing records. [2025-11-04T18:15:58.060312] [WARN ] [MAIN ] CUBI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:15:58.060553] [INFO ] [MAIN ] CUBI: Wiping existing primary data before full refresh. [2025-11-04T18:15:58.060617] [INFO ] [LOCAL ] CUBI: Starting deletion of primary data. [2025-11-04T18:15:58.060855] [DEBUG] [LOCAL ] CUBI: Deleted primary data file. [2025-11-04T18:15:58.060912] [INFO ] [MAIN ] CUBI: Starting full primary refresh. [2025-11-04T18:15:58.463837] [INFO ] [REMOTE ] CUBI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:15:58.567104] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-01. [2025-11-04T18:15:59.342506] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-11-04T18:15:59.367057] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-05. [2025-11-04T18:15:59.623944] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T18:16:00.167095] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-06. [2025-11-04T18:16:00.688680] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:16:00.967040] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-04. [2025-11-04T18:16:01.119249] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-11-04T18:16:01.767058] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-02. [2025-11-04T18:16:02.214200] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T18:16:02.543104] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-11-04T18:16:02.567094] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-03. [2025-11-04T18:16:03.367092] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-07. [2025-11-04T18:16:03.456072] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-11-04T18:16:04.167085] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-04. [2025-11-04T18:16:04.602389] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-11-04T18:16:04.967024] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-08. [2025-11-04T18:16:05.136217] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-11-04T18:16:05.767063] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-05. [2025-11-04T18:16:06.021959] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-11-04T18:16:06.395320] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-11-04T18:16:06.567054] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-05. [2025-11-04T18:16:07.367032] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-06. [2025-11-04T18:16:08.023699] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T18:16:08.167071] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-09. [2025-11-04T18:16:08.791836] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-11-04T18:16:08.967045] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-07. [2025-11-04T18:16:09.490497] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-11-04T18:16:09.767077] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-06. [2025-11-04T18:16:10.374578] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T18:16:10.567027] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-08. [2025-11-04T18:16:11.145028] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-11-04T18:16:11.367065] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-06. [2025-11-04T18:16:11.877556] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-11-04T18:16:12.167046] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-09. [2025-11-04T18:16:12.199011] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T18:16:12.967021] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-10. [2025-11-04T18:16:13.509533] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T18:16:13.767049] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-10. [2025-11-04T18:16:14.121929] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-11-04T18:16:14.567035] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-07. [2025-11-04T18:16:15.161307] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T18:16:15.367048] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-11. [2025-11-04T18:16:15.962556] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T18:16:16.167027] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-11. [2025-11-04T18:16:16.654989] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-11 (1.29 seconds). [2025-11-04T18:16:16.657031] [INFO ] [REMOTE ] CTRI: Full fetch complete (55875 records). [2025-11-04T18:16:16.657149] [DEBUG] [VALIDATOR ] CTRI: Validating primary data. [2025-11-04T18:16:16.973051] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-08. [2025-11-04T18:16:17.073417] [DEBUG] [VALIDATOR ] CTRI: All expected trading days are present. [2025-11-04T18:16:17.073517] [DEBUG] [VALIDATOR ] CTRI: Primary data validation passed. [2025-11-04T18:16:17.074494] [INFO ] [MAIN ] CTRI: Fetched primary data is valid. [2025-11-04T18:16:17.074562] [DEBUG] [LOCAL ] CTRI: Saving primary data. [2025-11-04T18:16:17.075902] [DEBUG] [LOCAL ] CTRI: Trimmed 55875 records to 41783 (keeping 200 days of data) [2025-11-04T18:16:17.328466] [DEBUG] [LOCAL ] CTRI: Primary data saved (41783 records). [2025-11-04T18:16:17.328595] [INFO ] [MAIN ] CTRI: Full history saved. [2025-11-04T18:16:17.328686] [INFO ] [MAIN ] Update Progress: 554/2367 symbols completed. CTRI took 49.3s. Estimated remaining: 4h 9m 29s. Rate: 7.3 symbols/min [2025-11-04T18:16:17.495667] [INFO ] [MAIN ] CUK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:16:17.495830] [DEBUG] [LOCAL ] CUK: No existing data file found in primary storage. [2025-11-04T18:16:17.495866] [WARN ] [MAIN ] CUK: No existing data found, performing full refresh instead. [2025-11-04T18:16:17.495971] [INFO ] [MAIN ] CUK: Wiping existing primary data before full refresh. [2025-11-04T18:16:17.496012] [INFO ] [LOCAL ] CUK: Starting deletion of primary data. [2025-11-04T18:16:17.496051] [DEBUG] [LOCAL ] CUK: No primary data file found to delete. [2025-11-04T18:16:17.496082] [INFO ] [MAIN ] CUK: Starting full primary refresh. [2025-11-04T18:16:17.577987] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-11 (1.41 seconds). [2025-11-04T18:16:17.600717] [INFO ] [REMOTE ] CTVA: Full fetch complete (76863 records). [2025-11-04T18:16:17.611379] [DEBUG] [VALIDATOR ] CTVA: Validating primary data. [2025-11-04T18:16:17.627878] [FATAL] [VALIDATOR ] CTVA: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:16:17.642927] [FATAL] [MAIN ] CTVA: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:16:17.659461] [WARN ] [MAIN ] CTVA: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:16:17.677745] [INFO ] [LOCAL ] CTVA: Starting deletion of primary data. [2025-11-04T18:16:17.712587] [DEBUG] [LOCAL ] CTVA: No primary data file found to delete. [2025-11-04T18:16:17.744198] [INFO ] [LOCAL ] CTVA: Starting deletion of derivative data. [2025-11-04T18:16:17.755492] [DEBUG] [LOCAL ] CTVA: Deleted derivative data file. [2025-11-04T18:16:17.771276] [INFO ] [MAIN ] Update Progress: 555/2367 symbols completed. CTVA took 39.9s. Estimated remaining: 4h 8m 55s. Rate: 7.3 symbols/min [2025-11-04T18:16:17.771130] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-07. [2025-11-04T18:16:17.966372] [INFO ] [REMOTE ] CUK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:16:18.035025] [INFO ] [MAIN ] CULP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:16:18.039980] [DEBUG] [LOCAL ] CULP: Loaded 2777 existing records from primary storage. [2025-11-04T18:16:18.040165] [INFO ] [MAIN ] CULP: Loaded 2777 existing records. [2025-11-04T18:16:18.217769] [WARN ] [MAIN ] CULP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:16:18.218024] [INFO ] [MAIN ] CULP: Wiping existing primary data before full refresh. [2025-11-04T18:16:18.218076] [INFO ] [LOCAL ] CULP: Starting deletion of primary data. [2025-11-04T18:16:18.218265] [DEBUG] [LOCAL ] CULP: Deleted primary data file. [2025-11-04T18:16:18.218379] [INFO ] [MAIN ] CULP: Starting full primary refresh. [2025-11-04T18:16:18.579226] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-01. [2025-11-04T18:16:18.601232] [INFO ] [REMOTE ] CULP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:16:18.605403] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-11-04T18:16:18.839410] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T18:16:19.367055] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-08. [2025-11-04T18:16:20.046509] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T18:16:20.070338] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T18:16:20.167033] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-09. [2025-11-04T18:16:20.967050] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-01. [2025-11-04T18:16:21.148139] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T18:16:21.435442] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-01 (0.47 seconds). [2025-11-04T18:16:21.767054] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-02. [2025-11-04T18:16:22.567101] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-02. [2025-11-04T18:16:22.937611] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-11-04T18:16:22.988904] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T18:16:23.367045] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-03. [2025-11-04T18:16:24.167102] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-10. [2025-11-04T18:16:24.684973] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T18:16:24.967028] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-09. [2025-11-04T18:16:25.460608] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T18:16:25.767101] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-11. [2025-11-04T18:16:26.129757] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T18:16:26.567014] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-10. [2025-11-04T18:16:26.577069] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T18:16:26.578305] [INFO ] [REMOTE ] CUBI: Full fetch complete (39020 records). [2025-11-04T18:16:26.578374] [DEBUG] [VALIDATOR ] CUBI: Validating primary data. [2025-11-04T18:16:26.942356] [DEBUG] [VALIDATOR ] CUBI: All expected trading days are present. [2025-11-04T18:16:26.942431] [DEBUG] [VALIDATOR ] CUBI: Primary data validation passed. [2025-11-04T18:16:26.942970] [INFO ] [MAIN ] CUBI: Fetched primary data is valid. [2025-11-04T18:16:26.943031] [DEBUG] [LOCAL ] CUBI: Saving primary data. [2025-11-04T18:16:26.943832] [DEBUG] [LOCAL ] CUBI: Trimmed 39020 records to 26550 (keeping 200 days of data) [2025-11-04T18:16:27.092758] [DEBUG] [LOCAL ] CUBI: Primary data saved (26550 records). [2025-11-04T18:16:27.092884] [INFO ] [MAIN ] CUBI: Full history saved. [2025-11-04T18:16:27.092959] [INFO ] [MAIN ] Update Progress: 556/2367 symbols completed. CUBI took 29.5s. Estimated remaining: 4h 8m 50s. Rate: 7.3 symbols/min [2025-11-04T18:16:27.273262] [INFO ] [MAIN ] CURB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:16:27.306556] [DEBUG] [LOCAL ] CURB: Loaded 32679 existing records from primary storage. [2025-11-04T18:16:27.306910] [INFO ] [MAIN ] CURB: Loaded 32679 existing records. [2025-11-04T18:16:27.372312] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-04. [2025-11-04T18:16:27.505901] [WARN ] [MAIN ] CURB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:16:27.506143] [INFO ] [MAIN ] CURB: Wiping existing primary data before full refresh. [2025-11-04T18:16:27.506188] [INFO ] [LOCAL ] CURB: Starting deletion of primary data. [2025-11-04T18:16:27.506386] [DEBUG] [LOCAL ] CURB: Deleted primary data file. [2025-11-04T18:16:27.506439] [INFO ] [MAIN ] CURB: Starting full primary refresh. [2025-11-04T18:16:27.853735] [INFO ] [REMOTE ] CURB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:16:28.013376] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T18:16:28.167010] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-11. [2025-11-04T18:16:28.974016] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-01. [2025-11-04T18:16:28.979490] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-11-04T18:16:29.228272] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T18:16:29.229835] [INFO ] [REMOTE ] CUBE: Full fetch complete (72977 records). [2025-11-04T18:16:29.229973] [DEBUG] [VALIDATOR ] CUBE: Validating primary data. [2025-11-04T18:16:29.230675] [FATAL] [VALIDATOR ] CUBE: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:16:29.230758] [FATAL] [MAIN ] CUBE: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:16:29.230802] [WARN ] [MAIN ] CUBE: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:16:29.230840] [INFO ] [LOCAL ] CUBE: Starting deletion of primary data. [2025-11-04T18:16:29.230894] [DEBUG] [LOCAL ] CUBE: No primary data file found to delete. [2025-11-04T18:16:29.230926] [INFO ] [LOCAL ] CUBE: Starting deletion of derivative data. [2025-11-04T18:16:29.231074] [DEBUG] [LOCAL ] CUBE: Deleted derivative data file. [2025-11-04T18:16:29.231163] [INFO ] [MAIN ] Update Progress: 557/2367 symbols completed. CUBE took 38.6s. Estimated remaining: 4h 8m 22s. Rate: 7.3 symbols/min [2025-11-04T18:16:29.767043] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-05. [2025-11-04T18:16:29.980313] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T18:16:30.567045] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-02. [2025-11-04T18:16:31.256594] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-11-04T18:16:31.367025] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-06. [2025-11-04T18:16:31.412723] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T18:16:32.167094] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-03. [2025-11-04T18:16:32.628157] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-11-04T18:16:32.846363] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-11-04T18:16:32.967031] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-04. [2025-11-04T18:16:33.330567] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T18:16:33.767102] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-07. [2025-11-04T18:16:34.567031] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-03. [2025-11-04T18:16:34.993822] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T18:16:35.367041] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-08. [2025-11-04T18:16:35.592278] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T18:16:36.167062] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-04. [2025-11-04T18:16:36.759992] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T18:16:36.967035] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-05. [2025-11-04T18:16:37.238912] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T18:16:37.271129] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-11-04T18:16:37.767031] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-05. [2025-11-04T18:16:38.567012] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-09. [2025-11-04T18:16:38.641313] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T18:16:39.367041] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-06. [2025-11-04T18:16:39.666562] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-06 (0.30 seconds). [2025-11-04T18:16:40.011065] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T18:16:40.167032] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-10. [2025-11-04T18:16:40.967074] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-07. [2025-11-04T18:16:41.276323] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-07 (0.31 seconds). [2025-11-04T18:16:41.594998] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T18:16:41.767044] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-11. [2025-11-04T18:16:42.567038] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-08. [2025-11-04T18:16:43.041418] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-11 (1.27 seconds). [2025-11-04T18:16:43.042930] [INFO ] [REMOTE ] CUK: Full fetch complete (74096 records). [2025-11-04T18:16:43.043022] [DEBUG] [VALIDATOR ] CUK: Validating primary data. [2025-11-04T18:16:43.043759] [FATAL] [VALIDATOR ] CUK: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:16:43.043854] [FATAL] [MAIN ] CUK: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:16:43.043898] [WARN ] [MAIN ] CUK: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:16:43.043965] [INFO ] [LOCAL ] CUK: Starting deletion of primary data. [2025-11-04T18:16:43.044028] [DEBUG] [LOCAL ] CUK: No primary data file found to delete. [2025-11-04T18:16:43.044063] [INFO ] [LOCAL ] CUK: Starting deletion of derivative data. [2025-11-04T18:16:43.044239] [DEBUG] [LOCAL ] CUK: No derivative data file found to delete. [2025-11-04T18:16:43.044323] [INFO ] [MAIN ] Update Progress: 558/2367 symbols completed. CUK took 25.7s. Estimated remaining: 4h 8m 32s. Rate: 7.3 symbols/min [2025-11-04T18:16:43.072110] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-11-04T18:16:43.367035] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-09. [2025-11-04T18:16:43.822107] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-11-04T18:16:44.167044] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-10. [2025-11-04T18:16:44.550051] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-11-04T18:16:44.967059] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-11. [2025-11-04T18:16:45.383554] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T18:16:45.384245] [INFO ] [REMOTE ] CULP: Full fetch complete (4396 records). [2025-11-04T18:16:45.384313] [DEBUG] [VALIDATOR ] CULP: Validating primary data. [2025-11-04T18:16:45.768589] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-06. [2025-11-04T18:16:45.802772] [DEBUG] [VALIDATOR ] CULP: All expected trading days are present. [2025-11-04T18:16:45.802887] [DEBUG] [VALIDATOR ] CULP: Primary data validation passed. [2025-11-04T18:16:45.803201] [INFO ] [MAIN ] CULP: Fetched primary data is valid. [2025-11-04T18:16:45.803264] [DEBUG] [LOCAL ] CULP: Saving primary data. [2025-11-04T18:16:45.804083] [DEBUG] [LOCAL ] CULP: Trimmed 4396 records to 2758 (keeping 200 days of data) [2025-11-04T18:16:45.831306] [DEBUG] [LOCAL ] CULP: Primary data saved (2758 records). [2025-11-04T18:16:45.831483] [INFO ] [MAIN ] CULP: Full history saved. [2025-11-04T18:16:45.831597] [INFO ] [MAIN ] Update Progress: 559/2367 symbols completed. CULP took 28.1s. Estimated remaining: 4h 8m 6s. Rate: 7.3 symbols/min [2025-11-04T18:16:46.678711] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T18:16:46.687098] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-07. [2025-11-04T18:16:47.695866] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T18:16:47.704261] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-08. [2025-11-04T18:16:48.550338] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-11-04T18:16:48.558786] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-09. [2025-11-04T18:16:49.565949] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T18:16:49.574002] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-10. [2025-11-04T18:16:50.561854] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-11-04T18:16:50.570790] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-11. [2025-11-04T18:16:51.470959] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T18:16:51.472451] [INFO ] [REMOTE ] CURB: Full fetch complete (47812 records). [2025-11-04T18:16:51.472523] [DEBUG] [VALIDATOR ] CURB: Validating primary data. [2025-11-04T18:16:51.874826] [DEBUG] [VALIDATOR ] CURB: All expected trading days are present. [2025-11-04T18:16:51.874926] [DEBUG] [VALIDATOR ] CURB: Primary data validation passed. [2025-11-04T18:16:51.875530] [INFO ] [MAIN ] CURB: Fetched primary data is valid. [2025-11-04T18:16:51.875569] [DEBUG] [LOCAL ] CURB: Saving primary data. [2025-11-04T18:16:51.876704] [DEBUG] [LOCAL ] CURB: Trimmed 47812 records to 32621 (keeping 200 days of data) [2025-11-04T18:16:52.069584] [DEBUG] [LOCAL ] CURB: Primary data saved (32621 records). [2025-11-04T18:16:52.069750] [INFO ] [MAIN ] CURB: Full history saved. [2025-11-04T18:16:52.069838] [INFO ] [MAIN ] Update Progress: 560/2367 symbols completed. CURB took 25.0s. Estimated remaining: 4h 7m 52s. Rate: 7.3 symbols/min [2025-11-04T18:16:52.666431] [INFO ] [MAIN ] CVEO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:16:52.733337] [INFO ] [MAIN ] CUZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:16:52.843867] [INFO ] [MAIN ] CVE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:16:52.841293] [INFO ] [MAIN ] CURV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:16:52.859201] [DEBUG] [LOCAL ] CVEO: Loaded 15285 existing records from primary storage. [2025-11-04T18:16:52.859795] [INFO ] [MAIN ] CVEO: Loaded 15285 existing records. [2025-11-04T18:16:53.031076] [DEBUG] [LOCAL ] CURV: Loaded 27710 existing records from primary storage. [2025-11-04T18:16:53.050543] [INFO ] [MAIN ] CURV: Loaded 27710 existing records. [2025-11-04T18:16:53.118494] [WARN ] [MAIN ] CVEO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:16:53.148742] [INFO ] [MAIN ] CVEO: Wiping existing primary data before full refresh. [2025-11-04T18:16:53.163984] [INFO ] [LOCAL ] CVEO: Starting deletion of primary data. [2025-11-04T18:16:53.207615] [DEBUG] [LOCAL ] CUZ: Loaded 49414 existing records from primary storage. [2025-11-04T18:16:53.219335] [INFO ] [MAIN ] CUZ: Loaded 49414 existing records. [2025-11-04T18:16:53.226886] [DEBUG] [LOCAL ] CVEO: Deleted primary data file. [2025-11-04T18:16:53.265036] [INFO ] [MAIN ] CVEO: Starting full primary refresh. [2025-11-04T18:16:53.317278] [DEBUG] [LOCAL ] CVE: Loaded 53557 existing records from primary storage. [2025-11-04T18:16:53.446857] [INFO ] [MAIN ] CVE: Loaded 53557 existing records. [2025-11-04T18:16:53.545167] [WARN ] [MAIN ] CURV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:16:53.574162] [INFO ] [MAIN ] CURV: Wiping existing primary data before full refresh. [2025-11-04T18:16:53.594915] [INFO ] [LOCAL ] CURV: Starting deletion of primary data. [2025-11-04T18:16:53.704606] [DEBUG] [LOCAL ] CURV: Deleted primary data file. [2025-11-04T18:16:53.753637] [INFO ] [MAIN ] CURV: Starting full primary refresh. [2025-11-04T18:16:53.820154] [WARN ] [MAIN ] CUZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:16:53.914387] [INFO ] [MAIN ] CUZ: Wiping existing primary data before full refresh. [2025-11-04T18:16:53.975702] [INFO ] [LOCAL ] CUZ: Starting deletion of primary data. [2025-11-04T18:16:54.069764] [WARN ] [MAIN ] CVE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:16:54.080491] [DEBUG] [LOCAL ] CUZ: Deleted primary data file. [2025-11-04T18:16:54.085842] [INFO ] [MAIN ] CUZ: Starting full primary refresh. [2025-11-04T18:16:54.104038] [INFO ] [MAIN ] CVE: Wiping existing primary data before full refresh. [2025-11-04T18:16:54.124051] [INFO ] [LOCAL ] CVE: Starting deletion of primary data. [2025-11-04T18:16:54.244470] [DEBUG] [LOCAL ] CVE: Deleted primary data file. [2025-11-04T18:16:54.260871] [INFO ] [MAIN ] CVE: Starting full primary refresh. [2025-11-04T18:16:54.588606] [INFO ] [REMOTE ] CVEO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:16:54.668495] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-01. [2025-11-04T18:16:55.094261] [INFO ] [REMOTE ] CURV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:16:55.146023] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-01. [2025-11-04T18:16:55.260956] [INFO ] [REMOTE ] CVE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:16:55.299148] [INFO ] [REMOTE ] CUZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:16:55.300258] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-01. [2025-11-04T18:16:55.300503] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-01. [2025-11-04T18:16:55.346938] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-11-04T18:16:55.349168] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-02. [2025-11-04T18:16:55.996477] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-11-04T18:16:55.999768] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-03. [2025-11-04T18:16:56.303207] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T18:16:56.308599] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-02. [2025-11-04T18:16:56.548813] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T18:16:56.557995] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-02. [2025-11-04T18:16:56.620028] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-11-04T18:16:56.789111] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-11-04T18:16:56.967009] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-02. [2025-11-04T18:16:57.029847] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T18:16:57.728868] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T18:16:57.767006] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-03. [2025-11-04T18:16:58.288024] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-11-04T18:16:58.567037] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-04. [2025-11-04T18:16:58.868136] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T18:16:59.245737] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T18:16:59.367053] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-03. [2025-11-04T18:17:00.176944] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-05. [2025-11-04T18:17:00.179113] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-11-04T18:17:00.967008] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-03. [2025-11-04T18:17:01.043189] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T18:17:01.767109] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-06. [2025-11-04T18:17:02.304029] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T18:17:02.326488] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-11-04T18:17:02.567091] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-07. [2025-11-04T18:17:03.143967] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-11-04T18:17:03.367079] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-04. [2025-11-04T18:17:04.167048] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-04. [2025-11-04T18:17:04.652219] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T18:17:04.890581] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-11-04T18:17:04.967052] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-08. [2025-11-04T18:17:05.709967] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-11-04T18:17:05.767105] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-05. [2025-11-04T18:17:06.567096] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-05. [2025-11-04T18:17:06.627048] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-11-04T18:17:07.367024] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-06. [2025-11-04T18:17:07.883005] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:17:08.125978] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-11-04T18:17:08.167073] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-07. [2025-11-04T18:17:08.967082] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-06. [2025-11-04T18:17:09.153294] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-11-04T18:17:09.767085] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-04. [2025-11-04T18:17:10.102239] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-11-04T18:17:10.567028] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-09. [2025-11-04T18:17:11.156045] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-11-04T18:17:11.286264] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-11-04T18:17:11.367032] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-05. [2025-11-04T18:17:12.167052] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-10. [2025-11-04T18:17:12.802778] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-11-04T18:17:12.859598] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-11-04T18:17:12.967038] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-11. [2025-11-04T18:17:13.540117] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T18:17:13.541161] [INFO ] [REMOTE ] CVEO: Full fetch complete (21364 records). [2025-11-04T18:17:13.541231] [DEBUG] [VALIDATOR ] CVEO: Validating primary data. [2025-11-04T18:17:13.772259] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-06. [2025-11-04T18:17:13.916945] [DEBUG] [VALIDATOR ] CVEO: All expected trading days are present. [2025-11-04T18:17:13.917024] [DEBUG] [VALIDATOR ] CVEO: Primary data validation passed. [2025-11-04T18:17:13.917403] [INFO ] [MAIN ] CVEO: Fetched primary data is valid. [2025-11-04T18:17:13.917443] [DEBUG] [LOCAL ] CVEO: Saving primary data. [2025-11-04T18:17:13.918140] [DEBUG] [LOCAL ] CVEO: Trimmed 21364 records to 15286 (keeping 200 days of data) [2025-11-04T18:17:14.003091] [DEBUG] [LOCAL ] CVEO: Primary data saved (15286 records). [2025-11-04T18:17:14.003226] [INFO ] [MAIN ] CVEO: Full history saved. [2025-11-04T18:17:14.003304] [INFO ] [MAIN ] Update Progress: 561/2367 symbols completed. CVEO took 21.9s. Estimated remaining: 4h 8m 27s. Rate: 7.3 symbols/min [2025-11-04T18:17:14.170218] [INFO ] [MAIN ] CVI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:17:14.210328] [DEBUG] [LOCAL ] CVI: Loaded 46511 existing records from primary storage. [2025-11-04T18:17:14.210615] [INFO ] [MAIN ] CVI: Loaded 46511 existing records. [2025-11-04T18:17:14.384757] [WARN ] [MAIN ] CVI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:17:14.384988] [INFO ] [MAIN ] CVI: Wiping existing primary data before full refresh. [2025-11-04T18:17:14.385058] [INFO ] [LOCAL ] CVI: Starting deletion of primary data. [2025-11-04T18:17:14.385301] [DEBUG] [LOCAL ] CVI: Deleted primary data file. [2025-11-04T18:17:14.385369] [INFO ] [MAIN ] CVI: Starting full primary refresh. [2025-11-04T18:17:14.572220] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-08. [2025-11-04T18:17:14.761442] [INFO ] [REMOTE ] CVI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:17:15.274322] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-11-04T18:17:15.367008] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-07. [2025-11-04T18:17:15.517956] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T18:17:16.167039] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-01. [2025-11-04T18:17:16.618830] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:17:16.967064] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-08. [2025-11-04T18:17:17.489877] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T18:17:17.767028] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-02. [2025-11-04T18:17:18.468466] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-11-04T18:17:18.567088] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-09. [2025-11-04T18:17:18.817533] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T18:17:19.367050] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-09. [2025-11-04T18:17:19.985293] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T18:17:20.167010] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-10. [2025-11-04T18:17:20.490234] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T18:17:20.967040] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-07. [2025-11-04T18:17:21.767017] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-03. [2025-11-04T18:17:21.813055] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-11-04T18:17:22.398266] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-11-04T18:17:22.567057] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-08. [2025-11-04T18:17:23.090278] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T18:17:23.367060] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-10. [2025-11-04T18:17:23.831916] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T18:17:24.167041] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-11. [2025-11-04T18:17:24.513304] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-11 (0.35 seconds). [2025-11-04T18:17:24.514794] [INFO ] [REMOTE ] CVE: Full fetch complete (77320 records). [2025-11-04T18:17:24.514891] [DEBUG] [VALIDATOR ] CVE: Validating primary data. [2025-11-04T18:17:24.515661] [FATAL] [VALIDATOR ] CVE: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:17:24.515786] [FATAL] [MAIN ] CVE: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:17:24.515828] [WARN ] [MAIN ] CVE: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:17:24.515866] [INFO ] [LOCAL ] CVE: Starting deletion of primary data. [2025-11-04T18:17:24.515933] [DEBUG] [LOCAL ] CVE: No primary data file found to delete. [2025-11-04T18:17:24.515967] [INFO ] [LOCAL ] CVE: Starting deletion of derivative data. [2025-11-04T18:17:24.516140] [DEBUG] [LOCAL ] CVE: Deleted derivative data file. [2025-11-04T18:17:24.516227] [INFO ] [MAIN ] Update Progress: 562/2367 symbols completed. CVE took 32.4s. Estimated remaining: 4h 8m 26s. Rate: 7.3 symbols/min [2025-11-04T18:17:24.687628] [INFO ] [MAIN ] CVLG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:17:24.703511] [DEBUG] [LOCAL ] CVLG: Loaded 15090 existing records from primary storage. [2025-11-04T18:17:24.703747] [INFO ] [MAIN ] CVLG: Loaded 15090 existing records. [2025-11-04T18:17:24.885677] [WARN ] [MAIN ] CVLG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:17:24.885939] [INFO ] [MAIN ] CVLG: Wiping existing primary data before full refresh. [2025-11-04T18:17:24.886159] [INFO ] [LOCAL ] CVLG: Starting deletion of primary data. [2025-11-04T18:17:24.886438] [DEBUG] [LOCAL ] CVLG: Deleted primary data file. [2025-11-04T18:17:24.886497] [INFO ] [MAIN ] CVLG: Starting full primary refresh. [2025-11-04T18:17:24.970468] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-04. [2025-11-04T18:17:25.014146] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-11-04T18:17:25.271571] [INFO ] [REMOTE ] CVLG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:17:25.767073] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-11. [2025-11-04T18:17:26.374082] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T18:17:26.567092] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-01. [2025-11-04T18:17:26.650835] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-11 (0.88 seconds). [2025-11-04T18:17:26.652373] [INFO ] [REMOTE ] CURV: Full fetch complete (39120 records). [2025-11-04T18:17:26.652469] [DEBUG] [VALIDATOR ] CURV: Validating primary data. [2025-11-04T18:17:27.036469] [DEBUG] [VALIDATOR ] CURV: All expected trading days are present. [2025-11-04T18:17:27.036545] [DEBUG] [VALIDATOR ] CURV: Primary data validation passed. [2025-11-04T18:17:27.037099] [INFO ] [MAIN ] CURV: Fetched primary data is valid. [2025-11-04T18:17:27.037160] [DEBUG] [LOCAL ] CURV: Saving primary data. [2025-11-04T18:17:27.037986] [DEBUG] [LOCAL ] CURV: Trimmed 39120 records to 27677 (keeping 200 days of data) [2025-11-04T18:17:27.184984] [DEBUG] [LOCAL ] CURV: Primary data saved (27677 records). [2025-11-04T18:17:27.185110] [INFO ] [MAIN ] CURV: Full history saved. [2025-11-04T18:17:27.185184] [INFO ] [MAIN ] Update Progress: 563/2367 symbols completed. CURV took 35.1s. Estimated remaining: 4h 8m 0s. Rate: 7.3 symbols/min [2025-11-04T18:17:27.354526] [INFO ] [MAIN ] CVNA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:17:27.365011] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-11-04T18:17:27.368562] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-05. [2025-11-04T18:17:27.414085] [DEBUG] [LOCAL ] CVNA: Loaded 52816 existing records from primary storage. [2025-11-04T18:17:27.414373] [INFO ] [MAIN ] CVNA: Loaded 52816 existing records. [2025-11-04T18:17:27.587225] [WARN ] [MAIN ] CVNA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:17:27.587452] [INFO ] [MAIN ] CVNA: Wiping existing primary data before full refresh. [2025-11-04T18:17:27.587503] [INFO ] [LOCAL ] CVNA: Starting deletion of primary data. [2025-11-04T18:17:27.587741] [DEBUG] [LOCAL ] CVNA: Deleted primary data file. [2025-11-04T18:17:27.587796] [INFO ] [MAIN ] CVNA: Starting full primary refresh. [2025-11-04T18:17:27.959497] [INFO ] [REMOTE ] CVNA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:17:28.167031] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-01. [2025-11-04T18:17:28.627041] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T18:17:28.967074] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-09. [2025-11-04T18:17:29.767028] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-02. [2025-11-04T18:17:30.066572] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-11-04T18:17:30.266442] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T18:17:30.319042] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T18:17:30.567032] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-03. [2025-11-04T18:17:31.143578] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-11-04T18:17:31.367029] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-02. [2025-11-04T18:17:32.167049] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-06. [2025-11-04T18:17:32.885960] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-11-04T18:17:32.967030] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-10. [2025-11-04T18:17:33.297469] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T18:17:33.767081] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-04. [2025-11-04T18:17:34.531525] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-11-04T18:17:34.567027] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-05. [2025-11-04T18:17:35.367050] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-03. [2025-11-04T18:17:35.516205] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-10 (2.55 seconds). [2025-11-04T18:17:35.766556] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T18:17:36.167260] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-07. [2025-11-04T18:17:36.967009] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-11. [2025-11-04T18:17:37.585063] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-11-04T18:17:37.746419] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-03 (2.38 seconds). [2025-11-04T18:17:37.767051] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-08. [2025-11-04T18:17:37.959267] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T18:17:37.961343] [INFO ] [REMOTE ] CUZ: Full fetch complete (70969 records). [2025-11-04T18:17:37.961465] [DEBUG] [VALIDATOR ] CUZ: Validating primary data. [2025-11-04T18:17:38.346596] [DEBUG] [VALIDATOR ] CUZ: All expected trading days are present. [2025-11-04T18:17:38.346710] [DEBUG] [VALIDATOR ] CUZ: Primary data validation passed. [2025-11-04T18:17:38.347559] [INFO ] [MAIN ] CUZ: Fetched primary data is valid. [2025-11-04T18:17:38.347622] [DEBUG] [LOCAL ] CUZ: Saving primary data. [2025-11-04T18:17:38.349056] [DEBUG] [LOCAL ] CUZ: Trimmed 70969 records to 49411 (keeping 200 days of data) [2025-11-04T18:17:38.567697] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-04. [2025-11-04T18:17:38.650908] [DEBUG] [LOCAL ] CUZ: Primary data saved (49411 records). [2025-11-04T18:17:38.651053] [INFO ] [MAIN ] CUZ: Full history saved. [2025-11-04T18:17:38.651136] [INFO ] [MAIN ] Update Progress: 564/2367 symbols completed. CUZ took 46.6s. Estimated remaining: 4h 8m 2s. Rate: 7.3 symbols/min [2025-11-04T18:17:38.817102] [INFO ] [MAIN ] CVS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:17:38.869012] [DEBUG] [LOCAL ] CVS: Loaded 53694 existing records from primary storage. [2025-11-04T18:17:38.869985] [INFO ] [MAIN ] CVS: Loaded 53694 existing records. [2025-11-04T18:17:39.046955] [WARN ] [MAIN ] CVS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:17:39.047277] [INFO ] [MAIN ] CVS: Wiping existing primary data before full refresh. [2025-11-04T18:17:39.047336] [INFO ] [LOCAL ] CVS: Starting deletion of primary data. [2025-11-04T18:17:39.048063] [DEBUG] [LOCAL ] CVS: Deleted primary data file. [2025-11-04T18:17:39.048122] [INFO ] [MAIN ] CVS: Starting full primary refresh. [2025-11-04T18:17:39.371951] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-06. [2025-11-04T18:17:39.461351] [INFO ] [REMOTE ] CVS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:17:39.496099] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-11-04T18:17:40.167040] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-09. [2025-11-04T18:17:40.353275] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T18:17:40.663320] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-11-04T18:17:40.967035] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-07. [2025-11-04T18:17:41.584445] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T18:17:41.607477] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-11-04T18:17:41.767024] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-10. [2025-11-04T18:17:42.567141] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-08. [2025-11-04T18:17:43.036625] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T18:17:43.061194] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-11-04T18:17:43.367092] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-09. [2025-11-04T18:17:43.964190] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-11-04T18:17:44.167078] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-05. [2025-11-04T18:17:44.967062] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-11. [2025-11-04T18:17:45.767037] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-01. [2025-11-04T18:17:45.962946] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-11-04T18:17:46.158117] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T18:17:46.160090] [INFO ] [REMOTE ] CVI: Full fetch complete (67264 records). [2025-11-04T18:17:46.160232] [DEBUG] [VALIDATOR ] CVI: Validating primary data. [2025-11-04T18:17:46.567071] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-06. [2025-11-04T18:17:46.585821] [DEBUG] [VALIDATOR ] CVI: All expected trading days are present. [2025-11-04T18:17:46.585908] [DEBUG] [VALIDATOR ] CVI: Primary data validation passed. [2025-11-04T18:17:46.587354] [INFO ] [MAIN ] CVI: Fetched primary data is valid. [2025-11-04T18:17:46.587423] [DEBUG] [LOCAL ] CVI: Saving primary data. [2025-11-04T18:17:46.589550] [DEBUG] [LOCAL ] CVI: Trimmed 67264 records to 46551 (keeping 200 days of data) [2025-11-04T18:17:46.883416] [DEBUG] [LOCAL ] CVI: Primary data saved (46551 records). [2025-11-04T18:17:46.883636] [INFO ] [MAIN ] CVI: Full history saved. [2025-11-04T18:17:46.883814] [INFO ] [MAIN ] Update Progress: 565/2367 symbols completed. CVI took 32.9s. Estimated remaining: 4h 7m 54s. Rate: 7.3 symbols/min [2025-11-04T18:17:47.059110] [INFO ] [MAIN ] CVX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:17:47.059327] [DEBUG] [LOCAL ] CVX: No existing data file found in primary storage. [2025-11-04T18:17:47.059365] [WARN ] [MAIN ] CVX: No existing data found, performing full refresh instead. [2025-11-04T18:17:47.059482] [INFO ] [MAIN ] CVX: Wiping existing primary data before full refresh. [2025-11-04T18:17:47.059523] [INFO ] [LOCAL ] CVX: Starting deletion of primary data. [2025-11-04T18:17:47.059564] [DEBUG] [LOCAL ] CVX: No primary data file found to delete. [2025-11-04T18:17:47.059595] [INFO ] [MAIN ] CVX: Starting full primary refresh. [2025-11-04T18:17:47.370799] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-10. [2025-11-04T18:17:47.468086] [INFO ] [REMOTE ] CVX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:17:47.501805] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-11-04T18:17:48.086610] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-11-04T18:17:48.167011] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-11. [2025-11-04T18:17:48.296638] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-11-04T18:17:48.708411] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T18:17:48.709533] [INFO ] [REMOTE ] CVLG: Full fetch complete (21534 records). [2025-11-04T18:17:48.709621] [DEBUG] [VALIDATOR ] CVLG: Validating primary data. [2025-11-04T18:17:48.977146] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-07. [2025-11-04T18:17:49.100471] [DEBUG] [VALIDATOR ] CVLG: All expected trading days are present. [2025-11-04T18:17:49.100560] [DEBUG] [VALIDATOR ] CVLG: Primary data validation passed. [2025-11-04T18:17:49.101021] [INFO ] [MAIN ] CVLG: Fetched primary data is valid. [2025-11-04T18:17:49.101086] [DEBUG] [LOCAL ] CVLG: Saving primary data. [2025-11-04T18:17:49.101913] [DEBUG] [LOCAL ] CVLG: Trimmed 21534 records to 14990 (keeping 200 days of data) [2025-11-04T18:17:49.197378] [DEBUG] [LOCAL ] CVLG: Primary data saved (14990 records). [2025-11-04T18:17:49.197517] [INFO ] [MAIN ] CVLG: Full history saved. [2025-11-04T18:17:49.197601] [INFO ] [MAIN ] Update Progress: 566/2367 symbols completed. CVLG took 24.7s. Estimated remaining: 4h 7m 27s. Rate: 7.3 symbols/min [2025-11-04T18:17:49.402033] [INFO ] [MAIN ] CW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:17:49.424798] [DEBUG] [LOCAL ] CW: Loaded 22038 existing records from primary storage. [2025-11-04T18:17:49.425069] [INFO ] [MAIN ] CW: Loaded 22038 existing records. [2025-11-04T18:17:49.633182] [WARN ] [MAIN ] CW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:17:49.633439] [INFO ] [MAIN ] CW: Wiping existing primary data before full refresh. [2025-11-04T18:17:49.633494] [INFO ] [LOCAL ] CW: Starting deletion of primary data. [2025-11-04T18:17:49.633720] [DEBUG] [LOCAL ] CW: Deleted primary data file. [2025-11-04T18:17:49.633777] [INFO ] [MAIN ] CW: Starting full primary refresh. [2025-11-04T18:17:49.772187] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-02. [2025-11-04T18:17:50.059283] [INFO ] [REMOTE ] CW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:17:50.567028] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-01. [2025-11-04T18:17:51.136961] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-11-04T18:17:51.272317] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-11-04T18:17:51.367006] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-08. [2025-11-04T18:17:51.560005] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T18:17:52.167100] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-02. [2025-11-04T18:17:52.967019] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-03. [2025-11-04T18:17:53.080935] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-11-04T18:17:53.157878] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T18:17:53.767082] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-09. [2025-11-04T18:17:54.546726] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-11-04T18:17:54.567061] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-04. [2025-11-04T18:17:55.367106] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-03. [2025-11-04T18:17:55.423095] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-11-04T18:17:56.113042] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-11-04T18:17:56.144806] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-11-04T18:17:56.167046] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-05. [2025-11-04T18:17:56.967088] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-10. [2025-11-04T18:17:57.752070] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-11-04T18:17:57.766990] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-06. [2025-11-04T18:17:58.567134] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-01. [2025-11-04T18:17:59.045486] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-11-04T18:17:59.092441] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T18:17:59.367080] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-07. [2025-11-04T18:18:00.167062] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-04. [2025-11-04T18:18:00.306325] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-11-04T18:18:00.967054] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-02. [2025-11-04T18:18:01.087982] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-11-04T18:18:01.143071] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-11-04T18:18:01.767076] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-05. [2025-11-04T18:18:02.567069] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-11. [2025-11-04T18:18:02.733220] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-11-04T18:18:03.370760] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-08. [2025-11-04T18:18:03.378950] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-11-04T18:18:04.097340] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-11 (1.53 seconds). [2025-11-04T18:18:04.099985] [INFO ] [REMOTE ] CVNA: Full fetch complete (76714 records). [2025-11-04T18:18:04.100123] [DEBUG] [VALIDATOR ] CVNA: Validating primary data. [2025-11-04T18:18:04.172244] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-06. [2025-11-04T18:18:04.538202] [DEBUG] [VALIDATOR ] CVNA: All expected trading days are present. [2025-11-04T18:18:04.538319] [DEBUG] [VALIDATOR ] CVNA: Primary data validation passed. [2025-11-04T18:18:04.539354] [INFO ] [MAIN ] CVNA: Fetched primary data is valid. [2025-11-04T18:18:04.539426] [DEBUG] [LOCAL ] CVNA: Saving primary data. [2025-11-04T18:18:04.541490] [DEBUG] [LOCAL ] CVNA: Trimmed 76714 records to 52823 (keeping 200 days of data) [2025-11-04T18:18:04.891101] [DEBUG] [LOCAL ] CVNA: Primary data saved (52823 records). [2025-11-04T18:18:04.891315] [INFO ] [MAIN ] CVNA: Full history saved. [2025-11-04T18:18:04.891424] [INFO ] [MAIN ] Update Progress: 567/2367 symbols completed. CVNA took 37.7s. Estimated remaining: 4h 7m 42s. Rate: 7.3 symbols/min [2025-11-04T18:18:04.967636] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-03. [2025-11-04T18:18:05.101545] [INFO ] [MAIN ] CWAN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:18:05.122948] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-11-04T18:18:05.162905] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-11-04T18:18:05.221194] [DEBUG] [LOCAL ] CWAN: Loaded 52688 existing records from primary storage. [2025-11-04T18:18:05.221676] [INFO ] [MAIN ] CWAN: Loaded 52688 existing records. [2025-11-04T18:18:05.416915] [WARN ] [MAIN ] CWAN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:18:05.417257] [INFO ] [MAIN ] CWAN: Wiping existing primary data before full refresh. [2025-11-04T18:18:05.417316] [INFO ] [LOCAL ] CWAN: Starting deletion of primary data. [2025-11-04T18:18:05.417553] [DEBUG] [LOCAL ] CWAN: Deleted primary data file. [2025-11-04T18:18:05.417592] [INFO ] [MAIN ] CWAN: Starting full primary refresh. [2025-11-04T18:18:05.772427] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-09. [2025-11-04T18:18:05.811374] [INFO ] [REMOTE ] CWAN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:18:06.567065] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-01. [2025-11-04T18:18:07.103224] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-11-04T18:18:07.172079] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T18:18:07.367069] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-04. [2025-11-04T18:18:07.799712] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T18:18:08.167043] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-02. [2025-11-04T18:18:08.967008] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-07. [2025-11-04T18:18:09.305770] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T18:18:09.766218] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-11-04T18:18:09.771204] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-10. [2025-11-04T18:18:09.780966] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-04 (2.41 seconds). [2025-11-04T18:18:10.567051] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-03. [2025-11-04T18:18:11.367077] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-05. [2025-11-04T18:18:11.453894] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-11-04T18:18:11.688023] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T18:18:12.167037] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-11. [2025-11-04T18:18:12.412192] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-11 (0.25 seconds). [2025-11-04T18:18:12.414211] [INFO ] [REMOTE ] CVS: Full fetch complete (77874 records). [2025-11-04T18:18:12.414310] [DEBUG] [VALIDATOR ] CVS: Validating primary data. [2025-11-04T18:18:12.814283] [DEBUG] [VALIDATOR ] CVS: All expected trading days are present. [2025-11-04T18:18:12.814376] [DEBUG] [VALIDATOR ] CVS: Primary data validation passed. [2025-11-04T18:18:12.815212] [INFO ] [MAIN ] CVS: Fetched primary data is valid. [2025-11-04T18:18:12.815314] [DEBUG] [LOCAL ] CVS: Saving primary data. [2025-11-04T18:18:12.816883] [DEBUG] [LOCAL ] CVS: Trimmed 77874 records to 53694 (keeping 200 days of data) [2025-11-04T18:18:12.994905] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-04. [2025-11-04T18:18:13.150347] [DEBUG] [LOCAL ] CVS: Primary data saved (53694 records). [2025-11-04T18:18:13.150472] [INFO ] [MAIN ] CVS: Full history saved. [2025-11-04T18:18:13.151040] [INFO ] [MAIN ] Update Progress: 568/2367 symbols completed. CVS took 34.5s. Estimated remaining: 4h 7m 34s. Rate: 7.3 symbols/min [2025-11-04T18:18:13.337517] [INFO ] [MAIN ] CWEN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:18:13.337738] [DEBUG] [LOCAL ] CWEN: No existing data file found in primary storage. [2025-11-04T18:18:13.337808] [WARN ] [MAIN ] CWEN: No existing data found, performing full refresh instead. [2025-11-04T18:18:13.337976] [INFO ] [MAIN ] CWEN: Wiping existing primary data before full refresh. [2025-11-04T18:18:13.338106] [INFO ] [LOCAL ] CWEN: Starting deletion of primary data. [2025-11-04T18:18:13.338247] [DEBUG] [LOCAL ] CWEN: No primary data file found to delete. [2025-11-04T18:18:13.338370] [INFO ] [MAIN ] CWEN: Starting full primary refresh. [2025-11-04T18:18:13.742582] [INFO ] [REMOTE ] CWEN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:18:13.747527] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-05 (2.38 seconds). [2025-11-04T18:18:13.767007] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-01. [2025-11-04T18:18:14.200113] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T18:18:14.567072] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-08. [2025-11-04T18:18:15.072757] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T18:18:15.316482] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T18:18:15.367020] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-09. [2025-11-04T18:18:16.167055] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-06. [2025-11-04T18:18:16.315404] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-11-04T18:18:16.967012] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-05. [2025-11-04T18:18:17.767040] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-02. [2025-11-04T18:18:18.012154] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-11-04T18:18:18.320787] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-11-04T18:18:18.567044] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-07. [2025-11-04T18:18:18.744080] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-11-04T18:18:19.367041] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-10. [2025-11-04T18:18:20.167002] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-03. [2025-11-04T18:18:20.206751] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-11-04T18:18:20.300619] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-11-04T18:18:20.967041] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-06. [2025-11-04T18:18:21.353475] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T18:18:21.767041] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-11. [2025-11-04T18:18:22.295771] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T18:18:22.563514] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-11 (0.80 seconds). [2025-11-04T18:18:22.565121] [INFO ] [REMOTE ] CW: Full fetch complete (33476 records). [2025-11-04T18:18:22.565335] [DEBUG] [VALIDATOR ] CW: Validating primary data. [2025-11-04T18:18:22.576886] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-08. [2025-11-04T18:18:22.941907] [DEBUG] [VALIDATOR ] CW: All expected trading days are present. [2025-11-04T18:18:22.941986] [DEBUG] [VALIDATOR ] CW: Primary data validation passed. [2025-11-04T18:18:22.942458] [INFO ] [MAIN ] CW: Fetched primary data is valid. [2025-11-04T18:18:22.942497] [DEBUG] [LOCAL ] CW: Saving primary data. [2025-11-04T18:18:22.943325] [DEBUG] [LOCAL ] CW: Trimmed 33476 records to 22229 (keeping 200 days of data) [2025-11-04T18:18:23.077889] [DEBUG] [LOCAL ] CW: Primary data saved (22229 records). [2025-11-04T18:18:23.078007] [INFO ] [MAIN ] CW: Full history saved. [2025-11-04T18:18:23.078080] [INFO ] [MAIN ] Update Progress: 569/2367 symbols completed. CW took 33.9s. Estimated remaining: 4h 7m 31s. Rate: 7.3 symbols/min [2025-11-04T18:18:23.244539] [INFO ] [MAIN ] CWH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:18:23.292007] [DEBUG] [LOCAL ] CWH: Loaded 51221 existing records from primary storage. [2025-11-04T18:18:23.292306] [INFO ] [MAIN ] CWH: Loaded 51221 existing records. [2025-11-04T18:18:23.372149] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-07. [2025-11-04T18:18:23.473086] [WARN ] [MAIN ] CWH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:18:23.473327] [INFO ] [MAIN ] CWH: Wiping existing primary data before full refresh. [2025-11-04T18:18:23.473389] [INFO ] [LOCAL ] CWH: Starting deletion of primary data. [2025-11-04T18:18:23.473568] [DEBUG] [LOCAL ] CWH: Deleted primary data file. [2025-11-04T18:18:23.473606] [INFO ] [MAIN ] CWH: Starting full primary refresh. [2025-11-04T18:18:23.816801] [INFO ] [REMOTE ] CWH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:18:24.167010] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-04. [2025-11-04T18:18:24.407350] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-11-04T18:18:24.727151] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T18:18:24.967036] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-08. [2025-11-04T18:18:25.612588] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T18:18:25.767028] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-05. [2025-11-04T18:18:26.332272] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-11-04T18:18:26.567038] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-09. [2025-11-04T18:18:26.825006] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T18:18:27.367037] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-01. [2025-11-04T18:18:27.803242] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-11-04T18:18:28.167101] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-06. [2025-11-04T18:18:28.731390] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T18:18:28.967039] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-10. [2025-11-04T18:18:29.235505] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T18:18:29.767098] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-07. [2025-11-04T18:18:30.489119] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-11-04T18:18:30.567044] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-11. [2025-11-04T18:18:30.971080] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T18:18:31.367039] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-02. [2025-11-04T18:18:31.574611] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-11 (1.01 seconds). [2025-11-04T18:18:31.576744] [INFO ] [REMOTE ] CWAN: Full fetch complete (75289 records). [2025-11-04T18:18:31.576988] [DEBUG] [VALIDATOR ] CWAN: Validating primary data. [2025-11-04T18:18:31.577906] [FATAL] [VALIDATOR ] CWAN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:18:31.578005] [FATAL] [MAIN ] CWAN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:18:31.578139] [WARN ] [MAIN ] CWAN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:18:31.578218] [INFO ] [LOCAL ] CWAN: Starting deletion of primary data. [2025-11-04T18:18:31.578295] [DEBUG] [LOCAL ] CWAN: No primary data file found to delete. [2025-11-04T18:18:31.578335] [INFO ] [LOCAL ] CWAN: Starting deletion of derivative data. [2025-11-04T18:18:31.578508] [DEBUG] [LOCAL ] CWAN: Deleted derivative data file. [2025-11-04T18:18:31.578614] [INFO ] [MAIN ] Update Progress: 570/2367 symbols completed. CWAN took 26.7s. Estimated remaining: 4h 7m 23s. Rate: 7.3 symbols/min [2025-11-04T18:18:31.765136] [INFO ] [MAIN ] CWK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:18:31.814886] [DEBUG] [LOCAL ] CWK: Loaded 49584 existing records from primary storage. [2025-11-04T18:18:31.815232] [INFO ] [MAIN ] CWK: Loaded 49584 existing records. [2025-11-04T18:18:32.025509] [WARN ] [MAIN ] CWK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:18:32.025765] [INFO ] [MAIN ] CWK: Wiping existing primary data before full refresh. [2025-11-04T18:18:32.025827] [INFO ] [LOCAL ] CWK: Starting deletion of primary data. [2025-11-04T18:18:32.026060] [DEBUG] [LOCAL ] CWK: Deleted primary data file. [2025-11-04T18:18:32.026124] [INFO ] [MAIN ] CWK: Starting full primary refresh. [2025-11-04T18:18:32.172241] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-09. [2025-11-04T18:18:32.465372] [INFO ] [REMOTE ] CWK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:18:32.510197] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T18:18:32.967043] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-03. [2025-11-04T18:18:33.767042] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-08. [2025-11-04T18:18:34.231228] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-11-04T18:18:34.272377] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T18:18:34.567042] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-10. [2025-11-04T18:18:35.104526] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T18:18:35.367045] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-09. [2025-11-04T18:18:36.167023] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-01. [2025-11-04T18:18:36.590836] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T18:18:36.708632] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-10 (2.14 seconds). [2025-11-04T18:18:36.967059] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-11. [2025-11-04T18:18:37.324385] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T18:18:37.340372] [INFO ] [REMOTE ] CVX: Full fetch complete (77940 records). [2025-11-04T18:18:37.349747] [DEBUG] [VALIDATOR ] CVX: Validating primary data. [2025-11-04T18:18:37.351355] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T18:18:37.763478] [DEBUG] [VALIDATOR ] CVX: All expected trading days are present. [2025-11-04T18:18:37.763551] [DEBUG] [VALIDATOR ] CVX: Primary data validation passed. [2025-11-04T18:18:37.764372] [INFO ] [MAIN ] CVX: Fetched primary data is valid. [2025-11-04T18:18:37.764433] [DEBUG] [LOCAL ] CVX: Saving primary data. [2025-11-04T18:18:37.765821] [DEBUG] [LOCAL ] CVX: Trimmed 77940 records to 53760 (keeping 200 days of data) [2025-11-04T18:18:37.767084] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-04. [2025-11-04T18:18:38.090401] [DEBUG] [LOCAL ] CVX: Primary data saved (53760 records). [2025-11-04T18:18:38.090526] [INFO ] [MAIN ] CVX: Full history saved. [2025-11-04T18:18:38.090594] [INFO ] [MAIN ] Update Progress: 571/2367 symbols completed. CVX took 51.2s. Estimated remaining: 4h 7m 10s. Rate: 7.3 symbols/min [2025-11-04T18:18:38.255399] [INFO ] [MAIN ] CWT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:18:38.280964] [DEBUG] [LOCAL ] CWT: Loaded 27723 existing records from primary storage. [2025-11-04T18:18:38.281229] [INFO ] [MAIN ] CWT: Loaded 27723 existing records. [2025-11-04T18:18:38.457149] [WARN ] [MAIN ] CWT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:18:38.457352] [INFO ] [MAIN ] CWT: Wiping existing primary data before full refresh. [2025-11-04T18:18:38.457398] [INFO ] [LOCAL ] CWT: Starting deletion of primary data. [2025-11-04T18:18:38.457563] [DEBUG] [LOCAL ] CWT: Deleted primary data file. [2025-11-04T18:18:38.457600] [INFO ] [MAIN ] CWT: Starting full primary refresh. [2025-11-04T18:18:38.572221] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-10. [2025-11-04T18:18:38.817693] [INFO ] [REMOTE ] CWT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:18:39.255720] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-11-04T18:18:39.366999] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-01. [2025-11-04T18:18:39.901788] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-11-04T18:18:40.167084] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-05. [2025-11-04T18:18:40.422107] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T18:18:40.967055] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-02. [2025-11-04T18:18:41.623871] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T18:18:41.767113] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-11. [2025-11-04T18:18:42.016714] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T18:18:42.567102] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-02. [2025-11-04T18:18:42.738250] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T18:18:42.739740] [INFO ] [REMOTE ] CWEN: Full fetch complete (65587 records). [2025-11-04T18:18:42.739849] [DEBUG] [VALIDATOR ] CWEN: Validating primary data. [2025-11-04T18:18:42.740539] [FATAL] [VALIDATOR ] CWEN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:18:42.740670] [FATAL] [MAIN ] CWEN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:18:42.740746] [WARN ] [MAIN ] CWEN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:18:42.740858] [INFO ] [LOCAL ] CWEN: Starting deletion of primary data. [2025-11-04T18:18:42.740935] [DEBUG] [LOCAL ] CWEN: No primary data file found to delete. [2025-11-04T18:18:42.740972] [INFO ] [LOCAL ] CWEN: Starting deletion of derivative data. [2025-11-04T18:18:42.741011] [DEBUG] [LOCAL ] CWEN: No derivative data file found to delete. [2025-11-04T18:18:42.741064] [INFO ] [MAIN ] Update Progress: 572/2367 symbols completed. CWEN took 29.6s. Estimated remaining: 4h 6m 50s. Rate: 7.3 symbols/min [2025-11-04T18:18:42.915395] [INFO ] [MAIN ] CX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:18:42.915586] [DEBUG] [LOCAL ] CX: No existing data file found in primary storage. [2025-11-04T18:18:42.915624] [WARN ] [MAIN ] CX: No existing data found, performing full refresh instead. [2025-11-04T18:18:42.915787] [INFO ] [MAIN ] CX: Wiping existing primary data before full refresh. [2025-11-04T18:18:42.915854] [INFO ] [LOCAL ] CX: Starting deletion of primary data. [2025-11-04T18:18:42.915960] [DEBUG] [LOCAL ] CX: No primary data file found to delete. [2025-11-04T18:18:42.916014] [INFO ] [MAIN ] CX: Starting full primary refresh. [2025-11-04T18:18:43.279373] [INFO ] [REMOTE ] CX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:18:43.304948] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-11-04T18:18:43.367038] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-01. [2025-11-04T18:18:44.167071] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-06. [2025-11-04T18:18:44.942150] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-11-04T18:18:44.967000] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-02. [2025-11-04T18:18:45.331431] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T18:18:45.767073] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-03. [2025-11-04T18:18:46.373808] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-11-04T18:18:46.567069] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-03. [2025-11-04T18:18:46.926370] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-11-04T18:18:47.367058] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-04. [2025-11-04T18:18:47.871605] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T18:18:48.167049] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-04. [2025-11-04T18:18:48.729329] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T18:18:48.967036] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-05. [2025-11-04T18:18:49.723499] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T18:18:49.767019] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-05. [2025-11-04T18:18:50.111156] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T18:18:50.567031] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-07. [2025-11-04T18:18:51.001003] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T18:18:51.367034] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-03. [2025-11-04T18:18:51.789752] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-11-04T18:18:52.167010] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-06. [2025-11-04T18:18:52.418714] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-11-04T18:18:52.967020] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-08. [2025-11-04T18:18:53.167841] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T18:18:53.767070] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-06. [2025-11-04T18:18:54.149508] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-11-04T18:18:54.567078] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-07. [2025-11-04T18:18:55.064209] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T18:18:55.367061] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-04. [2025-11-04T18:18:56.071172] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-11-04T18:18:56.167004] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-08. [2025-11-04T18:18:56.320718] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-11-04T18:18:56.967048] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-07. [2025-11-04T18:18:57.370791] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T18:18:57.767061] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-05. [2025-11-04T18:18:58.216921] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:18:58.460607] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-11-04T18:18:58.567019] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-06. [2025-11-04T18:18:59.367055] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-08. [2025-11-04T18:18:59.452980] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T18:19:00.167096] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-07. [2025-11-04T18:19:00.838059] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-11-04T18:19:00.967016] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-09. [2025-11-04T18:19:01.114281] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-11-04T18:19:01.767099] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-08. [2025-11-04T18:19:02.196104] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T18:19:02.567154] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-10. [2025-11-04T18:19:02.766740] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T18:19:03.367016] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-09. [2025-11-04T18:19:03.946314] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T18:19:04.167005] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-11. [2025-11-04T18:19:04.201965] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-11-04T18:19:04.967107] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-10. [2025-11-04T18:19:05.106531] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T18:19:05.108061] [INFO ] [REMOTE ] CWH: Full fetch complete (73052 records). [2025-11-04T18:19:05.108174] [DEBUG] [VALIDATOR ] CWH: Validating primary data. [2025-11-04T18:19:05.109087] [FATAL] [VALIDATOR ] CWH: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:19:05.109175] [FATAL] [MAIN ] CWH: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:19:05.109220] [WARN ] [MAIN ] CWH: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:19:05.109286] [INFO ] [LOCAL ] CWH: Starting deletion of primary data. [2025-11-04T18:19:05.109350] [DEBUG] [LOCAL ] CWH: No primary data file found to delete. [2025-11-04T18:19:05.109386] [INFO ] [LOCAL ] CWH: Starting deletion of derivative data. [2025-11-04T18:19:05.109534] [DEBUG] [LOCAL ] CWH: Deleted derivative data file. [2025-11-04T18:19:05.109625] [INFO ] [MAIN ] Update Progress: 573/2367 symbols completed. CWH took 42.0s. Estimated remaining: 4h 7m 26s. Rate: 7.3 symbols/min [2025-11-04T18:19:05.287248] [INFO ] [MAIN ] CXE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:19:05.287538] [DEBUG] [LOCAL ] CXE: No existing data file found in primary storage. [2025-11-04T18:19:05.287588] [WARN ] [MAIN ] CXE: No existing data found, performing full refresh instead. [2025-11-04T18:19:05.287722] [INFO ] [MAIN ] CXE: Wiping existing primary data before full refresh. [2025-11-04T18:19:05.287768] [INFO ] [LOCAL ] CXE: Starting deletion of primary data. [2025-11-04T18:19:05.287809] [DEBUG] [LOCAL ] CXE: No primary data file found to delete. [2025-11-04T18:19:05.287840] [INFO ] [MAIN ] CXE: Starting full primary refresh. [2025-11-04T18:19:05.634382] [INFO ] [REMOTE ] CXE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:19:05.767023] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-01. [2025-11-04T18:19:05.885596] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-11-04T18:19:06.399897] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-11-04T18:19:06.567039] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-11. [2025-11-04T18:19:07.367041] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-09. [2025-11-04T18:19:07.441333] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T18:19:07.442895] [INFO ] [REMOTE ] CWT: Full fetch complete (41588 records). [2025-11-04T18:19:07.442970] [DEBUG] [VALIDATOR ] CWT: Validating primary data. [2025-11-04T18:19:07.829794] [DEBUG] [VALIDATOR ] CWT: All expected trading days are present. [2025-11-04T18:19:07.829871] [DEBUG] [VALIDATOR ] CWT: Primary data validation passed. [2025-11-04T18:19:07.830464] [INFO ] [MAIN ] CWT: Fetched primary data is valid. [2025-11-04T18:19:07.830507] [DEBUG] [LOCAL ] CWT: Saving primary data. [2025-11-04T18:19:07.831400] [DEBUG] [LOCAL ] CWT: Trimmed 41588 records to 27837 (keeping 200 days of data) [2025-11-04T18:19:07.992479] [DEBUG] [LOCAL ] CWT: Primary data saved (27837 records). [2025-11-04T18:19:07.992603] [INFO ] [MAIN ] CWT: Full history saved. [2025-11-04T18:19:07.992833] [INFO ] [MAIN ] Update Progress: 574/2367 symbols completed. CWT took 29.9s. Estimated remaining: 4h 7m 1s. Rate: 7.3 symbols/min [2025-11-04T18:19:08.165677] [INFO ] [MAIN ] CXH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:19:08.167140] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-09. [2025-11-04T18:19:08.173289] [DEBUG] [LOCAL ] CXH: Loaded 4337 existing records from primary storage. [2025-11-04T18:19:08.173610] [INFO ] [MAIN ] CXH: Loaded 4337 existing records. [2025-11-04T18:19:08.355266] [WARN ] [MAIN ] CXH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:19:08.355520] [INFO ] [MAIN ] CXH: Wiping existing primary data before full refresh. [2025-11-04T18:19:08.355568] [INFO ] [LOCAL ] CXH: Starting deletion of primary data. [2025-11-04T18:19:08.355730] [DEBUG] [LOCAL ] CXH: Deleted primary data file. [2025-11-04T18:19:08.355787] [INFO ] [MAIN ] CXH: Starting full primary refresh. [2025-11-04T18:19:08.717021] [INFO ] [REMOTE ] CXH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:19:08.752463] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T18:19:08.967011] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-01. [2025-11-04T18:19:09.416963] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-11-04T18:19:09.608394] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T18:19:09.767065] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-02. [2025-11-04T18:19:10.317237] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T18:19:10.567052] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-10. [2025-11-04T18:19:11.367045] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-10. [2025-11-04T18:19:11.946387] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T18:19:12.167016] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-11. [2025-11-04T18:19:12.885375] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-11-04T18:19:12.967019] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-11. [2025-11-04T18:19:13.303602] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T18:19:13.305077] [INFO ] [REMOTE ] CWK: Full fetch complete (71227 records). [2025-11-04T18:19:13.305172] [DEBUG] [VALIDATOR ] CWK: Validating primary data. [2025-11-04T18:19:13.305921] [FATAL] [VALIDATOR ] CWK: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:19:13.306039] [FATAL] [MAIN ] CWK: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:19:13.306081] [WARN ] [MAIN ] CWK: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:19:13.306156] [INFO ] [LOCAL ] CWK: Starting deletion of primary data. [2025-11-04T18:19:13.306217] [DEBUG] [LOCAL ] CWK: No primary data file found to delete. [2025-11-04T18:19:13.306250] [INFO ] [LOCAL ] CWK: Starting deletion of derivative data. [2025-11-04T18:19:13.306382] [DEBUG] [LOCAL ] CWK: Deleted derivative data file. [2025-11-04T18:19:13.306449] [INFO ] [MAIN ] Update Progress: 575/2367 symbols completed. CWK took 41.7s. Estimated remaining: 4h 6m 44s. Rate: 7.3 symbols/min [2025-11-04T18:19:13.469728] [INFO ] [MAIN ] CXM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:19:13.469922] [DEBUG] [LOCAL ] CXM: No existing data file found in primary storage. [2025-11-04T18:19:13.469959] [WARN ] [MAIN ] CXM: No existing data found, performing full refresh instead. [2025-11-04T18:19:13.470070] [INFO ] [MAIN ] CXM: Wiping existing primary data before full refresh. [2025-11-04T18:19:13.470111] [INFO ] [LOCAL ] CXM: Starting deletion of primary data. [2025-11-04T18:19:13.470154] [DEBUG] [LOCAL ] CXM: No primary data file found to delete. [2025-11-04T18:19:13.470184] [INFO ] [MAIN ] CXM: Starting full primary refresh. [2025-11-04T18:19:13.776794] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-03. [2025-11-04T18:19:13.817946] [INFO ] [REMOTE ] CXM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:19:14.078106] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T18:19:14.079628] [INFO ] [REMOTE ] CX: Full fetch complete (76173 records). [2025-11-04T18:19:14.079775] [DEBUG] [VALIDATOR ] CX: Validating primary data. [2025-11-04T18:19:14.080612] [FATAL] [VALIDATOR ] CX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:19:14.080722] [FATAL] [MAIN ] CX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:19:14.080793] [WARN ] [MAIN ] CX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:19:14.080864] [INFO ] [LOCAL ] CX: Starting deletion of primary data. [2025-11-04T18:19:14.080947] [DEBUG] [LOCAL ] CX: No primary data file found to delete. [2025-11-04T18:19:14.081022] [INFO ] [LOCAL ] CX: Starting deletion of derivative data. [2025-11-04T18:19:14.081108] [DEBUG] [LOCAL ] CX: No derivative data file found to delete. [2025-11-04T18:19:14.081214] [INFO ] [MAIN ] Update Progress: 576/2367 symbols completed. CX took 31.3s. Estimated remaining: 4h 6m 12s. Rate: 7.3 symbols/min [2025-11-04T18:19:14.280498] [INFO ] [MAIN ] CXT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:19:14.280742] [DEBUG] [LOCAL ] CXT: No existing data file found in primary storage. [2025-11-04T18:19:14.280792] [WARN ] [MAIN ] CXT: No existing data found, performing full refresh instead. [2025-11-04T18:19:14.280901] [INFO ] [MAIN ] CXT: Wiping existing primary data before full refresh. [2025-11-04T18:19:14.280945] [INFO ] [LOCAL ] CXT: Starting deletion of primary data. [2025-11-04T18:19:14.281009] [DEBUG] [LOCAL ] CXT: No primary data file found to delete. [2025-11-04T18:19:14.281065] [INFO ] [MAIN ] CXT: Starting full primary refresh. [2025-11-04T18:19:14.483981] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T18:19:14.576606] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-04. [2025-11-04T18:19:14.702638] [INFO ] [REMOTE ] CXT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:19:15.130446] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-11-04T18:19:15.367049] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-02. [2025-11-04T18:19:15.734341] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-11-04T18:19:16.167049] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-01. [2025-11-04T18:19:16.967030] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-03. [2025-11-04T18:19:17.313557] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-11-04T18:19:17.357803] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T18:19:17.767062] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-05. [2025-11-04T18:19:18.405041] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T18:19:18.567073] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-06. [2025-11-04T18:19:19.081625] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T18:19:19.367071] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-07. [2025-11-04T18:19:19.810545] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-11-04T18:19:20.167049] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-02. [2025-11-04T18:19:20.967060] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-04. [2025-11-04T18:19:21.112564] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-11-04T18:19:21.397528] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-11-04T18:19:21.767093] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-08. [2025-11-04T18:19:22.094172] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-11-04T18:19:22.567041] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-03. [2025-11-04T18:19:23.367043] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-05. [2025-11-04T18:19:23.731761] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-11-04T18:19:23.817511] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-11-04T18:19:24.167042] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-04. [2025-11-04T18:19:24.967013] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-09. [2025-11-04T18:19:25.152354] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-11-04T18:19:25.573059] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-11-04T18:19:25.767022] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-10. [2025-11-04T18:19:26.271229] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-11-04T18:19:26.567036] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-01. [2025-11-04T18:19:27.367036] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-11. [2025-11-04T18:19:27.644398] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T18:19:27.929178] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-11 (0.56 seconds). [2025-11-04T18:19:27.929956] [INFO ] [REMOTE ] CXE: Full fetch complete (10987 records). [2025-11-04T18:19:27.930029] [DEBUG] [VALIDATOR ] CXE: Validating primary data. [2025-11-04T18:19:28.172248] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-05. [2025-11-04T18:19:28.287550] [DEBUG] [VALIDATOR ] CXE: All expected trading days are present. [2025-11-04T18:19:28.287634] [DEBUG] [VALIDATOR ] CXE: Primary data validation passed. [2025-11-04T18:19:28.287943] [INFO ] [MAIN ] CXE: Fetched primary data is valid. [2025-11-04T18:19:28.287988] [DEBUG] [LOCAL ] CXE: Saving primary data. [2025-11-04T18:19:28.288394] [DEBUG] [LOCAL ] CXE: Trimmed 10987 records to 6327 (keeping 200 days of data) [2025-11-04T18:19:28.323793] [DEBUG] [LOCAL ] CXE: Primary data saved (6327 records). [2025-11-04T18:19:28.323885] [INFO ] [MAIN ] CXE: Full history saved. [2025-11-04T18:19:28.323950] [INFO ] [MAIN ] Update Progress: 577/2367 symbols completed. CXE took 23.2s. Estimated remaining: 4h 6m 22s. Rate: 7.3 symbols/min [2025-11-04T18:19:28.488359] [INFO ] [MAIN ] CXW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:19:28.526785] [DEBUG] [LOCAL ] CXW: Loaded 43377 existing records from primary storage. [2025-11-04T18:19:28.527094] [INFO ] [MAIN ] CXW: Loaded 43377 existing records. [2025-11-04T18:19:28.700682] [WARN ] [MAIN ] CXW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:19:28.700877] [INFO ] [MAIN ] CXW: Wiping existing primary data before full refresh. [2025-11-04T18:19:28.700935] [INFO ] [LOCAL ] CXW: Starting deletion of primary data. [2025-11-04T18:19:28.701178] [DEBUG] [LOCAL ] CXW: Deleted primary data file. [2025-11-04T18:19:28.701244] [INFO ] [MAIN ] CXW: Starting full primary refresh. [2025-11-04T18:19:28.972273] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-06. [2025-11-04T18:19:29.073437] [INFO ] [REMOTE ] CXW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:19:29.172917] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-11-04T18:19:29.304683] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-11-04T18:19:29.767090] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-07. [2025-11-04T18:19:30.253939] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-11-04T18:19:30.567046] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-06. [2025-11-04T18:19:31.367031] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-08. [2025-11-04T18:19:31.455211] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T18:19:31.944950] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-11-04T18:19:32.167076] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-09. [2025-11-04T18:19:32.620877] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-11-04T18:19:32.967041] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-07. [2025-11-04T18:19:33.767064] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-10. [2025-11-04T18:19:33.888708] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T18:19:34.003120] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-10 (0.24 seconds). [2025-11-04T18:19:34.567041] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-08. [2025-11-04T18:19:35.367018] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-02. [2025-11-04T18:19:35.531731] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-11-04T18:19:36.167042] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-01. [2025-11-04T18:19:36.335681] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T18:19:36.967055] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-03. [2025-11-04T18:19:37.575194] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T18:19:37.767051] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-02. [2025-11-04T18:19:38.157457] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T18:19:38.567034] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-11. [2025-11-04T18:19:38.902071] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T18:19:38.902766] [INFO ] [REMOTE ] CXH: Full fetch complete (5280 records). [2025-11-04T18:19:38.902840] [DEBUG] [VALIDATOR ] CXH: Validating primary data. [2025-11-04T18:19:38.903198] [FATAL] [VALIDATOR ] CXH: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:19:38.903280] [FATAL] [MAIN ] CXH: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:19:38.903323] [WARN ] [MAIN ] CXH: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:19:38.903385] [INFO ] [LOCAL ] CXH: Starting deletion of primary data. [2025-11-04T18:19:38.903445] [DEBUG] [LOCAL ] CXH: No primary data file found to delete. [2025-11-04T18:19:38.903518] [INFO ] [LOCAL ] CXH: Starting deletion of derivative data. [2025-11-04T18:19:38.903693] [DEBUG] [LOCAL ] CXH: Deleted derivative data file. [2025-11-04T18:19:38.903797] [INFO ] [MAIN ] Update Progress: 578/2367 symbols completed. CXH took 30.9s. Estimated remaining: 4h 6m 21s. Rate: 7.3 symbols/min [2025-11-04T18:19:39.098716] [INFO ] [MAIN ] CYD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:19:39.124593] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T18:19:39.153498] [DEBUG] [LOCAL ] CYD: Loaded 26788 existing records from primary storage. [2025-11-04T18:19:39.153776] [INFO ] [MAIN ] CYD: Loaded 26788 existing records. [2025-11-04T18:19:39.331327] [WARN ] [MAIN ] CYD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:19:39.331552] [INFO ] [MAIN ] CYD: Wiping existing primary data before full refresh. [2025-11-04T18:19:39.331601] [INFO ] [LOCAL ] CYD: Starting deletion of primary data. [2025-11-04T18:19:39.331905] [DEBUG] [LOCAL ] CYD: Deleted primary data file. [2025-11-04T18:19:39.331959] [INFO ] [MAIN ] CYD: Starting full primary refresh. [2025-11-04T18:19:39.372347] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-03. [2025-11-04T18:19:39.693314] [INFO ] [REMOTE ] CYD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:19:40.167073] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-09. [2025-11-04T18:19:40.597194] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T18:19:40.967036] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-04. [2025-11-04T18:19:41.350236] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:19:41.767039] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-01. [2025-11-04T18:19:42.047555] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T18:19:42.211670] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-11-04T18:19:42.567059] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-10. [2025-11-04T18:19:43.367045] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-04. [2025-11-04T18:19:43.495806] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-11-04T18:19:44.167084] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-02. [2025-11-04T18:19:44.714307] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T18:19:44.967008] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-11. [2025-11-04T18:19:45.077459] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T18:19:45.766241] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-11 (0.80 seconds). [2025-11-04T18:19:45.767530] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-05. [2025-11-04T18:19:45.769174] [INFO ] [REMOTE ] CXT: Full fetch complete (45416 records). [2025-11-04T18:19:45.769297] [DEBUG] [VALIDATOR ] CXT: Validating primary data. [2025-11-04T18:19:46.144481] [DEBUG] [VALIDATOR ] CXT: All expected trading days are present. [2025-11-04T18:19:46.144564] [DEBUG] [VALIDATOR ] CXT: Primary data validation passed. [2025-11-04T18:19:46.145215] [INFO ] [MAIN ] CXT: Fetched primary data is valid. [2025-11-04T18:19:46.145277] [DEBUG] [LOCAL ] CXT: Saving primary data. [2025-11-04T18:19:46.146213] [DEBUG] [LOCAL ] CXT: Trimmed 45416 records to 30346 (keeping 200 days of data) [2025-11-04T18:19:46.330422] [DEBUG] [LOCAL ] CXT: Primary data saved (30346 records). [2025-11-04T18:19:46.330554] [INFO ] [MAIN ] CXT: Full history saved. [2025-11-04T18:19:46.330626] [INFO ] [MAIN ] Update Progress: 579/2367 symbols completed. CXT took 32.2s. Estimated remaining: 4h 6m 10s. Rate: 7.3 symbols/min [2025-11-04T18:19:46.497574] [INFO ] [MAIN ] CYH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:19:46.542833] [DEBUG] [LOCAL ] CYH: Loaded 47658 existing records from primary storage. [2025-11-04T18:19:46.543156] [INFO ] [MAIN ] CYH: Loaded 47658 existing records. [2025-11-04T18:19:46.573526] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-05. [2025-11-04T18:19:46.723854] [WARN ] [MAIN ] CYH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:19:46.724112] [INFO ] [MAIN ] CYH: Wiping existing primary data before full refresh. [2025-11-04T18:19:46.724160] [INFO ] [LOCAL ] CYH: Starting deletion of primary data. [2025-11-04T18:19:46.724337] [DEBUG] [LOCAL ] CYH: Deleted primary data file. [2025-11-04T18:19:46.724376] [INFO ] [MAIN ] CYH: Starting full primary refresh. [2025-11-04T18:19:47.085911] [INFO ] [REMOTE ] CYH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:19:47.123414] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T18:19:47.367084] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-03. [2025-11-04T18:19:47.943769] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T18:19:48.167079] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-01. [2025-11-04T18:19:48.444969] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T18:19:48.967085] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-06. [2025-11-04T18:19:49.500334] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T18:19:49.767048] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-06. [2025-11-04T18:19:50.394443] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-11-04T18:19:50.567049] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-07. [2025-11-04T18:19:50.921883] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T18:19:51.367064] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-07. [2025-11-04T18:19:51.684094] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-11-04T18:19:52.167091] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-04. [2025-11-04T18:19:52.766869] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T18:19:52.967054] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-02. [2025-11-04T18:19:53.009609] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-11-04T18:19:53.767080] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-08. [2025-11-04T18:19:54.222775] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T18:19:54.567036] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-08. [2025-11-04T18:19:55.159149] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T18:19:55.367077] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-09. [2025-11-04T18:19:55.718873] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-11-04T18:19:56.167060] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-03. [2025-11-04T18:19:56.694594] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T18:19:56.967072] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-09. [2025-11-04T18:19:57.296185] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-11-04T18:19:57.767041] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-04. [2025-11-04T18:19:58.146531] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:19:58.567066] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-10. [2025-11-04T18:19:58.980351] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T18:19:59.367058] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-05. [2025-11-04T18:19:59.709882] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T18:20:00.167011] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-11. [2025-11-04T18:20:00.229737] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-11-04T18:20:00.967076] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-10. [2025-11-04T18:20:01.009597] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T18:20:01.011012] [INFO ] [REMOTE ] CXW: Full fetch complete (64438 records). [2025-11-04T18:20:01.011105] [DEBUG] [VALIDATOR ] CXW: Validating primary data. [2025-11-04T18:20:01.011791] [FATAL] [VALIDATOR ] CXW: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:20:01.011874] [FATAL] [MAIN ] CXW: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:20:01.011916] [WARN ] [MAIN ] CXW: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:20:01.011954] [INFO ] [LOCAL ] CXW: Starting deletion of primary data. [2025-11-04T18:20:01.012032] [DEBUG] [LOCAL ] CXW: No primary data file found to delete. [2025-11-04T18:20:01.012086] [INFO ] [LOCAL ] CXW: Starting deletion of derivative data. [2025-11-04T18:20:01.012493] [DEBUG] [LOCAL ] CXW: Deleted derivative data file. [2025-11-04T18:20:01.012580] [INFO ] [MAIN ] Update Progress: 580/2367 symbols completed. CXW took 32.7s. Estimated remaining: 4h 6m 22s. Rate: 7.3 symbols/min [2025-11-04T18:20:01.195971] [INFO ] [MAIN ] D: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:20:01.246183] [DEBUG] [LOCAL ] D: Loaded 53649 existing records from primary storage. [2025-11-04T18:20:01.246490] [INFO ] [MAIN ] D: Loaded 53649 existing records. [2025-11-04T18:20:01.421414] [WARN ] [MAIN ] D: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:20:01.421629] [INFO ] [MAIN ] D: Wiping existing primary data before full refresh. [2025-11-04T18:20:01.421728] [INFO ] [LOCAL ] D: Starting deletion of primary data. [2025-11-04T18:20:01.421980] [DEBUG] [LOCAL ] D: Deleted primary data file. [2025-11-04T18:20:01.422036] [INFO ] [MAIN ] D: Starting full primary refresh. [2025-11-04T18:20:01.770421] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-05. [2025-11-04T18:20:01.792813] [INFO ] [REMOTE ] D: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:20:02.294930] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-11-04T18:20:02.567046] [INFO ] [REMOTE ] D: Fetching primary data for 2025-01. [2025-11-04T18:20:03.367093] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-11. [2025-11-04T18:20:03.399420] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-11-04T18:20:04.167083] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-06. [2025-11-04T18:20:04.341861] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-11-04T18:20:04.392098] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T18:20:04.393907] [INFO ] [REMOTE ] CXM: Full fetch complete (68431 records). [2025-11-04T18:20:04.394051] [DEBUG] [VALIDATOR ] CXM: Validating primary data. [2025-11-04T18:20:04.824563] [DEBUG] [VALIDATOR ] CXM: All expected trading days are present. [2025-11-04T18:20:04.824680] [DEBUG] [VALIDATOR ] CXM: Primary data validation passed. [2025-11-04T18:20:04.825515] [INFO ] [MAIN ] CXM: Fetched primary data is valid. [2025-11-04T18:20:04.825563] [DEBUG] [LOCAL ] CXM: Saving primary data. [2025-11-04T18:20:04.827270] [DEBUG] [LOCAL ] CXM: Trimmed 68431 records to 46867 (keeping 200 days of data) [2025-11-04T18:20:05.003765] [INFO ] [REMOTE ] D: Fetching primary data for 2025-02. [2025-11-04T18:20:05.113060] [DEBUG] [LOCAL ] CXM: Primary data saved (46867 records). [2025-11-04T18:20:05.113181] [INFO ] [MAIN ] CXM: Full history saved. [2025-11-04T18:20:05.113251] [INFO ] [MAIN ] Update Progress: 581/2367 symbols completed. CXM took 51.8s. Estimated remaining: 4h 6m 1s. Rate: 7.3 symbols/min [2025-11-04T18:20:05.287136] [INFO ] [MAIN ] DAC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:20:05.300863] [DEBUG] [LOCAL ] DAC: Loaded 11694 existing records from primary storage. [2025-11-04T18:20:05.310050] [INFO ] [MAIN ] DAC: Loaded 11694 existing records. [2025-11-04T18:20:05.417040] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-11-04T18:20:05.512861] [WARN ] [MAIN ] DAC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:20:05.513076] [INFO ] [MAIN ] DAC: Wiping existing primary data before full refresh. [2025-11-04T18:20:05.513124] [INFO ] [LOCAL ] DAC: Starting deletion of primary data. [2025-11-04T18:20:05.513274] [DEBUG] [LOCAL ] DAC: Deleted primary data file. [2025-11-04T18:20:05.513311] [INFO ] [MAIN ] DAC: Starting full primary refresh. [2025-11-04T18:20:05.779482] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-07. [2025-11-04T18:20:05.919277] [INFO ] [REMOTE ] DAC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:20:06.309488] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T18:20:06.567036] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-06. [2025-11-04T18:20:07.013929] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T18:20:07.367023] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-11-04T18:20:07.367095] [INFO ] [REMOTE ] D: Fetching primary data for 2025-03. [2025-11-04T18:20:08.167094] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-07. [2025-11-04T18:20:08.695423] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T18:20:08.899611] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-11-04T18:20:08.966996] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-08. [2025-11-04T18:20:09.767019] [INFO ] [REMOTE ] D: Fetching primary data for 2025-04. [2025-11-04T18:20:09.915877] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T18:20:10.567073] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-08. [2025-11-04T18:20:11.209873] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T18:20:11.367046] [INFO ] [REMOTE ] D: Fetching primary data for 2025-05. [2025-11-04T18:20:11.672873] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T18:20:12.167095] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-09. [2025-11-04T18:20:12.766377] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T18:20:12.967023] [INFO ] [REMOTE ] D: Fetching primary data for 2025-06. [2025-11-04T18:20:13.391261] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T18:20:13.767024] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-01. [2025-11-04T18:20:14.303676] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T18:20:14.528255] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-11-04T18:20:14.567041] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-02. [2025-11-04T18:20:15.244767] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-11-04T18:20:15.367018] [INFO ] [REMOTE ] D: Fetching primary data for 2025-07. [2025-11-04T18:20:16.167091] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-03. [2025-11-04T18:20:16.865370] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T18:20:16.901185] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T18:20:16.967038] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-10. [2025-11-04T18:20:17.767049] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-04. [2025-11-04T18:20:18.116233] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-11-04T18:20:18.470255] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T18:20:18.567059] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-05. [2025-11-04T18:20:19.137704] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-11-04T18:20:19.367077] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-11. [2025-11-04T18:20:20.167156] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-09. [2025-11-04T18:20:20.585872] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T18:20:20.587780] [INFO ] [REMOTE ] CYH: Full fetch complete (70114 records). [2025-11-04T18:20:20.587884] [DEBUG] [VALIDATOR ] CYH: Validating primary data. [2025-11-04T18:20:20.968779] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-06. [2025-11-04T18:20:20.985910] [DEBUG] [VALIDATOR ] CYH: All expected trading days are present. [2025-11-04T18:20:20.987087] [DEBUG] [VALIDATOR ] CYH: Primary data validation passed. [2025-11-04T18:20:20.987892] [INFO ] [MAIN ] CYH: Fetched primary data is valid. [2025-11-04T18:20:20.988052] [DEBUG] [LOCAL ] CYH: Saving primary data. [2025-11-04T18:20:20.989397] [DEBUG] [LOCAL ] CYH: Trimmed 70114 records to 47628 (keeping 200 days of data) [2025-11-04T18:20:21.256386] [DEBUG] [LOCAL ] CYH: Primary data saved (47628 records). [2025-11-04T18:20:21.256554] [INFO ] [MAIN ] CYH: Full history saved. [2025-11-04T18:20:21.257174] [INFO ] [MAIN ] Update Progress: 582/2367 symbols completed. CYH took 34.9s. Estimated remaining: 4h 6m 17s. Rate: 7.2 symbols/min [2025-11-04T18:20:21.473971] [INFO ] [MAIN ] DAL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:20:21.545835] [DEBUG] [LOCAL ] DAL: Loaded 53753 existing records from primary storage. [2025-11-04T18:20:21.546496] [INFO ] [MAIN ] DAL: Loaded 53753 existing records. [2025-11-04T18:20:21.742003] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T18:20:21.772918] [INFO ] [REMOTE ] D: Fetching primary data for 2025-08. [2025-11-04T18:20:21.937396] [WARN ] [MAIN ] DAL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:20:21.938361] [INFO ] [MAIN ] DAL: Wiping existing primary data before full refresh. [2025-11-04T18:20:21.938427] [INFO ] [LOCAL ] DAL: Starting deletion of primary data. [2025-11-04T18:20:21.938945] [DEBUG] [LOCAL ] DAL: Deleted primary data file. [2025-11-04T18:20:21.940750] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-11-04T18:20:21.939010] [INFO ] [MAIN ] DAL: Starting full primary refresh. [2025-11-04T18:20:22.424443] [INFO ] [REMOTE ] DAL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:20:22.567068] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-01. [2025-11-04T18:20:23.367023] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-07. [2025-11-04T18:20:23.434771] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-11-04T18:20:23.965049] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T18:20:24.167069] [INFO ] [REMOTE ] D: Fetching primary data for 2025-09. [2025-11-04T18:20:24.345583] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-11-04T18:20:24.967047] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-10. [2025-11-04T18:20:25.497606] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T18:20:25.767074] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-08. [2025-11-04T18:20:26.002817] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-11-04T18:20:26.372708] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-11-04T18:20:26.567046] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-09. [2025-11-04T18:20:27.128356] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T18:20:27.367246] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-11. [2025-11-04T18:20:28.167041] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-02. [2025-11-04T18:20:28.281321] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T18:20:28.282677] [INFO ] [REMOTE ] CYD: Full fetch complete (37202 records). [2025-11-04T18:20:28.282774] [DEBUG] [VALIDATOR ] CYD: Validating primary data. [2025-11-04T18:20:28.283268] [FATAL] [VALIDATOR ] CYD: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:20:28.283355] [FATAL] [MAIN ] CYD: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:20:28.283397] [WARN ] [MAIN ] CYD: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:20:28.283434] [INFO ] [LOCAL ] CYD: Starting deletion of primary data. [2025-11-04T18:20:28.283487] [DEBUG] [LOCAL ] CYD: No primary data file found to delete. [2025-11-04T18:20:28.283521] [INFO ] [LOCAL ] CYD: Starting deletion of derivative data. [2025-11-04T18:20:28.283715] [DEBUG] [LOCAL ] CYD: Deleted derivative data file. [2025-11-04T18:20:28.283804] [INFO ] [MAIN ] Update Progress: 583/2367 symbols completed. CYD took 49.4s. Estimated remaining: 4h 6m 5s. Rate: 7.2 symbols/min [2025-11-04T18:20:28.456460] [INFO ] [MAIN ] DAN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:20:28.501023] [DEBUG] [LOCAL ] DAN: Loaded 49351 existing records from primary storage. [2025-11-04T18:20:28.501306] [INFO ] [MAIN ] DAN: Loaded 49351 existing records. [2025-11-04T18:20:28.681236] [WARN ] [MAIN ] DAN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:20:28.681441] [INFO ] [MAIN ] DAN: Wiping existing primary data before full refresh. [2025-11-04T18:20:28.681494] [INFO ] [LOCAL ] DAN: Starting deletion of primary data. [2025-11-04T18:20:28.681709] [DEBUG] [LOCAL ] DAN: Deleted primary data file. [2025-11-04T18:20:28.681764] [INFO ] [MAIN ] DAN: Starting full primary refresh. [2025-11-04T18:20:28.972166] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-10. [2025-11-04T18:20:29.036150] [INFO ] [REMOTE ] DAN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:20:29.560959] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-11-04T18:20:29.767028] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-01. [2025-11-04T18:20:30.046362] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-11-04T18:20:30.567057] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-03. [2025-11-04T18:20:31.032804] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T18:20:31.367065] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-11. [2025-11-04T18:20:31.870167] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T18:20:31.871460] [INFO ] [REMOTE ] DAC: Full fetch complete (18536 records). [2025-11-04T18:20:31.871528] [DEBUG] [VALIDATOR ] DAC: Validating primary data. [2025-11-04T18:20:32.172174] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-02. [2025-11-04T18:20:32.235728] [DEBUG] [VALIDATOR ] DAC: All expected trading days are present. [2025-11-04T18:20:32.236044] [DEBUG] [VALIDATOR ] DAC: Primary data validation passed. [2025-11-04T18:20:32.236434] [INFO ] [MAIN ] DAC: Fetched primary data is valid. [2025-11-04T18:20:32.236474] [DEBUG] [LOCAL ] DAC: Saving primary data. [2025-11-04T18:20:32.237044] [DEBUG] [LOCAL ] DAC: Trimmed 18536 records to 11691 (keeping 200 days of data) [2025-11-04T18:20:32.306734] [DEBUG] [LOCAL ] DAC: Primary data saved (11691 records). [2025-11-04T18:20:32.306863] [INFO ] [MAIN ] DAC: Full history saved. [2025-11-04T18:20:32.306938] [INFO ] [MAIN ] Update Progress: 584/2367 symbols completed. DAC took 27.2s. Estimated remaining: 4h 5m 43s. Rate: 7.3 symbols/min [2025-11-04T18:20:32.477735] [INFO ] [MAIN ] DAO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:20:32.509963] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-11-04T18:20:32.532753] [DEBUG] [LOCAL ] DAO: Loaded 19646 existing records from primary storage. [2025-11-04T18:20:32.532988] [INFO ] [MAIN ] DAO: Loaded 19646 existing records. [2025-11-04T18:20:32.706109] [WARN ] [MAIN ] DAO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:20:32.706320] [INFO ] [MAIN ] DAO: Wiping existing primary data before full refresh. [2025-11-04T18:20:32.706366] [INFO ] [LOCAL ] DAO: Starting deletion of primary data. [2025-11-04T18:20:32.706543] [DEBUG] [LOCAL ] DAO: Deleted primary data file. [2025-11-04T18:20:32.706598] [INFO ] [MAIN ] DAO: Starting full primary refresh. [2025-11-04T18:20:32.972191] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-04. [2025-11-04T18:20:33.092478] [INFO ] [REMOTE ] DAO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:20:33.404337] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T18:20:33.767058] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-01. [2025-11-04T18:20:34.567031] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-03. [2025-11-04T18:20:34.633889] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T18:20:34.764706] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-11-04T18:20:35.367046] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-05. [2025-11-04T18:20:35.828130] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-11-04T18:20:36.167078] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-04. [2025-11-04T18:20:36.967042] [INFO ] [REMOTE ] D: Fetching primary data for 2025-10. [2025-11-04T18:20:37.158836] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-11-04T18:20:37.241966] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T18:20:37.767046] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-02. [2025-11-04T18:20:38.567052] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-06. [2025-11-04T18:20:38.685259] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-11-04T18:20:38.742862] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-11-04T18:20:39.367051] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-05. [2025-11-04T18:20:40.047098] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-11-04T18:20:40.167087] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-03. [2025-11-04T18:20:40.642687] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T18:20:40.967063] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-07. [2025-11-04T18:20:41.204463] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T18:20:41.767069] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-04. [2025-11-04T18:20:42.550810] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-11-04T18:20:42.566988] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-05. [2025-11-04T18:20:42.979510] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-11-04T18:20:43.230869] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T18:20:43.367008] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-06. [2025-11-04T18:20:44.013086] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-11-04T18:20:44.167068] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-07. [2025-11-04T18:20:44.967026] [INFO ] [REMOTE ] D: Fetching primary data for 2025-11. [2025-11-04T18:20:44.996324] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T18:20:45.271108] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-11 (0.30 seconds). [2025-11-04T18:20:45.273093] [INFO ] [REMOTE ] D: Full fetch complete (77813 records). [2025-11-04T18:20:45.273232] [DEBUG] [VALIDATOR ] D: Validating primary data. [2025-11-04T18:20:45.660194] [DEBUG] [VALIDATOR ] D: All expected trading days are present. [2025-11-04T18:20:45.660274] [DEBUG] [VALIDATOR ] D: Primary data validation passed. [2025-11-04T18:20:45.661120] [INFO ] [MAIN ] D: Fetched primary data is valid. [2025-11-04T18:20:45.661183] [DEBUG] [LOCAL ] D: Saving primary data. [2025-11-04T18:20:45.662549] [DEBUG] [LOCAL ] D: Trimmed 77813 records to 53649 (keeping 200 days of data) [2025-11-04T18:20:45.770257] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-06. [2025-11-04T18:20:45.980169] [DEBUG] [LOCAL ] D: Primary data saved (53649 records). [2025-11-04T18:20:45.980297] [INFO ] [MAIN ] D: Full history saved. [2025-11-04T18:20:45.980410] [INFO ] [MAIN ] Update Progress: 585/2367 symbols completed. D took 45.0s. Estimated remaining: 4h 5m 52s. Rate: 7.2 symbols/min [2025-11-04T18:20:46.153217] [INFO ] [MAIN ] DAR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:20:46.199899] [DEBUG] [LOCAL ] DAR: Loaded 51976 existing records from primary storage. [2025-11-04T18:20:46.200210] [INFO ] [MAIN ] DAR: Loaded 51976 existing records. [2025-11-04T18:20:46.379983] [WARN ] [MAIN ] DAR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:20:46.380194] [INFO ] [MAIN ] DAR: Wiping existing primary data before full refresh. [2025-11-04T18:20:46.380253] [INFO ] [LOCAL ] DAR: Starting deletion of primary data. [2025-11-04T18:20:46.380488] [DEBUG] [LOCAL ] DAR: Deleted primary data file. [2025-11-04T18:20:46.380560] [INFO ] [MAIN ] DAR: Starting full primary refresh. [2025-11-04T18:20:46.572286] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-08. [2025-11-04T18:20:46.749785] [INFO ] [REMOTE ] DAR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:20:47.126679] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T18:20:47.367060] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-07. [2025-11-04T18:20:48.167060] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-01. [2025-11-04T18:20:48.403334] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-11-04T18:20:48.693849] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T18:20:48.967038] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-08. [2025-11-04T18:20:49.592435] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T18:20:49.767089] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-08. [2025-11-04T18:20:50.152515] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T18:20:50.510693] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-11-04T18:20:50.567032] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-09. [2025-11-04T18:20:51.356677] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T18:20:51.366987] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-10. [2025-11-04T18:20:52.167011] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-09. [2025-11-04T18:20:52.264807] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-11-04T18:20:52.967059] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-02. [2025-11-04T18:20:53.767078] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-09. [2025-11-04T18:20:53.934551] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-11-04T18:20:54.297923] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T18:20:54.567069] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-03. [2025-11-04T18:20:54.956201] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T18:20:55.367044] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-10. [2025-11-04T18:20:56.001557] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-11-04T18:20:56.167025] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-04. [2025-11-04T18:20:56.967054] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-11. [2025-11-04T18:20:57.090280] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-11-04T18:20:57.203251] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-11-04T18:20:57.680567] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T18:20:57.682508] [INFO ] [REMOTE ] DAO: Full fetch complete (33182 records). [2025-11-04T18:20:57.682591] [DEBUG] [VALIDATOR ] DAO: Validating primary data. [2025-11-04T18:20:57.772204] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-11. [2025-11-04T18:20:58.150788] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T18:20:58.151260] [DEBUG] [VALIDATOR ] DAO: All expected trading days are present. [2025-11-04T18:20:58.153310] [DEBUG] [VALIDATOR ] DAO: Primary data validation passed. [2025-11-04T18:20:58.154408] [INFO ] [MAIN ] DAO: Fetched primary data is valid. [2025-11-04T18:20:58.154549] [INFO ] [REMOTE ] DAL: Full fetch complete (77932 records). [2025-11-04T18:20:58.154742] [DEBUG] [VALIDATOR ] DAL: Validating primary data. [2025-11-04T18:20:58.154992] [DEBUG] [LOCAL ] DAO: Saving primary data. [2025-11-04T18:20:58.198022] [DEBUG] [LOCAL ] DAO: Trimmed 33182 records to 19700 (keeping 200 days of data) [2025-11-04T18:20:58.568403] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-05. [2025-11-04T18:20:58.657050] [DEBUG] [LOCAL ] DAO: Primary data saved (19700 records). [2025-11-04T18:20:58.673245] [INFO ] [MAIN ] DAO: Full history saved. [2025-11-04T18:20:58.683677] [INFO ] [MAIN ] Update Progress: 586/2367 symbols completed. DAO took 26.4s. Estimated remaining: 4h 5m 57s. Rate: 7.2 symbols/min [2025-11-04T18:20:58.885715] [DEBUG] [VALIDATOR ] DAL: All expected trading days are present. [2025-11-04T18:20:58.908018] [DEBUG] [VALIDATOR ] DAL: Primary data validation passed. [2025-11-04T18:20:58.915317] [INFO ] [MAIN ] DAL: Fetched primary data is valid. [2025-11-04T18:20:58.915400] [DEBUG] [LOCAL ] DAL: Saving primary data. [2025-11-04T18:20:58.939851] [INFO ] [MAIN ] DAVA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:20:58.940013] [DEBUG] [LOCAL ] DAVA: No existing data file found in primary storage. [2025-11-04T18:20:58.940120] [WARN ] [MAIN ] DAVA: No existing data found, performing full refresh instead. [2025-11-04T18:20:58.940348] [INFO ] [MAIN ] DAVA: Wiping existing primary data before full refresh. [2025-11-04T18:20:58.940398] [INFO ] [LOCAL ] DAVA: Starting deletion of primary data. [2025-11-04T18:20:58.940505] [DEBUG] [LOCAL ] DAL: Trimmed 77932 records to 53753 (keeping 200 days of data) [2025-11-04T18:20:58.940444] [DEBUG] [LOCAL ] DAVA: No primary data file found to delete. [2025-11-04T18:20:58.940612] [INFO ] [MAIN ] DAVA: Starting full primary refresh. [2025-11-04T18:20:59.368853] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-10. [2025-11-04T18:20:59.488446] [INFO ] [REMOTE ] DAVA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:20:59.651686] [DEBUG] [LOCAL ] DAL: Primary data saved (53753 records). [2025-11-04T18:20:59.651842] [INFO ] [MAIN ] DAL: Full history saved. [2025-11-04T18:20:59.652385] [INFO ] [MAIN ] Update Progress: 587/2367 symbols completed. DAL took 38.4s. Estimated remaining: 4h 5m 26s. Rate: 7.3 symbols/min [2025-11-04T18:20:59.819198] [INFO ] [MAIN ] DAY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:20:59.862252] [DEBUG] [LOCAL ] DAY: Loaded 49061 existing records from primary storage. [2025-11-04T18:20:59.862548] [INFO ] [MAIN ] DAY: Loaded 49061 existing records. [2025-11-04T18:21:00.039528] [WARN ] [MAIN ] DAY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:21:00.039842] [INFO ] [MAIN ] DAY: Wiping existing primary data before full refresh. [2025-11-04T18:21:00.039971] [INFO ] [LOCAL ] DAY: Starting deletion of primary data. [2025-11-04T18:21:00.040345] [DEBUG] [LOCAL ] DAY: Deleted primary data file. [2025-11-04T18:21:00.040384] [INFO ] [MAIN ] DAY: Starting full primary refresh. [2025-11-04T18:21:00.172710] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-01. [2025-11-04T18:21:00.409611] [INFO ] [REMOTE ] DAY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:21:00.439680] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-11-04T18:21:00.967063] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-06. [2025-11-04T18:21:01.334981] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-11-04T18:21:01.653561] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-11-04T18:21:01.767046] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-02. [2025-11-04T18:21:02.567031] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-11. [2025-11-04T18:21:02.745620] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-11-04T18:21:02.812379] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T18:21:03.367076] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-07. [2025-11-04T18:21:03.714485] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T18:21:03.716426] [INFO ] [REMOTE ] DAN: Full fetch complete (71554 records). [2025-11-04T18:21:03.716544] [DEBUG] [VALIDATOR ] DAN: Validating primary data. [2025-11-04T18:21:04.115031] [DEBUG] [VALIDATOR ] DAN: All expected trading days are present. [2025-11-04T18:21:04.115186] [DEBUG] [VALIDATOR ] DAN: Primary data validation passed. [2025-11-04T18:21:04.116675] [INFO ] [MAIN ] DAN: Fetched primary data is valid. [2025-11-04T18:21:04.116816] [DEBUG] [LOCAL ] DAN: Saving primary data. [2025-11-04T18:21:04.119050] [DEBUG] [LOCAL ] DAN: Trimmed 71554 records to 49347 (keeping 200 days of data) [2025-11-04T18:21:04.174500] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-01. [2025-11-04T18:21:04.439804] [DEBUG] [LOCAL ] DAN: Primary data saved (49347 records). [2025-11-04T18:21:04.440038] [INFO ] [MAIN ] DAN: Full history saved. [2025-11-04T18:21:04.440154] [INFO ] [MAIN ] Update Progress: 588/2367 symbols completed. DAN took 36.2s. Estimated remaining: 4h 5m 7s. Rate: 7.3 symbols/min [2025-11-04T18:21:04.619614] [INFO ] [MAIN ] DB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:21:04.620156] [DEBUG] [LOCAL ] DB: No existing data file found in primary storage. [2025-11-04T18:21:04.620262] [WARN ] [MAIN ] DB: No existing data found, performing full refresh instead. [2025-11-04T18:21:04.620509] [INFO ] [MAIN ] DB: Wiping existing primary data before full refresh. [2025-11-04T18:21:04.620612] [INFO ] [LOCAL ] DB: Starting deletion of primary data. [2025-11-04T18:21:04.620784] [DEBUG] [LOCAL ] DB: No primary data file found to delete. [2025-11-04T18:21:04.620839] [INFO ] [MAIN ] DB: Starting full primary refresh. [2025-11-04T18:21:04.984100] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-11-04T18:21:04.975675] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-03. [2025-11-04T18:21:05.066929] [INFO ] [REMOTE ] DB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:21:05.580146] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T18:21:05.767038] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-02. [2025-11-04T18:21:05.983169] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T18:21:06.567089] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-08. [2025-11-04T18:21:07.024982] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T18:21:07.367054] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-04. [2025-11-04T18:21:07.979935] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-11-04T18:21:08.167081] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-09. [2025-11-04T18:21:08.458226] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T18:21:08.967015] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-01. [2025-11-04T18:21:09.551135] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T18:21:09.767051] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-05. [2025-11-04T18:21:10.376883] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T18:21:10.567081] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-10. [2025-11-04T18:21:10.884935] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-11-04T18:21:11.367067] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-02. [2025-11-04T18:21:11.985387] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-11-04T18:21:12.167016] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-11. [2025-11-04T18:21:12.604815] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T18:21:12.967041] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-03. [2025-11-04T18:21:13.228125] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T18:21:13.230566] [INFO ] [REMOTE ] DAR: Full fetch complete (75303 records). [2025-11-04T18:21:13.230736] [DEBUG] [VALIDATOR ] DAR: Validating primary data. [2025-11-04T18:21:13.710632] [DEBUG] [VALIDATOR ] DAR: All expected trading days are present. [2025-11-04T18:21:13.710767] [DEBUG] [VALIDATOR ] DAR: Primary data validation passed. [2025-11-04T18:21:13.711730] [INFO ] [MAIN ] DAR: Fetched primary data is valid. [2025-11-04T18:21:13.711796] [DEBUG] [LOCAL ] DAR: Saving primary data. [2025-11-04T18:21:13.713323] [DEBUG] [LOCAL ] DAR: Trimmed 75303 records to 51978 (keeping 200 days of data) [2025-11-04T18:21:13.775369] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-03. [2025-11-04T18:21:14.054847] [DEBUG] [LOCAL ] DAR: Primary data saved (51978 records). [2025-11-04T18:21:14.054977] [INFO ] [MAIN ] DAR: Full history saved. [2025-11-04T18:21:14.055057] [INFO ] [MAIN ] Update Progress: 589/2367 symbols completed. DAR took 28.1s. Estimated remaining: 4h 5m 3s. Rate: 7.3 symbols/min [2025-11-04T18:21:14.256486] [INFO ] [MAIN ] DBD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:21:14.256708] [DEBUG] [LOCAL ] DBD: No existing data file found in primary storage. [2025-11-04T18:21:14.256798] [WARN ] [MAIN ] DBD: No existing data found, performing full refresh instead. [2025-11-04T18:21:14.256924] [INFO ] [MAIN ] DBD: Wiping existing primary data before full refresh. [2025-11-04T18:21:14.256971] [INFO ] [LOCAL ] DBD: Starting deletion of primary data. [2025-11-04T18:21:14.257046] [DEBUG] [LOCAL ] DBD: No primary data file found to delete. [2025-11-04T18:21:14.257100] [INFO ] [MAIN ] DBD: Starting full primary refresh. [2025-11-04T18:21:14.572667] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-06. [2025-11-04T18:21:14.683328] [INFO ] [REMOTE ] DBD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:21:14.713838] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-11-04T18:21:14.925158] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T18:21:15.367048] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-01. [2025-11-04T18:21:15.879913] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T18:21:15.897128] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T18:21:16.167052] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-04. [2025-11-04T18:21:16.967068] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-02. [2025-11-04T18:21:17.664779] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T18:21:17.767008] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-03. [2025-11-04T18:21:17.840197] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-11-04T18:21:18.446206] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-11-04T18:21:18.567089] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-05. [2025-11-04T18:21:19.367061] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-07. [2025-11-04T18:21:19.971296] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T18:21:20.167064] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-04. [2025-11-04T18:21:20.268591] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T18:21:20.943895] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-11-04T18:21:20.966996] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-05. [2025-11-04T18:21:21.611361] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T18:21:21.767028] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-06. [2025-11-04T18:21:22.567051] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-04. [2025-11-04T18:21:23.218598] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T18:21:23.367035] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-07. [2025-11-04T18:21:23.866268] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T18:21:24.167042] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-05. [2025-11-04T18:21:24.872707] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T18:21:24.967078] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-08. [2025-11-04T18:21:25.466701] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T18:21:25.767039] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-08. [2025-11-04T18:21:26.492551] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T18:21:26.567040] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-09. [2025-11-04T18:21:26.961369] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T18:21:27.367068] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-06. [2025-11-04T18:21:28.066737] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T18:21:28.167052] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-09. [2025-11-04T18:21:28.555096] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-11-04T18:21:28.967070] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-06. [2025-11-04T18:21:29.392323] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T18:21:29.767028] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-07. [2025-11-04T18:21:29.816856] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T18:21:30.567123] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-10. [2025-11-04T18:21:31.012951] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:21:31.367030] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-10. [2025-11-04T18:21:31.830467] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-11-04T18:21:32.167038] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-08. [2025-11-04T18:21:32.838683] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T18:21:32.967029] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-11. [2025-11-04T18:21:33.636898] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-11-04T18:21:33.767046] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-09. [2025-11-04T18:21:34.123493] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T18:21:34.125392] [INFO ] [REMOTE ] DAVA: Full fetch complete (58056 records). [2025-11-04T18:21:34.125509] [DEBUG] [VALIDATOR ] DAVA: Validating primary data. [2025-11-04T18:21:34.558348] [DEBUG] [VALIDATOR ] DAVA: All expected trading days are present. [2025-11-04T18:21:34.558440] [DEBUG] [VALIDATOR ] DAVA: Primary data validation passed. [2025-11-04T18:21:34.559237] [INFO ] [MAIN ] DAVA: Fetched primary data is valid. [2025-11-04T18:21:34.559305] [DEBUG] [LOCAL ] DAVA: Saving primary data. [2025-11-04T18:21:34.560555] [DEBUG] [LOCAL ] DAVA: Trimmed 58056 records to 42238 (keeping 200 days of data) [2025-11-04T18:21:34.607148] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-07. [2025-11-04T18:21:34.798035] [DEBUG] [LOCAL ] DAVA: Primary data saved (42238 records). [2025-11-04T18:21:34.798160] [INFO ] [MAIN ] DAVA: Full history saved. [2025-11-04T18:21:34.798228] [INFO ] [MAIN ] Update Progress: 590/2367 symbols completed. DAVA took 36.1s. Estimated remaining: 4h 5m 33s. Rate: 7.2 symbols/min [2025-11-04T18:21:34.976074] [INFO ] [MAIN ] DBI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:21:35.023581] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T18:21:35.045910] [DEBUG] [LOCAL ] DBI: Loaded 42480 existing records from primary storage. [2025-11-04T18:21:35.046188] [INFO ] [MAIN ] DBI: Loaded 42480 existing records. [2025-11-04T18:21:35.227369] [WARN ] [MAIN ] DBI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:21:35.227599] [INFO ] [MAIN ] DBI: Wiping existing primary data before full refresh. [2025-11-04T18:21:35.227665] [INFO ] [LOCAL ] DBI: Starting deletion of primary data. [2025-11-04T18:21:35.227865] [DEBUG] [LOCAL ] DBI: Deleted primary data file. [2025-11-04T18:21:35.227905] [INFO ] [MAIN ] DBI: Starting full primary refresh. [2025-11-04T18:21:35.372184] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-10. [2025-11-04T18:21:35.594874] [INFO ] [REMOTE ] DBI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:21:35.612037] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T18:21:36.167046] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-11. [2025-11-04T18:21:36.668140] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-11-04T18:21:36.967042] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-11. [2025-11-04T18:21:37.384711] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T18:21:37.386780] [INFO ] [REMOTE ] DB: Full fetch complete (75886 records). [2025-11-04T18:21:37.386925] [DEBUG] [VALIDATOR ] DB: Validating primary data. [2025-11-04T18:21:37.770849] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-08. [2025-11-04T18:21:37.794267] [DEBUG] [VALIDATOR ] DB: All expected trading days are present. [2025-11-04T18:21:37.794376] [DEBUG] [VALIDATOR ] DB: Primary data validation passed. [2025-11-04T18:21:37.795319] [INFO ] [MAIN ] DB: Fetched primary data is valid. [2025-11-04T18:21:37.795518] [DEBUG] [LOCAL ] DB: Saving primary data. [2025-11-04T18:21:37.797224] [DEBUG] [LOCAL ] DB: Trimmed 75886 records to 52373 (keeping 200 days of data) [2025-11-04T18:21:38.105866] [DEBUG] [LOCAL ] DB: Primary data saved (52373 records). [2025-11-04T18:21:38.106007] [INFO ] [MAIN ] DB: Full history saved. [2025-11-04T18:21:38.106080] [INFO ] [MAIN ] Update Progress: 591/2367 symbols completed. DB took 33.7s. Estimated remaining: 4h 5m 9s. Rate: 7.2 symbols/min [2025-11-04T18:21:38.279409] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-11 (1.31 seconds). [2025-11-04T18:21:38.279249] [INFO ] [MAIN ] DBL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:21:38.281747] [INFO ] [REMOTE ] DAY: Full fetch complete (71977 records). [2025-11-04T18:21:38.281862] [DEBUG] [VALIDATOR ] DAY: Validating primary data. [2025-11-04T18:21:38.393548] [DEBUG] [LOCAL ] DBL: Loaded 9199 existing records from primary storage. [2025-11-04T18:21:38.404459] [INFO ] [MAIN ] DBL: Loaded 9199 existing records. [2025-11-04T18:21:38.571323] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-01. [2025-11-04T18:21:38.756348] [WARN ] [MAIN ] DBL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:21:38.774261] [INFO ] [MAIN ] DBL: Wiping existing primary data before full refresh. [2025-11-04T18:21:38.785769] [INFO ] [LOCAL ] DBL: Starting deletion of primary data. [2025-11-04T18:21:38.785980] [DEBUG] [LOCAL ] DBL: Deleted primary data file. [2025-11-04T18:21:38.786115] [INFO ] [MAIN ] DBL: Starting full primary refresh. [2025-11-04T18:21:39.009527] [DEBUG] [VALIDATOR ] DAY: All expected trading days are present. [2025-11-04T18:21:39.009061] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T18:21:39.030391] [DEBUG] [VALIDATOR ] DAY: Primary data validation passed. [2025-11-04T18:21:39.048857] [INFO ] [MAIN ] DAY: Fetched primary data is valid. [2025-11-04T18:21:39.054205] [DEBUG] [LOCAL ] DAY: Saving primary data. [2025-11-04T18:21:39.166999] [DEBUG] [LOCAL ] DAY: Trimmed 71977 records to 49053 (keeping 200 days of data) [2025-11-04T18:21:39.379703] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-09. [2025-11-04T18:21:39.394254] [INFO ] [REMOTE ] DBL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:21:39.572164] [DEBUG] [LOCAL ] DAY: Primary data saved (49053 records). [2025-11-04T18:21:39.572291] [INFO ] [MAIN ] DAY: Full history saved. [2025-11-04T18:21:39.572874] [INFO ] [MAIN ] Update Progress: 592/2367 symbols completed. DAY took 39.9s. Estimated remaining: 4h 4m 41s. Rate: 7.3 symbols/min [2025-11-04T18:21:39.742698] [INFO ] [MAIN ] DBRG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:21:39.789212] [DEBUG] [LOCAL ] DBRG: Loaded 50025 existing records from primary storage. [2025-11-04T18:21:39.789495] [INFO ] [MAIN ] DBRG: Loaded 50025 existing records. [2025-11-04T18:21:39.967678] [WARN ] [MAIN ] DBRG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:21:39.968039] [INFO ] [MAIN ] DBRG: Wiping existing primary data before full refresh. [2025-11-04T18:21:39.968185] [INFO ] [LOCAL ] DBRG: Starting deletion of primary data. [2025-11-04T18:21:39.968488] [DEBUG] [LOCAL ] DBRG: Deleted primary data file. [2025-11-04T18:21:39.968607] [INFO ] [MAIN ] DBRG: Starting full primary refresh. [2025-11-04T18:21:40.170808] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-01. [2025-11-04T18:21:40.208079] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-11-04T18:21:40.374249] [INFO ] [REMOTE ] DBRG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:21:40.388314] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T18:21:40.725561] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-11-04T18:21:40.967044] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-02. [2025-11-04T18:21:41.346152] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T18:21:41.767071] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-01. [2025-11-04T18:21:42.567041] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-03. [2025-11-04T18:21:43.034165] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T18:21:43.095741] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-11-04T18:21:43.367038] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-10. [2025-11-04T18:21:44.167015] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-02. [2025-11-04T18:21:44.242585] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-11-04T18:21:44.967117] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-04. [2025-11-04T18:21:45.342740] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T18:21:45.578833] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-11-04T18:21:45.767095] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-05. [2025-11-04T18:21:46.315222] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T18:21:46.567104] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-03. [2025-11-04T18:21:47.367062] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-02. [2025-11-04T18:21:47.825878] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-11-04T18:21:48.167070] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-11. [2025-11-04T18:21:48.557805] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T18:21:48.730300] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-11 (0.56 seconds). [2025-11-04T18:21:48.731570] [INFO ] [REMOTE ] DBD: Full fetch complete (32661 records). [2025-11-04T18:21:48.731680] [DEBUG] [VALIDATOR ] DBD: Validating primary data. [2025-11-04T18:21:48.732169] [FATAL] [VALIDATOR ] DBD: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:21:48.732288] [FATAL] [MAIN ] DBD: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:21:48.732365] [WARN ] [MAIN ] DBD: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:21:48.732460] [INFO ] [LOCAL ] DBD: Starting deletion of primary data. [2025-11-04T18:21:48.732541] [DEBUG] [LOCAL ] DBD: No primary data file found to delete. [2025-11-04T18:21:48.732580] [INFO ] [LOCAL ] DBD: Starting deletion of derivative data. [2025-11-04T18:21:48.732621] [DEBUG] [LOCAL ] DBD: No derivative data file found to delete. [2025-11-04T18:21:48.732719] [INFO ] [MAIN ] Update Progress: 593/2367 symbols completed. DBD took 34.7s. Estimated remaining: 4h 4m 35s. Rate: 7.3 symbols/min [2025-11-04T18:21:48.900352] [INFO ] [MAIN ] DCI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:21:48.932319] [DEBUG] [LOCAL ] DCI: Loaded 36649 existing records from primary storage. [2025-11-04T18:21:48.932615] [INFO ] [MAIN ] DCI: Loaded 36649 existing records. [2025-11-04T18:21:48.972289] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-04. [2025-11-04T18:21:49.112663] [WARN ] [MAIN ] DCI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:21:49.112891] [INFO ] [MAIN ] DCI: Wiping existing primary data before full refresh. [2025-11-04T18:21:49.112948] [INFO ] [LOCAL ] DCI: Starting deletion of primary data. [2025-11-04T18:21:49.113231] [DEBUG] [LOCAL ] DCI: Deleted primary data file. [2025-11-04T18:21:49.113269] [INFO ] [MAIN ] DCI: Starting full primary refresh. [2025-11-04T18:21:49.485890] [INFO ] [REMOTE ] DCI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:21:49.767082] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-06. [2025-11-04T18:21:50.274440] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T18:21:50.567780] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-01. [2025-11-04T18:21:50.573391] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-11-04T18:21:51.367052] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-05. [2025-11-04T18:21:51.762083] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T18:21:52.167063] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-02. [2025-11-04T18:21:52.472086] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T18:21:52.967051] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-03. [2025-11-04T18:21:53.267959] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T18:21:53.767070] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-03. [2025-11-04T18:21:54.451048] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-11-04T18:21:54.567038] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-04. [2025-11-04T18:21:54.898914] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-11-04T18:21:55.367111] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-06. [2025-11-04T18:21:55.929883] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T18:21:56.167097] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-07. [2025-11-04T18:21:56.616044] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-11-04T18:21:56.869359] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T18:21:56.967074] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-08. [2025-11-04T18:21:57.393049] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-11-04T18:21:57.767080] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-07. [2025-11-04T18:21:58.567015] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-04. [2025-11-04T18:21:59.078096] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T18:21:59.367064] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-05. [2025-11-04T18:21:59.820867] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T18:22:00.167070] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-08. [2025-11-04T18:22:00.699960] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T18:22:00.967043] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-06. [2025-11-04T18:22:01.772192] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-05. [2025-11-04T18:22:01.785469] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-11-04T18:22:02.251084] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T18:22:02.567085] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-09. [2025-11-04T18:22:03.026510] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-11-04T18:22:03.132364] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-11-04T18:22:03.367033] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-10. [2025-11-04T18:22:03.880769] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-11-04T18:22:04.167102] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-11. [2025-11-04T18:22:04.627662] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-11 (0.46 seconds). [2025-11-04T18:22:04.628479] [INFO ] [REMOTE ] DBL: Full fetch complete (12687 records). [2025-11-04T18:22:04.628549] [DEBUG] [VALIDATOR ] DBL: Validating primary data. [2025-11-04T18:22:04.972236] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-06. [2025-11-04T18:22:05.053002] [DEBUG] [VALIDATOR ] DBL: All expected trading days are present. [2025-11-04T18:22:05.053228] [DEBUG] [VALIDATOR ] DBL: Primary data validation passed. [2025-11-04T18:22:05.053531] [INFO ] [MAIN ] DBL: Fetched primary data is valid. [2025-11-04T18:22:05.053571] [DEBUG] [LOCAL ] DBL: Saving primary data. [2025-11-04T18:22:05.054072] [DEBUG] [LOCAL ] DBL: Trimmed 12687 records to 9196 (keeping 200 days of data) [2025-11-04T18:22:05.108605] [DEBUG] [LOCAL ] DBL: Primary data saved (9196 records). [2025-11-04T18:22:05.108754] [INFO ] [MAIN ] DBL: Full history saved. [2025-11-04T18:22:05.108835] [INFO ] [MAIN ] Update Progress: 594/2367 symbols completed. DBL took 27.0s. Estimated remaining: 4h 4m 51s. Rate: 7.2 symbols/min [2025-11-04T18:22:05.276466] [INFO ] [MAIN ] DCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:22:05.292443] [DEBUG] [LOCAL ] DCO: Loaded 15330 existing records from primary storage. [2025-11-04T18:22:05.292701] [INFO ] [MAIN ] DCO: Loaded 15330 existing records. [2025-11-04T18:22:05.470154] [WARN ] [MAIN ] DCO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:22:05.470360] [INFO ] [MAIN ] DCO: Wiping existing primary data before full refresh. [2025-11-04T18:22:05.470407] [INFO ] [LOCAL ] DCO: Starting deletion of primary data. [2025-11-04T18:22:05.470637] [DEBUG] [LOCAL ] DCO: Deleted primary data file. [2025-11-04T18:22:05.470722] [INFO ] [MAIN ] DCO: Starting full primary refresh. [2025-11-04T18:22:05.783207] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-07. [2025-11-04T18:22:05.823586] [INFO ] [REMOTE ] DCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:22:06.211585] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T18:22:06.567072] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-01. [2025-11-04T18:22:07.150071] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-11-04T18:22:07.331671] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-11-04T18:22:07.366995] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-08. [2025-11-04T18:22:08.167042] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-07. [2025-11-04T18:22:08.652351] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T18:22:08.967057] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-02. [2025-11-04T18:22:09.191293] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T18:22:09.618545] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-11-04T18:22:09.767047] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-09. [2025-11-04T18:22:10.567047] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-08. [2025-11-04T18:22:10.902564] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T18:22:11.367039] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-03. [2025-11-04T18:22:11.645134] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-11-04T18:22:12.006360] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T18:22:12.167043] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-09. [2025-11-04T18:22:12.967082] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-04. [2025-11-04T18:22:13.479787] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T18:22:13.547354] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-11-04T18:22:13.767042] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-10. [2025-11-04T18:22:14.567086] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-05. [2025-11-04T18:22:14.854905] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-11-04T18:22:15.068419] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-11-04T18:22:15.367064] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-10. [2025-11-04T18:22:16.167065] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-11. [2025-11-04T18:22:16.613085] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T18:22:16.967060] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-06. [2025-11-04T18:22:17.022219] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T18:22:17.024429] [INFO ] [REMOTE ] DCI: Full fetch complete (53929 records). [2025-11-04T18:22:17.024558] [DEBUG] [VALIDATOR ] DCI: Validating primary data. [2025-11-04T18:22:17.576379] [DEBUG] [VALIDATOR ] DCI: All expected trading days are present. [2025-11-04T18:22:17.576500] [DEBUG] [VALIDATOR ] DCI: Primary data validation passed. [2025-11-04T18:22:17.579253] [INFO ] [MAIN ] DCI: Fetched primary data is valid. [2025-11-04T18:22:17.579479] [DEBUG] [LOCAL ] DCI: Saving primary data. [2025-11-04T18:22:17.581219] [DEBUG] [LOCAL ] DCI: Trimmed 53929 records to 36616 (keeping 200 days of data) [2025-11-04T18:22:17.748859] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T18:22:17.767079] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-07. [2025-11-04T18:22:17.959780] [DEBUG] [LOCAL ] DCI: Primary data saved (36616 records). [2025-11-04T18:22:17.959963] [INFO ] [MAIN ] DCI: Full history saved. [2025-11-04T18:22:17.960064] [INFO ] [MAIN ] Update Progress: 595/2367 symbols completed. DCI took 29.2s. Estimated remaining: 4h 4m 56s. Rate: 7.2 symbols/min [2025-11-04T18:22:18.180635] [INFO ] [MAIN ] DD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:22:18.180905] [DEBUG] [LOCAL ] DD: No existing data file found in primary storage. [2025-11-04T18:22:18.180963] [WARN ] [MAIN ] DD: No existing data found, performing full refresh instead. [2025-11-04T18:22:18.181082] [INFO ] [MAIN ] DD: Wiping existing primary data before full refresh. [2025-11-04T18:22:18.181124] [INFO ] [LOCAL ] DD: Starting deletion of primary data. [2025-11-04T18:22:18.181166] [DEBUG] [LOCAL ] DD: No primary data file found to delete. [2025-11-04T18:22:18.181197] [INFO ] [MAIN ] DD: Starting full primary refresh. [2025-11-04T18:22:18.554913] [INFO ] [REMOTE ] DD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:22:18.569036] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-11-04T18:22:18.567535] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-11. [2025-11-04T18:22:19.367049] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-01. [2025-11-04T18:22:19.485148] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T18:22:19.486575] [INFO ] [REMOTE ] DBI: Full fetch complete (63777 records). [2025-11-04T18:22:19.487049] [DEBUG] [VALIDATOR ] DBI: Validating primary data. [2025-11-04T18:22:19.487946] [FATAL] [VALIDATOR ] DBI: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:22:19.488037] [FATAL] [MAIN ] DBI: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:22:19.488082] [WARN ] [MAIN ] DBI: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:22:19.488122] [INFO ] [LOCAL ] DBI: Starting deletion of primary data. [2025-11-04T18:22:19.488177] [DEBUG] [LOCAL ] DBI: No primary data file found to delete. [2025-11-04T18:22:19.488212] [INFO ] [LOCAL ] DBI: Starting deletion of derivative data. [2025-11-04T18:22:19.488337] [DEBUG] [LOCAL ] DBI: Deleted derivative data file. [2025-11-04T18:22:19.488409] [INFO ] [MAIN ] Update Progress: 596/2367 symbols completed. DBI took 44.7s. Estimated remaining: 4h 4m 28s. Rate: 7.2 symbols/min [2025-11-04T18:22:19.698523] [INFO ] [MAIN ] DDD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:22:19.748469] [DEBUG] [LOCAL ] DDD: Loaded 51002 existing records from primary storage. [2025-11-04T18:22:19.749028] [INFO ] [MAIN ] DDD: Loaded 51002 existing records. [2025-11-04T18:22:19.926043] [WARN ] [MAIN ] DDD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:22:19.926282] [INFO ] [MAIN ] DDD: Wiping existing primary data before full refresh. [2025-11-04T18:22:19.926328] [INFO ] [LOCAL ] DDD: Starting deletion of primary data. [2025-11-04T18:22:19.926513] [DEBUG] [LOCAL ] DDD: Deleted primary data file. [2025-11-04T18:22:19.926550] [INFO ] [MAIN ] DDD: Starting full primary refresh. [2025-11-04T18:22:20.173710] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-08. [2025-11-04T18:22:20.289010] [INFO ] [REMOTE ] DDD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:22:20.963933] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T18:22:20.968413] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-01. [2025-11-04T18:22:21.001295] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-11-04T18:22:21.767074] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-09. [2025-11-04T18:22:22.509573] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-11-04T18:22:22.567023] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-02. [2025-11-04T18:22:23.373023] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-09. [2025-11-04T18:22:23.406528] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-11-04T18:22:23.865384] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T18:22:24.109035] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-11-04T18:22:24.167089] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-02. [2025-11-04T18:22:24.967074] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-10. [2025-11-04T18:22:25.447368] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T18:22:25.767020] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-03. [2025-11-04T18:22:25.784562] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T18:22:26.567036] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-10. [2025-11-04T18:22:27.314925] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T18:22:27.367165] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-04. [2025-11-04T18:22:28.178845] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-11. [2025-11-04T18:22:28.186083] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-11-04T18:22:28.930147] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T18:22:28.967074] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-11. [2025-11-04T18:22:29.080328] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T18:22:29.081601] [INFO ] [REMOTE ] DCO: Full fetch complete (20102 records). [2025-11-04T18:22:29.081698] [DEBUG] [VALIDATOR ] DCO: Validating primary data. [2025-11-04T18:22:29.503451] [DEBUG] [VALIDATOR ] DCO: All expected trading days are present. [2025-11-04T18:22:29.503535] [DEBUG] [VALIDATOR ] DCO: Primary data validation passed. [2025-11-04T18:22:29.504058] [INFO ] [MAIN ] DCO: Fetched primary data is valid. [2025-11-04T18:22:29.504121] [DEBUG] [LOCAL ] DCO: Saving primary data. [2025-11-04T18:22:29.504760] [DEBUG] [LOCAL ] DCO: Trimmed 20102 records to 15290 (keeping 200 days of data) [2025-11-04T18:22:29.594901] [DEBUG] [LOCAL ] DCO: Primary data saved (15290 records). [2025-11-04T18:22:29.595045] [INFO ] [MAIN ] DCO: Full history saved. [2025-11-04T18:22:29.595171] [INFO ] [MAIN ] Update Progress: 597/2367 symbols completed. DCO took 24.5s. Estimated remaining: 4h 4m 25s. Rate: 7.2 symbols/min [2025-11-04T18:22:29.767037] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-05. [2025-11-04T18:22:30.362715] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-11 (1.40 seconds). [2025-11-04T18:22:30.365397] [INFO ] [REMOTE ] DBRG: Full fetch complete (72827 records). [2025-11-04T18:22:30.365537] [DEBUG] [VALIDATOR ] DBRG: Validating primary data. [2025-11-04T18:22:30.581554] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-03. [2025-11-04T18:22:30.836213] [DEBUG] [VALIDATOR ] DBRG: All expected trading days are present. [2025-11-04T18:22:30.836323] [DEBUG] [VALIDATOR ] DBRG: Primary data validation passed. [2025-11-04T18:22:30.837301] [INFO ] [MAIN ] DBRG: Fetched primary data is valid. [2025-11-04T18:22:30.837368] [DEBUG] [LOCAL ] DBRG: Saving primary data. [2025-11-04T18:22:30.838930] [DEBUG] [LOCAL ] DBRG: Trimmed 72827 records to 50047 (keeping 200 days of data) [2025-11-04T18:22:31.166388] [DEBUG] [LOCAL ] DBRG: Primary data saved (50047 records). [2025-11-04T18:22:31.166528] [INFO ] [MAIN ] DBRG: Full history saved. [2025-11-04T18:22:31.167153] [INFO ] [MAIN ] Update Progress: 598/2367 symbols completed. DBRG took 51.6s. Estimated remaining: 4h 3m 57s. Rate: 7.3 symbols/min [2025-11-04T18:22:31.355733] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-11-04T18:22:31.367094] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-06. [2025-11-04T18:22:32.145061] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T18:22:32.167032] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-04. [2025-11-04T18:22:32.818318] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T18:22:32.967159] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-07. [2025-11-04T18:22:33.898616] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-11-04T18:22:33.910997] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-05. [2025-11-04T18:22:34.449198] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T18:22:34.567032] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-08. [2025-11-04T18:22:35.313381] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T18:22:35.367024] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-06. [2025-11-04T18:22:36.098259] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T18:22:36.167082] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-09. [2025-11-04T18:22:36.900025] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-11-04T18:22:36.967018] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-07. [2025-11-04T18:22:37.753840] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T18:22:37.767000] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-10. [2025-11-04T18:22:38.509145] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-11-04T18:22:38.567156] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-08. [2025-11-04T18:22:39.342114] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-11-04T18:22:39.367010] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-11. [2025-11-04T18:22:40.051556] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T18:22:40.167073] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-09. [2025-11-04T18:22:40.813336] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-11 (1.45 seconds). [2025-11-04T18:22:40.815433] [INFO ] [REMOTE ] DD: Full fetch complete (76646 records). [2025-11-04T18:22:40.815575] [DEBUG] [VALIDATOR ] DD: Validating primary data. [2025-11-04T18:22:41.239892] [DEBUG] [VALIDATOR ] DD: All expected trading days are present. [2025-11-04T18:22:41.239970] [DEBUG] [VALIDATOR ] DD: Primary data validation passed. [2025-11-04T18:22:41.240803] [INFO ] [MAIN ] DD: Fetched primary data is valid. [2025-11-04T18:22:41.240865] [DEBUG] [LOCAL ] DD: Saving primary data. [2025-11-04T18:22:41.242290] [DEBUG] [LOCAL ] DD: Trimmed 76646 records to 52910 (keeping 200 days of data) [2025-11-04T18:22:41.564744] [DEBUG] [LOCAL ] DD: Primary data saved (52910 records). [2025-11-04T18:22:41.564966] [INFO ] [MAIN ] DD: Full history saved. [2025-11-04T18:22:41.565046] [INFO ] [MAIN ] Update Progress: 599/2367 symbols completed. DD took 23.6s. Estimated remaining: 4h 3m 55s. Rate: 7.2 symbols/min [2025-11-04T18:22:41.759979] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T18:22:41.772392] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-10. [2025-11-04T18:22:43.478636] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-11-04T18:22:43.491407] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-11. [2025-11-04T18:22:45.069812] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-11 (1.58 seconds). [2025-11-04T18:22:45.071294] [INFO ] [REMOTE ] DDD: Full fetch complete (73753 records). [2025-11-04T18:22:45.071391] [DEBUG] [VALIDATOR ] DDD: Validating primary data. [2025-11-04T18:22:45.072187] [FATAL] [VALIDATOR ] DDD: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:22:45.072276] [FATAL] [MAIN ] DDD: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:22:45.072318] [WARN ] [MAIN ] DDD: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:22:45.072390] [INFO ] [LOCAL ] DDD: Starting deletion of primary data. [2025-11-04T18:22:45.072452] [DEBUG] [LOCAL ] DDD: No primary data file found to delete. [2025-11-04T18:22:45.072487] [INFO ] [LOCAL ] DDD: Starting deletion of derivative data. [2025-11-04T18:22:45.072691] [DEBUG] [LOCAL ] DDD: Deleted derivative data file. [2025-11-04T18:22:45.072839] [INFO ] [MAIN ] Update Progress: 600/2367 symbols completed. DDD took 25.6s. Estimated remaining: 4h 3m 32s. Rate: 7.3 symbols/min [2025-11-04T18:22:45.663290] [INFO ] [MAIN ] DDL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:22:45.749894] [INFO ] [MAIN ] DE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:22:45.756638] [INFO ] [MAIN ] DDS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:22:45.802032] [INFO ] [MAIN ] DDT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:22:45.807925] [DEBUG] [LOCAL ] DDT: Loaded 1778 existing records from primary storage. [2025-11-04T18:22:45.808433] [INFO ] [MAIN ] DDT: Loaded 1778 existing records. [2025-11-04T18:22:45.931600] [DEBUG] [LOCAL ] DDS: Loaded 11810 existing records from primary storage. [2025-11-04T18:22:45.953178] [INFO ] [MAIN ] DDS: Loaded 11810 existing records. [2025-11-04T18:22:46.055121] [WARN ] [MAIN ] DDT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:22:46.077673] [INFO ] [MAIN ] DDT: Wiping existing primary data before full refresh. [2025-11-04T18:22:46.108308] [DEBUG] [LOCAL ] DDL: Loaded 36735 existing records from primary storage. [2025-11-04T18:22:46.102200] [INFO ] [LOCAL ] DDT: Starting deletion of primary data. [2025-11-04T18:22:46.119170] [INFO ] [MAIN ] DDL: Loaded 36735 existing records. [2025-11-04T18:22:46.176564] [DEBUG] [LOCAL ] DDT: Deleted primary data file. [2025-11-04T18:22:46.192891] [INFO ] [MAIN ] DDT: Starting full primary refresh. [2025-11-04T18:22:46.396330] [DEBUG] [LOCAL ] DE: Loaded 47290 existing records from primary storage. [2025-11-04T18:22:46.447757] [WARN ] [MAIN ] DDS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:22:46.453343] [INFO ] [MAIN ] DE: Loaded 47290 existing records. [2025-11-04T18:22:46.497617] [INFO ] [MAIN ] DDS: Wiping existing primary data before full refresh. [2025-11-04T18:22:46.554745] [INFO ] [LOCAL ] DDS: Starting deletion of primary data. [2025-11-04T18:22:46.638147] [DEBUG] [LOCAL ] DDS: Deleted primary data file. [2025-11-04T18:22:46.627387] [WARN ] [MAIN ] DDL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:22:46.652953] [INFO ] [MAIN ] DDL: Wiping existing primary data before full refresh. [2025-11-04T18:22:46.653220] [INFO ] [LOCAL ] DDL: Starting deletion of primary data. [2025-11-04T18:22:46.653091] [INFO ] [MAIN ] DDS: Starting full primary refresh. [2025-11-04T18:22:46.736625] [DEBUG] [LOCAL ] DDL: Deleted primary data file. [2025-11-04T18:22:46.787338] [INFO ] [MAIN ] DDL: Starting full primary refresh. [2025-11-04T18:22:47.095025] [WARN ] [MAIN ] DE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:22:47.126848] [INFO ] [MAIN ] DE: Wiping existing primary data before full refresh. [2025-11-04T18:22:47.147554] [INFO ] [LOCAL ] DE: Starting deletion of primary data. [2025-11-04T18:22:47.252183] [DEBUG] [LOCAL ] DE: Deleted primary data file. [2025-11-04T18:22:47.317562] [INFO ] [MAIN ] DE: Starting full primary refresh. [2025-11-04T18:22:47.817803] [INFO ] [REMOTE ] DDS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:22:47.818238] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-01. [2025-11-04T18:22:47.807092] [INFO ] [REMOTE ] DDT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:22:47.819601] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-01. [2025-11-04T18:22:48.045923] [INFO ] [REMOTE ] DDL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:22:48.067967] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-01. [2025-11-04T18:22:48.184440] [INFO ] [REMOTE ] DE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:22:48.184792] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-01. [2025-11-04T18:22:48.375102] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T18:22:48.376189] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-02. [2025-11-04T18:22:48.623905] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T18:22:48.627279] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-02. [2025-11-04T18:22:48.686303] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-11-04T18:22:48.687617] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-03. [2025-11-04T18:22:49.004417] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-11-04T18:22:49.006226] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-04. [2025-11-04T18:22:49.273285] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T18:22:49.280879] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-02. [2025-11-04T18:22:49.293991] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-11-04T18:22:49.333420] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-11-04T18:22:49.585030] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T18:22:49.767057] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-03. [2025-11-04T18:22:50.253242] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T18:22:50.470303] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T18:22:50.567008] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-04. [2025-11-04T18:22:51.316343] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T18:22:51.367056] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-05. [2025-11-04T18:22:52.056788] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-11-04T18:22:52.167065] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-06. [2025-11-04T18:22:52.813797] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-11-04T18:22:52.967023] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-07. [2025-11-04T18:22:53.725629] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-11-04T18:22:53.767051] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-08. [2025-11-04T18:22:54.470530] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T18:22:54.567055] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-02. [2025-11-04T18:22:55.367052] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-09. [2025-11-04T18:22:56.063340] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-11-04T18:22:56.167050] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-03. [2025-11-04T18:22:56.228915] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-11-04T18:22:56.967050] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-10. [2025-11-04T18:22:57.239695] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T18:22:57.501878] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-11-04T18:22:57.767019] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-11. [2025-11-04T18:22:58.369810] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-11 (0.60 seconds). [2025-11-04T18:22:58.370855] [INFO ] [REMOTE ] DDS: Full fetch complete (18223 records). [2025-11-04T18:22:58.370925] [DEBUG] [VALIDATOR ] DDS: Validating primary data. [2025-11-04T18:22:58.574588] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-04. [2025-11-04T18:22:58.758382] [DEBUG] [VALIDATOR ] DDS: All expected trading days are present. [2025-11-04T18:22:58.758474] [DEBUG] [VALIDATOR ] DDS: Primary data validation passed. [2025-11-04T18:22:58.758869] [INFO ] [MAIN ] DDS: Fetched primary data is valid. [2025-11-04T18:22:58.758930] [DEBUG] [LOCAL ] DDS: Saving primary data. [2025-11-04T18:22:58.759769] [DEBUG] [LOCAL ] DDS: Trimmed 18223 records to 11808 (keeping 200 days of data) [2025-11-04T18:22:58.837338] [DEBUG] [LOCAL ] DDS: Primary data saved (11808 records). [2025-11-04T18:22:58.837490] [INFO ] [MAIN ] DDS: Full history saved. [2025-11-04T18:22:58.837568] [INFO ] [MAIN ] Update Progress: 601/2367 symbols completed. DDS took 13.7s. Estimated remaining: 4h 3m 40s. Rate: 7.2 symbols/min [2025-11-04T18:22:59.013160] [INFO ] [MAIN ] DEA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:22:59.055101] [DEBUG] [LOCAL ] DEA: Loaded 39407 existing records from primary storage. [2025-11-04T18:22:59.055405] [INFO ] [MAIN ] DEA: Loaded 39407 existing records. [2025-11-04T18:22:59.256883] [WARN ] [MAIN ] DEA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:22:59.257164] [INFO ] [MAIN ] DEA: Wiping existing primary data before full refresh. [2025-11-04T18:22:59.257216] [INFO ] [LOCAL ] DEA: Starting deletion of primary data. [2025-11-04T18:22:59.257398] [DEBUG] [LOCAL ] DEA: Deleted primary data file. [2025-11-04T18:22:59.257455] [INFO ] [MAIN ] DEA: Starting full primary refresh. [2025-11-04T18:22:59.367039] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-03. [2025-11-04T18:22:59.638215] [INFO ] [REMOTE ] DEA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:23:00.167043] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-05. [2025-11-04T18:23:00.967080] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-01. [2025-11-04T18:23:04.113635] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-05 (3.95 seconds). [2025-11-04T18:23:04.114752] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-06. [2025-11-04T18:23:05.155100] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T18:23:05.156413] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-07. [2025-11-04T18:23:05.317779] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-04 (6.74 seconds). [2025-11-04T18:23:05.329865] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-05. [2025-11-04T18:23:05.843718] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-11-04T18:23:05.845027] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-08. [2025-11-04T18:23:06.192968] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-11-04T18:23:06.194049] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-09. [2025-11-04T18:23:06.449174] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-03 (7.08 seconds). [2025-11-04T18:23:06.460212] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-04. [2025-11-04T18:23:06.513626] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-11-04T18:23:06.567048] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-10. [2025-11-04T18:23:06.610066] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-01 (5.64 seconds). [2025-11-04T18:23:06.901569] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-10 (0.33 seconds). [2025-11-04T18:23:07.367063] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-11. [2025-11-04T18:23:07.530277] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-05 (2.20 seconds). [2025-11-04T18:23:07.805799] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-11 (0.44 seconds). [2025-11-04T18:23:07.806470] [INFO ] [REMOTE ] DDT: Full fetch complete (2814 records). [2025-11-04T18:23:07.806538] [DEBUG] [VALIDATOR ] DDT: Validating primary data. [2025-11-04T18:23:08.178828] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-06. [2025-11-04T18:23:08.186941] [DEBUG] [VALIDATOR ] DDT: All expected trading days are present. [2025-11-04T18:23:08.187112] [DEBUG] [VALIDATOR ] DDT: Primary data validation passed. [2025-11-04T18:23:08.187325] [INFO ] [MAIN ] DDT: Fetched primary data is valid. [2025-11-04T18:23:08.187431] [DEBUG] [LOCAL ] DDT: Saving primary data. [2025-11-04T18:23:08.187959] [DEBUG] [LOCAL ] DDT: Trimmed 2814 records to 1778 (keeping 200 days of data) [2025-11-04T18:23:08.200113] [DEBUG] [LOCAL ] DDT: Primary data saved (1778 records). [2025-11-04T18:23:08.200226] [INFO ] [MAIN ] DDT: Full history saved. [2025-11-04T18:23:08.200427] [INFO ] [MAIN ] Update Progress: 602/2367 symbols completed. DDT took 23.1s. Estimated remaining: 4h 3m 35s. Rate: 7.2 symbols/min [2025-11-04T18:23:08.386909] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-11-04T18:23:08.418685] [INFO ] [MAIN ] DEC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:23:08.449430] [DEBUG] [LOCAL ] DEC: Loaded 29761 existing records from primary storage. [2025-11-04T18:23:08.449795] [INFO ] [MAIN ] DEC: Loaded 29761 existing records. [2025-11-04T18:23:08.631992] [WARN ] [MAIN ] DEC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:23:08.632213] [INFO ] [MAIN ] DEC: Wiping existing primary data before full refresh. [2025-11-04T18:23:08.632270] [INFO ] [LOCAL ] DEC: Starting deletion of primary data. [2025-11-04T18:23:08.632455] [DEBUG] [LOCAL ] DEC: Deleted primary data file. [2025-11-04T18:23:08.632512] [INFO ] [MAIN ] DEC: Starting full primary refresh. [2025-11-04T18:23:08.972907] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-02. [2025-11-04T18:23:08.995932] [INFO ] [REMOTE ] DEC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:23:09.016248] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T18:23:09.767100] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-01. [2025-11-04T18:23:10.382251] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-11-04T18:23:10.567065] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-05. [2025-11-04T18:23:10.717359] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T18:23:11.367039] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-02. [2025-11-04T18:23:12.039217] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-11-04T18:23:12.173697] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-03. [2025-11-04T18:23:12.175765] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-11-04T18:23:12.967019] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-03. [2025-11-04T18:23:13.500584] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T18:23:13.767035] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-04. [2025-11-04T18:23:13.977209] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T18:23:14.567063] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-06. [2025-11-04T18:23:15.059284] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T18:23:15.367043] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-07. [2025-11-04T18:23:15.916305] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-11-04T18:23:16.167083] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-07. [2025-11-04T18:23:16.499981] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T18:23:16.967092] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-04. [2025-11-04T18:23:17.674470] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T18:23:17.767086] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-08. [2025-11-04T18:23:17.829147] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T18:23:18.567051] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-05. [2025-11-04T18:23:18.859754] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-11-04T18:23:19.367034] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-08. [2025-11-04T18:23:19.793501] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T18:23:20.167011] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-05. [2025-11-04T18:23:20.863440] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-11-04T18:23:20.967004] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-09. [2025-11-04T18:23:21.137326] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-11-04T18:23:21.767031] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-06. [2025-11-04T18:23:22.431122] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-11-04T18:23:22.567041] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-09. [2025-11-04T18:23:22.836001] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T18:23:23.367038] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-06. [2025-11-04T18:23:23.747746] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:23:24.167048] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-07. [2025-11-04T18:23:24.376011] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-11-04T18:23:24.967007] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-10. [2025-11-04T18:23:25.165154] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T18:23:25.767080] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-07. [2025-11-04T18:23:26.123093] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T18:23:26.567048] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-11. [2025-11-04T18:23:26.847603] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T18:23:27.367051] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-08. [2025-11-04T18:23:27.670418] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-11 (1.10 seconds). [2025-11-04T18:23:27.672134] [INFO ] [REMOTE ] DDL: Full fetch complete (54288 records). [2025-11-04T18:23:27.672259] [DEBUG] [VALIDATOR ] DDL: Validating primary data. [2025-11-04T18:23:28.077036] [DEBUG] [VALIDATOR ] DDL: All expected trading days are present. [2025-11-04T18:23:28.077114] [DEBUG] [VALIDATOR ] DDL: Primary data validation passed. [2025-11-04T18:23:28.077775] [INFO ] [MAIN ] DDL: Fetched primary data is valid. [2025-11-04T18:23:28.077837] [DEBUG] [LOCAL ] DDL: Saving primary data. [2025-11-04T18:23:28.078910] [DEBUG] [LOCAL ] DDL: Trimmed 54288 records to 36682 (keeping 200 days of data) [2025-11-04T18:23:28.167052] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-10. [2025-11-04T18:23:28.282178] [DEBUG] [LOCAL ] DDL: Primary data saved (36682 records). [2025-11-04T18:23:28.282356] [INFO ] [MAIN ] DDL: Full history saved. [2025-11-04T18:23:28.282484] [INFO ] [MAIN ] Update Progress: 603/2367 symbols completed. DDL took 43.2s. Estimated remaining: 4h 4m 1s. Rate: 7.2 symbols/min [2025-11-04T18:23:28.459638] [INFO ] [MAIN ] DECK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:23:28.513428] [DEBUG] [LOCAL ] DECK: Loaded 53311 existing records from primary storage. [2025-11-04T18:23:28.513772] [INFO ] [MAIN ] DECK: Loaded 53311 existing records. [2025-11-04T18:23:28.734850] [WARN ] [MAIN ] DECK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:23:28.735081] [INFO ] [MAIN ] DECK: Wiping existing primary data before full refresh. [2025-11-04T18:23:28.735233] [INFO ] [LOCAL ] DECK: Starting deletion of primary data. [2025-11-04T18:23:28.743250] [DEBUG] [LOCAL ] DECK: Deleted primary data file. [2025-11-04T18:23:28.755977] [INFO ] [MAIN ] DECK: Starting full primary refresh. [2025-11-04T18:23:28.789380] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T18:23:28.991485] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-09. [2025-11-04T18:23:29.223152] [INFO ] [REMOTE ] DECK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:23:29.767041] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-08. [2025-11-04T18:23:29.816199] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-11-04T18:23:30.321631] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T18:23:30.567026] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-11. [2025-11-04T18:23:30.654095] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T18:23:30.923149] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T18:23:30.925089] [INFO ] [REMOTE ] DE: Full fetch complete (69818 records). [2025-11-04T18:23:30.925185] [DEBUG] [VALIDATOR ] DE: Validating primary data. [2025-11-04T18:23:31.320511] [DEBUG] [VALIDATOR ] DE: All expected trading days are present. [2025-11-04T18:23:31.320589] [DEBUG] [VALIDATOR ] DE: Primary data validation passed. [2025-11-04T18:23:31.321550] [INFO ] [MAIN ] DE: Fetched primary data is valid. [2025-11-04T18:23:31.321621] [DEBUG] [LOCAL ] DE: Saving primary data. [2025-11-04T18:23:31.323012] [DEBUG] [LOCAL ] DE: Trimmed 69818 records to 47342 (keeping 200 days of data) [2025-11-04T18:23:31.385273] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-09. [2025-11-04T18:23:31.625955] [DEBUG] [LOCAL ] DE: Primary data saved (47342 records). [2025-11-04T18:23:31.626078] [INFO ] [MAIN ] DE: Full history saved. [2025-11-04T18:23:31.626154] [INFO ] [MAIN ] Update Progress: 604/2367 symbols completed. DE took 46.5s. Estimated remaining: 4h 3m 39s. Rate: 7.2 symbols/min [2025-11-04T18:23:31.798341] [INFO ] [MAIN ] DEI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:23:31.843079] [DEBUG] [LOCAL ] DEI: Loaded 48175 existing records from primary storage. [2025-11-04T18:23:31.843360] [INFO ] [MAIN ] DEI: Loaded 48175 existing records. [2025-11-04T18:23:32.022373] [WARN ] [MAIN ] DEI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:23:32.022591] [INFO ] [MAIN ] DEI: Wiping existing primary data before full refresh. [2025-11-04T18:23:32.022640] [INFO ] [LOCAL ] DEI: Starting deletion of primary data. [2025-11-04T18:23:32.022857] [DEBUG] [LOCAL ] DEI: Deleted primary data file. [2025-11-04T18:23:32.022897] [INFO ] [MAIN ] DEI: Starting full primary refresh. [2025-11-04T18:23:32.176362] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-01. [2025-11-04T18:23:32.392816] [INFO ] [REMOTE ] DEI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:23:32.410543] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-11-04T18:23:32.967046] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-10. [2025-11-04T18:23:33.744068] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-11-04T18:23:33.767047] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-10. [2025-11-04T18:23:33.868191] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-11-04T18:23:34.567083] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-01. [2025-11-04T18:23:35.071562] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-11-04T18:23:35.367068] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-11. [2025-11-04T18:23:35.973621] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T18:23:36.167056] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-02. [2025-11-04T18:23:36.434856] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T18:23:36.436544] [INFO ] [REMOTE ] DEA: Full fetch complete (61321 records). [2025-11-04T18:23:36.436623] [DEBUG] [VALIDATOR ] DEA: Validating primary data. [2025-11-04T18:23:36.831720] [DEBUG] [VALIDATOR ] DEA: All expected trading days are present. [2025-11-04T18:23:36.831804] [DEBUG] [VALIDATOR ] DEA: Primary data validation passed. [2025-11-04T18:23:36.832540] [INFO ] [MAIN ] DEA: Fetched primary data is valid. [2025-11-04T18:23:36.832581] [DEBUG] [LOCAL ] DEA: Saving primary data. [2025-11-04T18:23:36.833861] [DEBUG] [LOCAL ] DEA: Trimmed 61321 records to 39304 (keeping 200 days of data) [2025-11-04T18:23:36.997756] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-02. [2025-11-04T18:23:37.064600] [DEBUG] [LOCAL ] DEA: Primary data saved (39304 records). [2025-11-04T18:23:37.064754] [INFO ] [MAIN ] DEA: Full history saved. [2025-11-04T18:23:37.065299] [INFO ] [MAIN ] Update Progress: 605/2367 symbols completed. DEA took 38.2s. Estimated remaining: 4h 3m 22s. Rate: 7.2 symbols/min [2025-11-04T18:23:37.235631] [INFO ] [MAIN ] DELL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:23:37.287183] [DEBUG] [LOCAL ] DELL: Loaded 53735 existing records from primary storage. [2025-11-04T18:23:37.287483] [INFO ] [MAIN ] DELL: Loaded 53735 existing records. [2025-11-04T18:23:37.471129] [WARN ] [MAIN ] DELL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:23:37.471396] [INFO ] [MAIN ] DELL: Wiping existing primary data before full refresh. [2025-11-04T18:23:37.471498] [INFO ] [LOCAL ] DELL: Starting deletion of primary data. [2025-11-04T18:23:37.479890] [DEBUG] [LOCAL ] DELL: Deleted primary data file. [2025-11-04T18:23:37.491020] [INFO ] [MAIN ] DELL: Starting full primary refresh. [2025-11-04T18:23:37.564797] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-11-04T18:23:37.776953] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-03. [2025-11-04T18:23:37.858103] [INFO ] [REMOTE ] DELL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:23:38.567011] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-11. [2025-11-04T18:23:38.774260] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-11-04T18:23:39.228692] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-11-04T18:23:39.367049] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-04. [2025-11-04T18:23:39.482451] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T18:23:39.483918] [INFO ] [REMOTE ] DEC: Full fetch complete (44964 records). [2025-11-04T18:23:39.483989] [DEBUG] [VALIDATOR ] DEC: Validating primary data. [2025-11-04T18:23:39.852452] [DEBUG] [VALIDATOR ] DEC: All expected trading days are present. [2025-11-04T18:23:39.852525] [DEBUG] [VALIDATOR ] DEC: Primary data validation passed. [2025-11-04T18:23:39.853105] [INFO ] [MAIN ] DEC: Fetched primary data is valid. [2025-11-04T18:23:39.853167] [DEBUG] [LOCAL ] DEC: Saving primary data. [2025-11-04T18:23:39.854036] [DEBUG] [LOCAL ] DEC: Trimmed 44964 records to 29703 (keeping 200 days of data) [2025-11-04T18:23:40.025157] [DEBUG] [LOCAL ] DEC: Primary data saved (29703 records). [2025-11-04T18:23:40.025290] [INFO ] [MAIN ] DEC: Full history saved. [2025-11-04T18:23:40.025366] [INFO ] [MAIN ] Update Progress: 606/2367 symbols completed. DEC took 31.8s. Estimated remaining: 4h 2m 58s. Rate: 7.2 symbols/min [2025-11-04T18:23:40.175367] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-03. [2025-11-04T18:23:40.194427] [INFO ] [MAIN ] DEO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:23:40.237465] [DEBUG] [LOCAL ] DEO: Loaded 49216 existing records from primary storage. [2025-11-04T18:23:40.237774] [INFO ] [MAIN ] DEO: Loaded 49216 existing records. [2025-11-04T18:23:40.416338] [WARN ] [MAIN ] DEO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:23:40.416520] [INFO ] [MAIN ] DEO: Wiping existing primary data before full refresh. [2025-11-04T18:23:40.416565] [INFO ] [LOCAL ] DEO: Starting deletion of primary data. [2025-11-04T18:23:40.416783] [DEBUG] [LOCAL ] DEO: Deleted primary data file. [2025-11-04T18:23:40.416866] [INFO ] [MAIN ] DEO: Starting full primary refresh. [2025-11-04T18:23:40.775569] [INFO ] [REMOTE ] DEO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:23:40.802498] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T18:23:40.967036] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-01. [2025-11-04T18:23:41.767819] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-05. [2025-11-04T18:23:41.773798] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-11-04T18:23:41.834320] [DEBUG] [REMOTE ] DEI: Error fetching primary data for 2025-05: 'API call returned an empty response.' [2025-11-04T18:23:41.834631] [WARN ] [REMOTE ] DEI: No data returned for month 2025-05 [2025-11-04T18:23:41.834756] [FATAL] [REMOTE ] DEI: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:23:41.834853] [FATAL] [MAIN ] DEI: Failed to fetch primary data. [2025-11-04T18:23:41.834921] [WARN ] [MAIN ] DEI: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:23:41.834991] [INFO ] [LOCAL ] DEI: Starting deletion of primary data. [2025-11-04T18:23:41.835057] [DEBUG] [LOCAL ] DEI: No primary data file found to delete. [2025-11-04T18:23:41.835093] [INFO ] [LOCAL ] DEI: Starting deletion of derivative data. [2025-11-04T18:23:41.835216] [DEBUG] [LOCAL ] DEI: Deleted derivative data file. [2025-11-04T18:23:41.835279] [INFO ] [MAIN ] Update Progress: 607/2367 symbols completed. DEI took 10.2s. Estimated remaining: 4h 2m 31s. Rate: 7.3 symbols/min [2025-11-04T18:23:42.011166] [INFO ] [MAIN ] DFH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:23:42.040261] [DEBUG] [LOCAL ] DFH: Loaded 30931 existing records from primary storage. [2025-11-04T18:23:42.040532] [INFO ] [MAIN ] DFH: Loaded 30931 existing records. [2025-11-04T18:23:42.216074] [WARN ] [MAIN ] DFH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:23:42.216309] [INFO ] [MAIN ] DFH: Wiping existing primary data before full refresh. [2025-11-04T18:23:42.216357] [INFO ] [LOCAL ] DFH: Starting deletion of primary data. [2025-11-04T18:23:42.216530] [DEBUG] [LOCAL ] DFH: Deleted primary data file. [2025-11-04T18:23:42.216568] [INFO ] [MAIN ] DFH: Starting full primary refresh. [2025-11-04T18:23:42.573866] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-01. [2025-11-04T18:23:42.579715] [INFO ] [REMOTE ] DFH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:23:42.608554] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-11-04T18:23:43.367067] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-04. [2025-11-04T18:23:44.167052] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-02. [2025-11-04T18:23:44.686804] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-11-04T18:23:44.967067] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-01. [2025-11-04T18:23:45.066052] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-11-04T18:23:45.686582] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-11-04T18:23:45.767020] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-03. [2025-11-04T18:23:46.248436] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T18:23:46.567087] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-02. [2025-11-04T18:23:47.348201] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-11-04T18:23:47.367007] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-04. [2025-11-04T18:23:48.167070] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-02. [2025-11-04T18:23:48.516205] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-11-04T18:23:48.725395] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T18:23:48.967001] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-05. [2025-11-04T18:23:49.096461] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-11-04T18:23:49.767042] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-05. [2025-11-04T18:23:50.485940] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-11-04T18:23:50.567016] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-03. [2025-11-04T18:23:51.367037] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-03. [2025-11-04T18:23:51.477956] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-11-04T18:23:52.167034] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-06. [2025-11-04T18:23:52.468215] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T18:23:52.516417] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-11-04T18:23:52.967111] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-04. [2025-11-04T18:23:53.775499] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-04. [2025-11-04T18:23:53.796085] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-11-04T18:23:54.567048] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-07. [2025-11-04T18:23:54.812454] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-11-04T18:23:55.153791] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-11-04T18:23:55.367050] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-05. [2025-11-04T18:23:56.167056] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-06. [2025-11-04T18:23:56.265251] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-11-04T18:23:56.967117] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-05. [2025-11-04T18:23:57.671415] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-05 (2.30 seconds). [2025-11-04T18:23:57.767050] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-08. [2025-11-04T18:23:57.971472] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-11-04T18:23:58.003037] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-11-04T18:23:58.567058] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-07. [2025-11-04T18:23:59.343666] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-11-04T18:23:59.367012] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-09. [2025-11-04T18:24:00.174822] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-06. [2025-11-04T18:24:00.177959] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-11-04T18:24:00.967018] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-06. [2025-11-04T18:24:01.222853] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-11-04T18:24:01.350375] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T18:24:01.767116] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-07. [2025-11-04T18:24:02.567033] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-10. [2025-11-04T18:24:02.793572] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T18:24:02.983454] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-06 (2.02 seconds). [2025-11-04T18:24:03.367062] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-08. [2025-11-04T18:24:04.167061] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-07. [2025-11-04T18:24:04.527109] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-11-04T18:24:04.855597] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T18:24:04.967059] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-09. [2025-11-04T18:24:05.767113] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-11. [2025-11-04T18:24:06.204775] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-11-04T18:24:06.449231] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-11-04T18:24:06.567108] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-10. [2025-11-04T18:24:07.367058] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-08. [2025-11-04T18:24:07.496595] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-11 (1.73 seconds). [2025-11-04T18:24:07.498602] [INFO ] [REMOTE ] DECK: Full fetch complete (77340 records). [2025-11-04T18:24:07.498731] [DEBUG] [VALIDATOR ] DECK: Validating primary data. [2025-11-04T18:24:07.894464] [DEBUG] [VALIDATOR ] DECK: All expected trading days are present. [2025-11-04T18:24:07.894568] [DEBUG] [VALIDATOR ] DECK: Primary data validation passed. [2025-11-04T18:24:07.895406] [INFO ] [MAIN ] DECK: Fetched primary data is valid. [2025-11-04T18:24:07.895469] [DEBUG] [LOCAL ] DECK: Saving primary data. [2025-11-04T18:24:07.897131] [DEBUG] [LOCAL ] DECK: Trimmed 77340 records to 53314 (keeping 200 days of data) [2025-11-04T18:24:08.169580] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-08. [2025-11-04T18:24:08.219991] [DEBUG] [LOCAL ] DECK: Primary data saved (53314 records). [2025-11-04T18:24:08.220451] [INFO ] [MAIN ] DECK: Full history saved. [2025-11-04T18:24:08.220717] [INFO ] [MAIN ] Update Progress: 608/2367 symbols completed. DECK took 39.9s. Estimated remaining: 4h 3m 15s. Rate: 7.2 symbols/min [2025-11-04T18:24:08.415841] [INFO ] [MAIN ] DFIN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:24:08.435703] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T18:24:08.461142] [DEBUG] [LOCAL ] DFIN: Loaded 22065 existing records from primary storage. [2025-11-04T18:24:08.461407] [INFO ] [MAIN ] DFIN: Loaded 22065 existing records. [2025-11-04T18:24:08.703221] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-10 (2.14 seconds). [2025-11-04T18:24:08.757965] [WARN ] [MAIN ] DFIN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:24:08.758625] [INFO ] [MAIN ] DFIN: Wiping existing primary data before full refresh. [2025-11-04T18:24:08.758732] [INFO ] [LOCAL ] DFIN: Starting deletion of primary data. [2025-11-04T18:24:08.759211] [DEBUG] [LOCAL ] DFIN: Deleted primary data file. [2025-11-04T18:24:08.759268] [INFO ] [MAIN ] DFIN: Starting full primary refresh. [2025-11-04T18:24:08.973088] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-11. [2025-11-04T18:24:09.106992] [INFO ] [REMOTE ] DFIN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:24:09.767093] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-01. [2025-11-04T18:24:10.145407] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-11-04T18:24:10.342746] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-11 (1.37 seconds). [2025-11-04T18:24:10.344721] [INFO ] [REMOTE ] DEO: Full fetch complete (71758 records). [2025-11-04T18:24:10.344811] [DEBUG] [VALIDATOR ] DEO: Validating primary data. [2025-11-04T18:24:10.574616] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-09. [2025-11-04T18:24:10.768623] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T18:24:10.796226] [DEBUG] [VALIDATOR ] DEO: All expected trading days are present. [2025-11-04T18:24:10.796322] [DEBUG] [VALIDATOR ] DEO: Primary data validation passed. [2025-11-04T18:24:10.797157] [INFO ] [MAIN ] DEO: Fetched primary data is valid. [2025-11-04T18:24:10.797219] [DEBUG] [LOCAL ] DEO: Saving primary data. [2025-11-04T18:24:10.798553] [DEBUG] [LOCAL ] DEO: Trimmed 71758 records to 49205 (keeping 200 days of data) [2025-11-04T18:24:11.094739] [DEBUG] [LOCAL ] DEO: Primary data saved (49205 records). [2025-11-04T18:24:11.094860] [INFO ] [MAIN ] DEO: Full history saved. [2025-11-04T18:24:11.094935] [INFO ] [MAIN ] Update Progress: 609/2367 symbols completed. DEO took 31.1s. Estimated remaining: 4h 2m 51s. Rate: 7.2 symbols/min [2025-11-04T18:24:11.263117] [INFO ] [MAIN ] DFP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:24:11.274212] [DEBUG] [LOCAL ] DFP: Loaded 10642 existing records from primary storage. [2025-11-04T18:24:11.274399] [INFO ] [MAIN ] DFP: Loaded 10642 existing records. [2025-11-04T18:24:11.372774] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-09. [2025-11-04T18:24:11.449591] [WARN ] [MAIN ] DFP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:24:11.449811] [INFO ] [MAIN ] DFP: Wiping existing primary data before full refresh. [2025-11-04T18:24:11.449860] [INFO ] [LOCAL ] DFP: Starting deletion of primary data. [2025-11-04T18:24:11.450058] [DEBUG] [LOCAL ] DFP: Deleted primary data file. [2025-11-04T18:24:11.450095] [INFO ] [MAIN ] DFP: Starting full primary refresh. [2025-11-04T18:24:11.801957] [INFO ] [REMOTE ] DFP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:24:12.167071] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-02. [2025-11-04T18:24:12.397107] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T18:24:12.625266] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-11-04T18:24:12.967029] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-10. [2025-11-04T18:24:12.989848] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-11-04T18:24:13.767065] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-03. [2025-11-04T18:24:14.013712] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-11-04T18:24:14.567014] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-10. [2025-11-04T18:24:14.725787] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T18:24:15.367072] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-11. [2025-11-04T18:24:16.167019] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-01. [2025-11-04T18:24:16.213564] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-11 (0.85 seconds). [2025-11-04T18:24:16.215081] [INFO ] [REMOTE ] DFH: Full fetch complete (46950 records). [2025-11-04T18:24:16.215173] [DEBUG] [VALIDATOR ] DFH: Validating primary data. [2025-11-04T18:24:16.592804] [DEBUG] [VALIDATOR ] DFH: All expected trading days are present. [2025-11-04T18:24:16.592898] [DEBUG] [VALIDATOR ] DFH: Primary data validation passed. [2025-11-04T18:24:16.593549] [INFO ] [MAIN ] DFH: Fetched primary data is valid. [2025-11-04T18:24:16.593614] [DEBUG] [LOCAL ] DFH: Saving primary data. [2025-11-04T18:24:16.594671] [DEBUG] [LOCAL ] DFH: Trimmed 46950 records to 30922 (keeping 200 days of data) [2025-11-04T18:24:16.763318] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-11-04T18:24:16.787030] [DEBUG] [LOCAL ] DFH: Primary data saved (30922 records). [2025-11-04T18:24:16.787311] [INFO ] [MAIN ] DFH: Full history saved. [2025-11-04T18:24:16.787498] [INFO ] [MAIN ] Update Progress: 610/2367 symbols completed. DFH took 35.0s. Estimated remaining: 4h 2m 36s. Rate: 7.2 symbols/min [2025-11-04T18:24:16.965373] [INFO ] [MAIN ] DG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:24:16.977888] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-04. [2025-11-04T18:24:17.002745] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-10 (2.44 seconds). [2025-11-04T18:24:17.062727] [DEBUG] [LOCAL ] DG: Loaded 53158 existing records from primary storage. [2025-11-04T18:24:17.063105] [INFO ] [MAIN ] DG: Loaded 53158 existing records. [2025-11-04T18:24:17.286604] [WARN ] [MAIN ] DG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:24:17.286920] [INFO ] [MAIN ] DG: Wiping existing primary data before full refresh. [2025-11-04T18:24:17.286980] [INFO ] [LOCAL ] DG: Starting deletion of primary data. [2025-11-04T18:24:17.287216] [DEBUG] [LOCAL ] DG: Deleted primary data file. [2025-11-04T18:24:17.287297] [INFO ] [MAIN ] DG: Starting full primary refresh. [2025-11-04T18:24:17.692981] [INFO ] [REMOTE ] DG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:24:17.767034] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-11. [2025-11-04T18:24:17.917903] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-11-04T18:24:18.567072] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-05. [2025-11-04T18:24:19.266918] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-11-04T18:24:19.367064] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-02. [2025-11-04T18:24:19.573280] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-11 (1.81 seconds). [2025-11-04T18:24:19.574827] [INFO ] [REMOTE ] DELL: Full fetch complete (77524 records). [2025-11-04T18:24:19.574918] [DEBUG] [VALIDATOR ] DELL: Validating primary data. [2025-11-04T18:24:19.575705] [FATAL] [VALIDATOR ] DELL: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:24:19.575788] [FATAL] [MAIN ] DELL: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:24:19.575830] [WARN ] [MAIN ] DELL: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:24:19.575894] [INFO ] [LOCAL ] DELL: Starting deletion of primary data. [2025-11-04T18:24:19.575954] [DEBUG] [LOCAL ] DELL: No primary data file found to delete. [2025-11-04T18:24:19.575991] [INFO ] [LOCAL ] DELL: Starting deletion of derivative data. [2025-11-04T18:24:19.576116] [DEBUG] [LOCAL ] DELL: Deleted derivative data file. [2025-11-04T18:24:19.576183] [INFO ] [MAIN ] Update Progress: 611/2367 symbols completed. DELL took 42.5s. Estimated remaining: 4h 2m 12s. Rate: 7.3 symbols/min [2025-11-04T18:24:19.747335] [INFO ] [MAIN ] DGX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:24:19.747532] [DEBUG] [LOCAL ] DGX: No existing data file found in primary storage. [2025-11-04T18:24:19.747587] [WARN ] [MAIN ] DGX: No existing data found, performing full refresh instead. [2025-11-04T18:24:19.747716] [INFO ] [MAIN ] DGX: Wiping existing primary data before full refresh. [2025-11-04T18:24:19.747762] [INFO ] [LOCAL ] DGX: Starting deletion of primary data. [2025-11-04T18:24:19.747806] [DEBUG] [LOCAL ] DGX: No primary data file found to delete. [2025-11-04T18:24:19.747838] [INFO ] [MAIN ] DGX: Starting full primary refresh. [2025-11-04T18:24:19.929751] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-11-04T18:24:20.111596] [INFO ] [REMOTE ] DGX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:24:20.167033] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-01. [2025-11-04T18:24:20.967079] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-01. [2025-11-04T18:24:21.506238] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T18:24:21.767066] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-03. [2025-11-04T18:24:22.302603] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-11-04T18:24:22.567026] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-04. [2025-11-04T18:24:22.634859] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-11-04T18:24:23.081263] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-11-04T18:24:23.367050] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-05. [2025-11-04T18:24:23.959624] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T18:24:24.167046] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-06. [2025-11-04T18:24:24.782830] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-11-04T18:24:24.967040] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-02. [2025-11-04T18:24:25.767039] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-07. [2025-11-04T18:24:26.371819] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T18:24:26.482748] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-11-04T18:24:26.567018] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-08. [2025-11-04T18:24:27.194207] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-11-04T18:24:27.367037] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-09. [2025-11-04T18:24:27.928128] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T18:24:28.167069] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-03. [2025-11-04T18:24:28.967090] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-10. [2025-11-04T18:24:29.539945] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-11-04T18:24:29.762040] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-11-04T18:24:29.772473] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-11. [2025-11-04T18:24:30.287404] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-11 (0.51 seconds). [2025-11-04T18:24:30.288479] [INFO ] [REMOTE ] DFP: Full fetch complete (15817 records). [2025-11-04T18:24:30.288568] [DEBUG] [VALIDATOR ] DFP: Validating primary data. [2025-11-04T18:24:30.572102] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-04. [2025-11-04T18:24:30.675307] [DEBUG] [VALIDATOR ] DFP: All expected trading days are present. [2025-11-04T18:24:30.675383] [DEBUG] [VALIDATOR ] DFP: Primary data validation passed. [2025-11-04T18:24:30.675778] [INFO ] [MAIN ] DFP: Fetched primary data is valid. [2025-11-04T18:24:30.675840] [DEBUG] [LOCAL ] DFP: Saving primary data. [2025-11-04T18:24:30.676366] [DEBUG] [LOCAL ] DFP: Trimmed 15817 records to 10638 (keeping 200 days of data) [2025-11-04T18:24:30.739890] [DEBUG] [LOCAL ] DFP: Primary data saved (10638 records). [2025-11-04T18:24:30.740015] [INFO ] [MAIN ] DFP: Full history saved. [2025-11-04T18:24:30.740089] [INFO ] [MAIN ] Update Progress: 612/2367 symbols completed. DFP took 19.6s. Estimated remaining: 4h 2m 12s. Rate: 7.2 symbols/min [2025-11-04T18:24:30.907917] [INFO ] [MAIN ] DHF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:24:30.926911] [DEBUG] [LOCAL ] DHF: Loaded 20277 existing records from primary storage. [2025-11-04T18:24:30.927149] [INFO ] [MAIN ] DHF: Loaded 20277 existing records. [2025-11-04T18:24:31.112066] [WARN ] [MAIN ] DHF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:24:31.112358] [INFO ] [MAIN ] DHF: Wiping existing primary data before full refresh. [2025-11-04T18:24:31.112448] [INFO ] [LOCAL ] DHF: Starting deletion of primary data. [2025-11-04T18:24:31.112688] [DEBUG] [LOCAL ] DHF: Deleted primary data file. [2025-11-04T18:24:31.112746] [INFO ] [MAIN ] DHF: Starting full primary refresh. [2025-11-04T18:24:31.374074] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-02. [2025-11-04T18:24:31.501026] [INFO ] [REMOTE ] DHF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:24:32.167017] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-06. [2025-11-04T18:24:32.361931] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-11-04T18:24:32.757802] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-11-04T18:24:32.915219] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-11-04T18:24:32.967043] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-07. [2025-11-04T18:24:33.711379] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T18:24:33.767191] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-05. [2025-11-04T18:24:34.567052] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-03. [2025-11-04T18:24:35.367057] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-01. [2025-11-04T18:24:35.542604] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-11-04T18:24:35.882189] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T18:24:36.167031] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-08. [2025-11-04T18:24:36.208728] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-11-04T18:24:36.967091] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-04. [2025-11-04T18:24:37.168009] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T18:24:37.767102] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-02. [2025-11-04T18:24:38.412403] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T18:24:38.484538] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T18:24:38.567056] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-05. [2025-11-04T18:24:39.367082] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-09. [2025-11-04T18:24:40.002873] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-11-04T18:24:40.073555] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-11-04T18:24:40.167088] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-06. [2025-11-04T18:24:40.967055] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-06. [2025-11-04T18:24:41.737941] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-11-04T18:24:41.767005] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-07. [2025-11-04T18:24:42.276237] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T18:24:42.567039] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-03. [2025-11-04T18:24:43.169007] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T18:24:43.288046] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-11-04T18:24:43.367057] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-10. [2025-11-04T18:24:44.167035] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-08. [2025-11-04T18:24:44.284212] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-11-04T18:24:44.967075] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-04. [2025-11-04T18:24:45.767025] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-07. [2025-11-04T18:24:45.848403] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-11-04T18:24:45.952621] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-11-04T18:24:46.567022] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-09. [2025-11-04T18:24:47.130297] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T18:24:47.367070] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-08. [2025-11-04T18:24:47.930750] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T18:24:48.167076] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-11. [2025-11-04T18:24:48.662099] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T18:24:48.967042] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-10. [2025-11-04T18:24:49.110219] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T18:24:49.111506] [INFO ] [REMOTE ] DFIN: Full fetch complete (34023 records). [2025-11-04T18:24:49.111604] [DEBUG] [VALIDATOR ] DFIN: Validating primary data. [2025-11-04T18:24:49.499477] [DEBUG] [VALIDATOR ] DFIN: All expected trading days are present. [2025-11-04T18:24:49.499581] [DEBUG] [VALIDATOR ] DFIN: Primary data validation passed. [2025-11-04T18:24:49.500145] [INFO ] [MAIN ] DFIN: Fetched primary data is valid. [2025-11-04T18:24:49.500205] [DEBUG] [LOCAL ] DFIN: Saving primary data. [2025-11-04T18:24:49.501018] [DEBUG] [LOCAL ] DFIN: Trimmed 34023 records to 22056 (keeping 200 days of data) [2025-11-04T18:24:49.631213] [DEBUG] [LOCAL ] DFIN: Primary data saved (22056 records). [2025-11-04T18:24:49.631331] [INFO ] [MAIN ] DFIN: Full history saved. [2025-11-04T18:24:49.631406] [INFO ] [MAIN ] Update Progress: 613/2367 symbols completed. DFIN took 41.4s. Estimated remaining: 4h 2m 34s. Rate: 7.2 symbols/min [2025-11-04T18:24:49.776128] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-09. [2025-11-04T18:24:49.804026] [INFO ] [MAIN ] DHI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:24:49.804140] [DEBUG] [LOCAL ] DHI: No existing data file found in primary storage. [2025-11-04T18:24:49.804177] [WARN ] [MAIN ] DHI: No existing data found, performing full refresh instead. [2025-11-04T18:24:49.804293] [INFO ] [MAIN ] DHI: Wiping existing primary data before full refresh. [2025-11-04T18:24:49.804355] [INFO ] [LOCAL ] DHI: Starting deletion of primary data. [2025-11-04T18:24:49.804412] [DEBUG] [LOCAL ] DHI: No primary data file found to delete. [2025-11-04T18:24:49.804457] [INFO ] [MAIN ] DHI: Starting full primary refresh. [2025-11-04T18:24:50.162108] [INFO ] [REMOTE ] DHI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:24:50.567056] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-01. [2025-11-04T18:24:51.206542] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-10 (2.24 seconds). [2025-11-04T18:24:51.367004] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-11. [2025-11-04T18:24:51.585167] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-11-04T18:24:52.167032] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-05. [2025-11-04T18:24:52.518412] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-11-04T18:24:52.834833] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-11 (1.47 seconds). [2025-11-04T18:24:52.837880] [INFO ] [REMOTE ] DG: Full fetch complete (77228 records). [2025-11-04T18:24:52.838134] [DEBUG] [VALIDATOR ] DG: Validating primary data. [2025-11-04T18:24:52.972737] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-02. [2025-11-04T18:24:53.219449] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T18:24:53.312092] [DEBUG] [VALIDATOR ] DG: All expected trading days are present. [2025-11-04T18:24:53.312206] [DEBUG] [VALIDATOR ] DG: Primary data validation passed. [2025-11-04T18:24:53.313075] [INFO ] [MAIN ] DG: Fetched primary data is valid. [2025-11-04T18:24:53.313148] [DEBUG] [LOCAL ] DG: Saving primary data. [2025-11-04T18:24:53.314701] [DEBUG] [LOCAL ] DG: Trimmed 77228 records to 53149 (keeping 200 days of data) [2025-11-04T18:24:53.642547] [DEBUG] [LOCAL ] DG: Primary data saved (53149 records). [2025-11-04T18:24:53.642708] [INFO ] [MAIN ] DG: Full history saved. [2025-11-04T18:24:53.642795] [INFO ] [MAIN ] Update Progress: 614/2367 symbols completed. DG took 36.9s. Estimated remaining: 4h 2m 13s. Rate: 7.2 symbols/min [2025-11-04T18:24:53.780320] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-10. [2025-11-04T18:24:53.816391] [INFO ] [MAIN ] DHR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:24:53.816537] [DEBUG] [LOCAL ] DHR: No existing data file found in primary storage. [2025-11-04T18:24:53.816585] [WARN ] [MAIN ] DHR: No existing data found, performing full refresh instead. [2025-11-04T18:24:53.816925] [INFO ] [MAIN ] DHR: Wiping existing primary data before full refresh. [2025-11-04T18:24:53.816973] [INFO ] [LOCAL ] DHR: Starting deletion of primary data. [2025-11-04T18:24:53.817017] [DEBUG] [LOCAL ] DHR: No primary data file found to delete. [2025-11-04T18:24:53.817048] [INFO ] [MAIN ] DHR: Starting full primary refresh. [2025-11-04T18:24:54.167788] [INFO ] [REMOTE ] DHR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:24:54.564867] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-11-04T18:24:54.574168] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-06. [2025-11-04T18:24:55.250877] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T18:24:55.336094] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-11-04T18:24:55.367052] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-03. [2025-11-04T18:24:56.167106] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-07. [2025-11-04T18:24:56.930197] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T18:24:56.960780] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-11-04T18:24:56.966986] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-08. [2025-11-04T18:24:57.767040] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-11. [2025-11-04T18:24:57.801027] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-11-04T18:24:58.567121] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-04. [2025-11-04T18:24:58.971689] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T18:24:58.974066] [INFO ] [REMOTE ] DGX: Full fetch complete (65925 records). [2025-11-04T18:24:58.974187] [DEBUG] [VALIDATOR ] DGX: Validating primary data. [2025-11-04T18:24:59.382559] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-09. [2025-11-04T18:24:59.409005] [DEBUG] [VALIDATOR ] DGX: All expected trading days are present. [2025-11-04T18:24:59.409084] [DEBUG] [VALIDATOR ] DGX: Primary data validation passed. [2025-11-04T18:24:59.409999] [INFO ] [MAIN ] DGX: Fetched primary data is valid. [2025-11-04T18:24:59.410064] [DEBUG] [LOCAL ] DGX: Saving primary data. [2025-11-04T18:24:59.411740] [DEBUG] [LOCAL ] DGX: Trimmed 65925 records to 44292 (keeping 200 days of data) [2025-11-04T18:24:59.685278] [DEBUG] [LOCAL ] DGX: Primary data saved (44292 records). [2025-11-04T18:24:59.685407] [INFO ] [MAIN ] DGX: Full history saved. [2025-11-04T18:24:59.685480] [INFO ] [MAIN ] Update Progress: 615/2367 symbols completed. DGX took 40.1s. Estimated remaining: 4h 1m 58s. Rate: 7.2 symbols/min [2025-11-04T18:24:59.856086] [INFO ] [MAIN ] DHT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:24:59.856218] [DEBUG] [LOCAL ] DHT: No existing data file found in primary storage. [2025-11-04T18:24:59.856257] [WARN ] [MAIN ] DHT: No existing data found, performing full refresh instead. [2025-11-04T18:24:59.856362] [INFO ] [MAIN ] DHT: Wiping existing primary data before full refresh. [2025-11-04T18:24:59.856403] [INFO ] [LOCAL ] DHT: Starting deletion of primary data. [2025-11-04T18:24:59.856442] [DEBUG] [LOCAL ] DHT: No primary data file found to delete. [2025-11-04T18:24:59.856472] [INFO ] [MAIN ] DHT: Starting full primary refresh. [2025-11-04T18:25:00.169347] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-01. [2025-11-04T18:25:00.225625] [INFO ] [REMOTE ] DHT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:25:00.238071] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-11-04T18:25:00.273403] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-11-04T18:25:00.967098] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-10. [2025-11-04T18:25:01.724997] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-11-04T18:25:01.767106] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-11. [2025-11-04T18:25:02.149815] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-11-04T18:25:02.567086] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-02. [2025-11-04T18:25:02.627396] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T18:25:02.628995] [INFO ] [REMOTE ] DHF: Full fetch complete (29942 records). [2025-11-04T18:25:02.629131] [DEBUG] [VALIDATOR ] DHF: Validating primary data. [2025-11-04T18:25:03.059354] [DEBUG] [VALIDATOR ] DHF: All expected trading days are present. [2025-11-04T18:25:03.059471] [DEBUG] [VALIDATOR ] DHF: Primary data validation passed. [2025-11-04T18:25:03.060265] [INFO ] [MAIN ] DHF: Fetched primary data is valid. [2025-11-04T18:25:03.060333] [DEBUG] [LOCAL ] DHF: Saving primary data. [2025-11-04T18:25:03.061547] [DEBUG] [LOCAL ] DHF: Trimmed 29942 records to 20330 (keeping 200 days of data) [2025-11-04T18:25:03.189032] [DEBUG] [LOCAL ] DHF: Primary data saved (20330 records). [2025-11-04T18:25:03.189221] [INFO ] [MAIN ] DHF: Full history saved. [2025-11-04T18:25:03.189322] [INFO ] [MAIN ] Update Progress: 616/2367 symbols completed. DHF took 32.4s. Estimated remaining: 4h 1m 37s. Rate: 7.2 symbols/min [2025-11-04T18:25:03.374981] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-01. [2025-11-04T18:25:03.378982] [INFO ] [MAIN ] DHX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:25:03.396603] [DEBUG] [LOCAL ] DHX: Loaded 12882 existing records from primary storage. [2025-11-04T18:25:03.397003] [INFO ] [MAIN ] DHX: Loaded 12882 existing records. [2025-11-04T18:25:03.582636] [WARN ] [MAIN ] DHX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:25:03.583025] [INFO ] [MAIN ] DHX: Wiping existing primary data before full refresh. [2025-11-04T18:25:03.583091] [INFO ] [LOCAL ] DHX: Starting deletion of primary data. [2025-11-04T18:25:03.583296] [DEBUG] [LOCAL ] DHX: Deleted primary data file. [2025-11-04T18:25:03.583339] [INFO ] [MAIN ] DHX: Starting full primary refresh. [2025-11-04T18:25:03.963838] [INFO ] [REMOTE ] DHX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:25:04.135743] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-11-04T18:25:04.167003] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-03. [2025-11-04T18:25:04.961203] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-11-04T18:25:04.971119] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-02. [2025-11-04T18:25:05.767102] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-05. [2025-11-04T18:25:05.900194] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-11-04T18:25:06.134339] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T18:25:06.567136] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-04. [2025-11-04T18:25:07.367051] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-03. [2025-11-04T18:25:07.634193] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-11-04T18:25:08.167058] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-06. [2025-11-04T18:25:08.215745] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-11-04T18:25:08.967084] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-05. [2025-11-04T18:25:09.010104] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-11-04T18:25:09.685209] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-11-04T18:25:09.767041] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-07. [2025-11-04T18:25:10.572894] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-04. [2025-11-04T18:25:10.580816] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-11-04T18:25:11.367011] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-06. [2025-11-04T18:25:11.766010] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-11-04T18:25:12.167024] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-01. [2025-11-04T18:25:12.270359] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-11-04T18:25:12.914985] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T18:25:12.967006] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-05. [2025-11-04T18:25:13.044389] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-11-04T18:25:13.767023] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-08. [2025-11-04T18:25:14.281467] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T18:25:14.567036] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-07. [2025-11-04T18:25:15.367014] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-02. [2025-11-04T18:25:15.460564] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-11-04T18:25:16.167041] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-09. [2025-11-04T18:25:16.186537] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-11-04T18:25:16.455350] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-11-04T18:25:16.967046] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-06. [2025-11-04T18:25:17.767038] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-03. [2025-11-04T18:25:18.084297] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-11-04T18:25:18.361460] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-11-04T18:25:18.567036] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-10. [2025-11-04T18:25:18.602312] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-11-04T18:25:19.367149] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-04. [2025-11-04T18:25:20.051286] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T18:25:20.167007] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-08. [2025-11-04T18:25:20.349132] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-11-04T18:25:20.967054] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-05. [2025-11-04T18:25:21.644916] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T18:25:21.739464] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-11-04T18:25:21.767040] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-06. [2025-11-04T18:25:22.307533] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-11-04T18:25:22.567046] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-09. [2025-11-04T18:25:23.367047] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-07. [2025-11-04T18:25:24.172124] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-11. [2025-11-04T18:25:24.188536] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-11-04T18:25:24.554120] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T18:25:24.556300] [INFO ] [REMOTE ] DHI: Full fetch complete (77398 records). [2025-11-04T18:25:24.556400] [DEBUG] [VALIDATOR ] DHI: Validating primary data. [2025-11-04T18:25:24.844229] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T18:25:24.968121] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-10. [2025-11-04T18:25:25.009163] [DEBUG] [VALIDATOR ] DHI: All expected trading days are present. [2025-11-04T18:25:25.009253] [DEBUG] [VALIDATOR ] DHI: Primary data validation passed. [2025-11-04T18:25:25.010148] [INFO ] [MAIN ] DHI: Fetched primary data is valid. [2025-11-04T18:25:25.010213] [DEBUG] [LOCAL ] DHI: Saving primary data. [2025-11-04T18:25:25.012009] [DEBUG] [LOCAL ] DHI: Trimmed 77398 records to 53339 (keeping 200 days of data) [2025-11-04T18:25:25.343377] [DEBUG] [LOCAL ] DHI: Primary data saved (53339 records). [2025-11-04T18:25:25.343515] [INFO ] [MAIN ] DHI: Full history saved. [2025-11-04T18:25:25.343590] [INFO ] [MAIN ] Update Progress: 617/2367 symbols completed. DHI took 35.7s. Estimated remaining: 4h 2m 8s. Rate: 7.2 symbols/min [2025-11-04T18:25:25.531459] [INFO ] [MAIN ] DIAX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:25:25.545234] [DEBUG] [LOCAL ] DIAX: Loaded 12070 existing records from primary storage. [2025-11-04T18:25:25.545460] [INFO ] [MAIN ] DIAX: Loaded 12070 existing records. [2025-11-04T18:25:25.732534] [WARN ] [MAIN ] DIAX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:25:25.732761] [INFO ] [MAIN ] DIAX: Wiping existing primary data before full refresh. [2025-11-04T18:25:25.732810] [INFO ] [LOCAL ] DIAX: Starting deletion of primary data. [2025-11-04T18:25:25.732980] [DEBUG] [LOCAL ] DIAX: Deleted primary data file. [2025-11-04T18:25:25.733036] [INFO ] [MAIN ] DIAX: Starting full primary refresh. [2025-11-04T18:25:25.775020] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-08. [2025-11-04T18:25:26.087176] [INFO ] [REMOTE ] DIAX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:25:26.567072] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-01. [2025-11-04T18:25:26.847633] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-11-04T18:25:27.079539] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-11-04T18:25:27.254963] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T18:25:27.367039] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-09. [2025-11-04T18:25:28.167034] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-02. [2025-11-04T18:25:28.664751] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-11-04T18:25:28.776581] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-11-04T18:25:28.967034] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-10. [2025-11-04T18:25:29.767030] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-03. [2025-11-04T18:25:30.382201] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-11-04T18:25:30.517414] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-11-04T18:25:30.567000] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-11. [2025-11-04T18:25:31.367075] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-04. [2025-11-04T18:25:31.798738] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T18:25:31.800607] [INFO ] [REMOTE ] DHT: Full fetch complete (70352 records). [2025-11-04T18:25:31.800725] [DEBUG] [VALIDATOR ] DHT: Validating primary data. [2025-11-04T18:25:32.169422] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-07. [2025-11-04T18:25:32.215456] [DEBUG] [VALIDATOR ] DHT: All expected trading days are present. [2025-11-04T18:25:32.215560] [DEBUG] [VALIDATOR ] DHT: Primary data validation passed. [2025-11-04T18:25:32.216429] [INFO ] [MAIN ] DHT: Fetched primary data is valid. [2025-11-04T18:25:32.226906] [DEBUG] [LOCAL ] DHT: Saving primary data. [2025-11-04T18:25:32.227305] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T18:25:32.233047] [DEBUG] [LOCAL ] DHT: Trimmed 70352 records to 47805 (keeping 200 days of data) [2025-11-04T18:25:32.511814] [DEBUG] [LOCAL ] DHT: Primary data saved (47805 records). [2025-11-04T18:25:32.511942] [INFO ] [MAIN ] DHT: Full history saved. [2025-11-04T18:25:32.512010] [INFO ] [MAIN ] Update Progress: 618/2367 symbols completed. DHT took 32.8s. Estimated remaining: 4h 1m 56s. Rate: 7.2 symbols/min [2025-11-04T18:25:32.684089] [INFO ] [MAIN ] DIN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:25:32.718937] [DEBUG] [LOCAL ] DIN: Loaded 36123 existing records from primary storage. [2025-11-04T18:25:32.719218] [INFO ] [MAIN ] DIN: Loaded 36123 existing records. [2025-11-04T18:25:32.872145] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T18:25:32.908041] [WARN ] [MAIN ] DIN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:25:32.908206] [INFO ] [MAIN ] DIN: Wiping existing primary data before full refresh. [2025-11-04T18:25:32.908258] [INFO ] [LOCAL ] DIN: Starting deletion of primary data. [2025-11-04T18:25:32.908428] [DEBUG] [LOCAL ] DIN: Deleted primary data file. [2025-11-04T18:25:32.908500] [INFO ] [MAIN ] DIN: Starting full primary refresh. [2025-11-04T18:25:32.972172] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-08. [2025-11-04T18:25:33.264149] [INFO ] [REMOTE ] DIN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:25:33.525119] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-11-04T18:25:33.767064] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-09. [2025-11-04T18:25:34.277306] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-11-04T18:25:34.567157] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-05. [2025-11-04T18:25:35.096408] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-11-04T18:25:35.367036] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-11. [2025-11-04T18:25:35.745563] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T18:25:35.748466] [INFO ] [REMOTE ] DHR: Full fetch complete (77451 records). [2025-11-04T18:25:35.748636] [DEBUG] [VALIDATOR ] DHR: Validating primary data. [2025-11-04T18:25:36.160841] [DEBUG] [VALIDATOR ] DHR: All expected trading days are present. [2025-11-04T18:25:36.160918] [DEBUG] [VALIDATOR ] DHR: Primary data validation passed. [2025-11-04T18:25:36.161746] [INFO ] [MAIN ] DHR: Fetched primary data is valid. [2025-11-04T18:25:36.161828] [DEBUG] [LOCAL ] DHR: Saving primary data. [2025-11-04T18:25:36.163188] [DEBUG] [LOCAL ] DHR: Trimmed 77451 records to 53367 (keeping 200 days of data) [2025-11-04T18:25:36.217419] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-06. [2025-11-04T18:25:36.488843] [DEBUG] [LOCAL ] DHR: Primary data saved (53367 records). [2025-11-04T18:25:36.488970] [INFO ] [MAIN ] DHR: Full history saved. [2025-11-04T18:25:36.489037] [INFO ] [MAIN ] Update Progress: 619/2367 symbols completed. DHR took 42.8s. Estimated remaining: 4h 1m 36s. Rate: 7.2 symbols/min [2025-11-04T18:25:36.655451] [INFO ] [MAIN ] DINO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:25:36.705124] [DEBUG] [LOCAL ] DINO: Loaded 52748 existing records from primary storage. [2025-11-04T18:25:36.705515] [INFO ] [MAIN ] DINO: Loaded 52748 existing records. [2025-11-04T18:25:36.940319] [WARN ] [MAIN ] DINO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:25:36.941743] [INFO ] [MAIN ] DINO: Wiping existing primary data before full refresh. [2025-11-04T18:25:36.941449] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-11-04T18:25:36.941880] [INFO ] [LOCAL ] DINO: Starting deletion of primary data. [2025-11-04T18:25:36.945371] [DEBUG] [LOCAL ] DINO: Deleted primary data file. [2025-11-04T18:25:36.945440] [INFO ] [MAIN ] DINO: Starting full primary refresh. [2025-11-04T18:25:36.972202] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-01. [2025-11-04T18:25:37.347979] [INFO ] [REMOTE ] DINO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:25:37.767111] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-10. [2025-11-04T18:25:38.398488] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T18:25:38.426883] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-11-04T18:25:38.567050] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-07. [2025-11-04T18:25:39.138079] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-11-04T18:25:39.367098] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-08. [2025-11-04T18:25:39.927344] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-11-04T18:25:40.167008] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-09. [2025-11-04T18:25:40.712832] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-11-04T18:25:40.967032] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-11. [2025-11-04T18:25:41.626844] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T18:25:41.627800] [INFO ] [REMOTE ] DHX: Full fetch complete (21371 records). [2025-11-04T18:25:41.627871] [DEBUG] [VALIDATOR ] DHX: Validating primary data. [2025-11-04T18:25:41.775437] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-02. [2025-11-04T18:25:41.991963] [DEBUG] [VALIDATOR ] DHX: All expected trading days are present. [2025-11-04T18:25:41.992038] [DEBUG] [VALIDATOR ] DHX: Primary data validation passed. [2025-11-04T18:25:41.992424] [INFO ] [MAIN ] DHX: Fetched primary data is valid. [2025-11-04T18:25:41.992463] [DEBUG] [LOCAL ] DHX: Saving primary data. [2025-11-04T18:25:41.993065] [DEBUG] [LOCAL ] DHX: Trimmed 21371 records to 12895 (keeping 200 days of data) [2025-11-04T18:25:42.061879] [DEBUG] [LOCAL ] DHX: Primary data saved (12895 records). [2025-11-04T18:25:42.062010] [INFO ] [MAIN ] DHX: Full history saved. [2025-11-04T18:25:42.062085] [INFO ] [MAIN ] Update Progress: 620/2367 symbols completed. DHX took 38.9s. Estimated remaining: 4h 1m 20s. Rate: 7.2 symbols/min [2025-11-04T18:25:42.243060] [INFO ] [MAIN ] DIS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:25:42.299678] [DEBUG] [LOCAL ] DIS: Loaded 53788 existing records from primary storage. [2025-11-04T18:25:42.300045] [INFO ] [MAIN ] DIS: Loaded 53788 existing records. [2025-11-04T18:25:42.471613] [WARN ] [MAIN ] DIS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:25:42.471839] [INFO ] [MAIN ] DIS: Wiping existing primary data before full refresh. [2025-11-04T18:25:42.471885] [INFO ] [LOCAL ] DIS: Starting deletion of primary data. [2025-11-04T18:25:42.472063] [DEBUG] [LOCAL ] DIS: Deleted primary data file. [2025-11-04T18:25:42.472116] [INFO ] [MAIN ] DIS: Starting full primary refresh. [2025-11-04T18:25:42.570345] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-01. [2025-11-04T18:25:42.827669] [INFO ] [REMOTE ] DIS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:25:42.911076] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T18:25:43.367039] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-01. [2025-11-04T18:25:43.875826] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T18:25:44.167082] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-02. [2025-11-04T18:25:44.967070] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-03. [2025-11-04T18:25:45.353806] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-11-04T18:25:45.556800] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-11-04T18:25:45.767077] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-02. [2025-11-04T18:25:46.048206] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T18:25:46.567054] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-04. [2025-11-04T18:25:47.367066] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-03. [2025-11-04T18:25:47.713362] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T18:25:47.754559] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-11-04T18:25:48.167061] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-05. [2025-11-04T18:25:48.781207] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T18:25:48.967036] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-03. [2025-11-04T18:25:49.383842] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T18:25:49.767089] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-04. [2025-11-04T18:25:50.567027] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-06. [2025-11-04T18:25:50.744282] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-11-04T18:25:51.081439] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T18:25:51.367025] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-05. [2025-11-04T18:25:51.399270] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T18:25:52.167070] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-10. [2025-11-04T18:25:52.648984] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T18:25:52.849614] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-11-04T18:25:52.967018] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-11. [2025-11-04T18:25:53.607309] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T18:25:53.608687] [INFO ] [REMOTE ] DIAX: Full fetch complete (17660 records). [2025-11-04T18:25:53.608796] [DEBUG] [VALIDATOR ] DIAX: Validating primary data. [2025-11-04T18:25:53.781530] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-06. [2025-11-04T18:25:54.220716] [DEBUG] [VALIDATOR ] DIAX: All expected trading days are present. [2025-11-04T18:25:54.220833] [DEBUG] [VALIDATOR ] DIAX: Primary data validation passed. [2025-11-04T18:25:54.221625] [INFO ] [MAIN ] DIAX: Fetched primary data is valid. [2025-11-04T18:25:54.221724] [DEBUG] [LOCAL ] DIAX: Saving primary data. [2025-11-04T18:25:54.222710] [DEBUG] [LOCAL ] DIAX: Trimmed 17660 records to 12068 (keeping 200 days of data) [2025-11-04T18:25:54.307529] [DEBUG] [LOCAL ] DIAX: Primary data saved (12068 records). [2025-11-04T18:25:54.307698] [INFO ] [MAIN ] DIAX: Full history saved. [2025-11-04T18:25:54.307794] [INFO ] [MAIN ] Update Progress: 621/2367 symbols completed. DIAX took 29.0s. Estimated remaining: 4h 1m 23s. Rate: 7.2 symbols/min [2025-11-04T18:25:54.475447] [INFO ] [MAIN ] DIVY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:25:54.475665] [DEBUG] [LOCAL ] DIVY: No existing data file found in primary storage. [2025-11-04T18:25:54.475721] [WARN ] [MAIN ] DIVY: No existing data found, performing full refresh instead. [2025-11-04T18:25:54.475827] [INFO ] [MAIN ] DIVY: Wiping existing primary data before full refresh. [2025-11-04T18:25:54.475867] [INFO ] [LOCAL ] DIVY: Starting deletion of primary data. [2025-11-04T18:25:54.475913] [DEBUG] [LOCAL ] DIVY: No primary data file found to delete. [2025-11-04T18:25:54.475970] [INFO ] [MAIN ] DIVY: Starting full primary refresh. [2025-11-04T18:25:54.572732] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-07. [2025-11-04T18:25:54.825947] [INFO ] [REMOTE ] DIVY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:25:55.065155] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T18:25:55.367048] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-01. [2025-11-04T18:25:55.652271] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T18:25:55.713019] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-01 (0.35 seconds). [2025-11-04T18:25:56.167086] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-07. [2025-11-04T18:25:56.967070] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-08. [2025-11-04T18:25:57.709029] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-11-04T18:25:57.767022] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-04. [2025-11-04T18:25:58.214282] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T18:25:58.567040] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-09. [2025-11-04T18:25:59.367013] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-02. [2025-11-04T18:25:59.536700] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-11-04T18:25:59.610514] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-02 (0.24 seconds). [2025-11-04T18:25:59.722894] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T18:26:00.167094] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-03. [2025-11-04T18:26:00.568110] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T18:26:00.967093] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-04. [2025-11-04T18:26:01.296461] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-11-04T18:26:01.767273] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-05. [2025-11-04T18:26:02.567095] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-08. [2025-11-04T18:26:03.367049] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-05. [2025-11-04T18:26:03.623178] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-05 (0.26 seconds). [2025-11-04T18:26:03.941541] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-11-04T18:26:04.015429] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-05 (2.25 seconds). [2025-11-04T18:26:04.167091] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-09. [2025-11-04T18:26:04.967103] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-06. [2025-11-04T18:26:05.220719] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-06 (0.25 seconds). [2025-11-04T18:26:05.699248] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-11-04T18:26:05.767133] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-10. [2025-11-04T18:26:06.567131] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-07. [2025-11-04T18:26:06.936903] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-11-04T18:26:07.055212] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T18:26:07.367062] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-08. [2025-11-04T18:26:07.629466] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-08 (0.26 seconds). [2025-11-04T18:26:08.167082] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-11. [2025-11-04T18:26:08.967083] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-06. [2025-11-04T18:26:09.235828] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T18:26:09.238409] [INFO ] [REMOTE ] DINO: Full fetch complete (76673 records). [2025-11-04T18:26:09.238571] [DEBUG] [VALIDATOR ] DINO: Validating primary data. [2025-11-04T18:26:09.710194] [DEBUG] [VALIDATOR ] DINO: All expected trading days are present. [2025-11-04T18:26:09.710278] [DEBUG] [VALIDATOR ] DINO: Primary data validation passed. [2025-11-04T18:26:09.711281] [INFO ] [MAIN ] DINO: Fetched primary data is valid. [2025-11-04T18:26:09.711347] [DEBUG] [LOCAL ] DINO: Saving primary data. [2025-11-04T18:26:09.712922] [DEBUG] [LOCAL ] DINO: Trimmed 76673 records to 52742 (keeping 200 days of data) [2025-11-04T18:26:09.772142] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-10. [2025-11-04T18:26:10.066080] [DEBUG] [LOCAL ] DINO: Primary data saved (52742 records). [2025-11-04T18:26:10.066212] [INFO ] [MAIN ] DINO: Full history saved. [2025-11-04T18:26:10.066291] [INFO ] [MAIN ] Update Progress: 622/2367 symbols completed. DINO took 33.6s. Estimated remaining: 4h 1m 35s. Rate: 7.2 symbols/min [2025-11-04T18:26:10.271408] [INFO ] [MAIN ] DK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:26:10.322186] [DEBUG] [LOCAL ] DK: Loaded 49045 existing records from primary storage. [2025-11-04T18:26:10.322574] [INFO ] [MAIN ] DK: Loaded 49045 existing records. [2025-11-04T18:26:10.547969] [WARN ] [MAIN ] DK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:26:10.548204] [INFO ] [MAIN ] DK: Wiping existing primary data before full refresh. [2025-11-04T18:26:10.548252] [INFO ] [LOCAL ] DK: Starting deletion of primary data. [2025-11-04T18:26:10.548481] [DEBUG] [LOCAL ] DK: Deleted primary data file. [2025-11-04T18:26:10.548538] [INFO ] [MAIN ] DK: Starting full primary refresh. [2025-11-04T18:26:10.568001] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-09. [2025-11-04T18:26:10.956714] [INFO ] [REMOTE ] DK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:26:10.959602] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-11-04T18:26:10.993342] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-11-04T18:26:11.078018] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T18:26:11.367045] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-07. [2025-11-04T18:26:12.167100] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-10. [2025-11-04T18:26:12.415713] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-10 (0.25 seconds). [2025-11-04T18:26:12.967041] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-11. [2025-11-04T18:26:13.029084] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-11-04T18:26:13.347148] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T18:26:13.347812] [INFO ] [REMOTE ] DIVY: Full fetch complete (1636 records). [2025-11-04T18:26:13.347878] [DEBUG] [VALIDATOR ] DIVY: Validating primary data. [2025-11-04T18:26:13.348207] [FATAL] [VALIDATOR ] DIVY: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:26:13.348286] [FATAL] [MAIN ] DIVY: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:26:13.348353] [WARN ] [MAIN ] DIVY: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:26:13.348423] [INFO ] [LOCAL ] DIVY: Starting deletion of primary data. [2025-11-04T18:26:13.348484] [DEBUG] [LOCAL ] DIVY: No primary data file found to delete. [2025-11-04T18:26:13.348518] [INFO ] [LOCAL ] DIVY: Starting deletion of derivative data. [2025-11-04T18:26:13.348555] [DEBUG] [LOCAL ] DIVY: No derivative data file found to delete. [2025-11-04T18:26:13.348605] [INFO ] [MAIN ] Update Progress: 623/2367 symbols completed. DIVY took 19.0s. Estimated remaining: 4h 1m 13s. Rate: 7.2 symbols/min [2025-11-04T18:26:13.526361] [INFO ] [MAIN ] DKL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:26:13.543305] [DEBUG] [LOCAL ] DKL: Loaded 15435 existing records from primary storage. [2025-11-04T18:26:13.543551] [INFO ] [MAIN ] DKL: Loaded 15435 existing records. [2025-11-04T18:26:13.717225] [WARN ] [MAIN ] DKL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:26:13.717478] [INFO ] [MAIN ] DKL: Wiping existing primary data before full refresh. [2025-11-04T18:26:13.717525] [INFO ] [LOCAL ] DKL: Starting deletion of primary data. [2025-11-04T18:26:13.717718] [DEBUG] [LOCAL ] DKL: Deleted primary data file. [2025-11-04T18:26:13.717774] [INFO ] [MAIN ] DKL: Starting full primary refresh. [2025-11-04T18:26:13.772211] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-01. [2025-11-04T18:26:14.067339] [INFO ] [REMOTE ] DKL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:26:14.567068] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-01. [2025-11-04T18:26:14.947752] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T18:26:15.372146] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-11. [2025-11-04T18:26:15.378755] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T18:26:16.167039] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-02. [2025-11-04T18:26:16.281415] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T18:26:16.282564] [INFO ] [REMOTE ] DIN: Full fetch complete (53719 records). [2025-11-04T18:26:16.282637] [DEBUG] [VALIDATOR ] DIN: Validating primary data. [2025-11-04T18:26:16.283188] [FATAL] [VALIDATOR ] DIN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:26:16.283267] [FATAL] [MAIN ] DIN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:26:16.283309] [WARN ] [MAIN ] DIN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:26:16.283347] [INFO ] [LOCAL ] DIN: Starting deletion of primary data. [2025-11-04T18:26:16.283401] [DEBUG] [LOCAL ] DIN: No primary data file found to delete. [2025-11-04T18:26:16.283460] [INFO ] [LOCAL ] DIN: Starting deletion of derivative data. [2025-11-04T18:26:16.283734] [DEBUG] [LOCAL ] DIN: Deleted derivative data file. [2025-11-04T18:26:16.283828] [INFO ] [MAIN ] Update Progress: 624/2367 symbols completed. DIN took 43.8s. Estimated remaining: 4h 0m 50s. Rate: 7.2 symbols/min [2025-11-04T18:26:16.453141] [INFO ] [MAIN ] DKS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:26:16.496531] [DEBUG] [LOCAL ] DKS: Loaded 48863 existing records from primary storage. [2025-11-04T18:26:16.496872] [INFO ] [MAIN ] DKS: Loaded 48863 existing records. [2025-11-04T18:26:16.671235] [WARN ] [MAIN ] DKS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:26:16.671436] [INFO ] [MAIN ] DKS: Wiping existing primary data before full refresh. [2025-11-04T18:26:16.671483] [INFO ] [LOCAL ] DKS: Starting deletion of primary data. [2025-11-04T18:26:16.671698] [DEBUG] [LOCAL ] DKS: Deleted primary data file. [2025-11-04T18:26:16.671753] [INFO ] [MAIN ] DKS: Starting full primary refresh. [2025-11-04T18:26:16.972228] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-02. [2025-11-04T18:26:17.040756] [INFO ] [REMOTE ] DKS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:26:17.382316] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T18:26:17.767000] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-03. [2025-11-04T18:26:17.875854] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T18:26:18.567079] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-03. [2025-11-04T18:26:19.217085] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-11-04T18:26:19.308017] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T18:26:19.367059] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-08. [2025-11-04T18:26:20.167043] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-01. [2025-11-04T18:26:20.967071] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-04. [2025-11-04T18:26:21.066538] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-11-04T18:26:21.457565] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T18:26:21.767030] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-04. [2025-11-04T18:26:21.890716] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T18:26:22.567034] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-09. [2025-11-04T18:26:23.135876] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-11-04T18:26:23.367015] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-05. [2025-11-04T18:26:24.167033] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-02. [2025-11-04T18:26:24.254550] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-11-04T18:26:24.543432] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-11-04T18:26:24.967101] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-06. [2025-11-04T18:26:25.480441] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T18:26:25.767047] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-10. [2025-11-04T18:26:26.308106] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T18:26:26.567039] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-05. [2025-11-04T18:26:27.333763] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-11-04T18:26:27.366997] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-06. [2025-11-04T18:26:27.804053] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-11-04T18:26:27.889902] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-11-04T18:26:28.167054] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-07. [2025-11-04T18:26:28.655343] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-11-04T18:26:28.967092] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-07. [2025-11-04T18:26:29.767029] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-03. [2025-11-04T18:26:30.176833] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-11-04T18:26:30.567025] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-08. [2025-11-04T18:26:31.186461] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T18:26:31.206473] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-11-04T18:26:31.367097] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-09. [2025-11-04T18:26:31.864123] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-11-04T18:26:32.167034] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-10. [2025-11-04T18:26:32.784671] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-11-04T18:26:32.967092] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-08. [2025-11-04T18:26:33.767094] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-04. [2025-11-04T18:26:34.220672] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T18:26:34.567055] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-11. [2025-11-04T18:26:34.748024] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-11 (0.18 seconds). [2025-11-04T18:26:34.749566] [INFO ] [REMOTE ] DIS: Full fetch complete (77577 records). [2025-11-04T18:26:34.749695] [DEBUG] [VALIDATOR ] DIS: Validating primary data. [2025-11-04T18:26:34.750453] [FATAL] [VALIDATOR ] DIS: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:26:34.750569] [FATAL] [MAIN ] DIS: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:26:34.750612] [WARN ] [MAIN ] DIS: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:26:34.750678] [INFO ] [LOCAL ] DIS: Starting deletion of primary data. [2025-11-04T18:26:34.750753] [DEBUG] [LOCAL ] DIS: No primary data file found to delete. [2025-11-04T18:26:34.750789] [INFO ] [LOCAL ] DIS: Starting deletion of derivative data. [2025-11-04T18:26:34.751045] [DEBUG] [LOCAL ] DIS: Deleted derivative data file. [2025-11-04T18:26:34.751135] [INFO ] [MAIN ] Update Progress: 625/2367 symbols completed. DIS took 52.7s. Estimated remaining: 4h 1m 10s. Rate: 7.2 symbols/min [2025-11-04T18:26:34.929395] [INFO ] [MAIN ] DLB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:26:34.960435] [DEBUG] [LOCAL ] DLB: Loaded 32348 existing records from primary storage. [2025-11-04T18:26:34.960743] [INFO ] [MAIN ] DLB: Loaded 32348 existing records. [2025-11-04T18:26:35.147801] [WARN ] [MAIN ] DLB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:26:35.148064] [INFO ] [MAIN ] DLB: Wiping existing primary data before full refresh. [2025-11-04T18:26:35.148120] [INFO ] [LOCAL ] DLB: Starting deletion of primary data. [2025-11-04T18:26:35.148444] [DEBUG] [LOCAL ] DLB: Deleted primary data file. [2025-11-04T18:26:35.148541] [INFO ] [MAIN ] DLB: Starting full primary refresh. [2025-11-04T18:26:35.372241] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-09. [2025-11-04T18:26:35.397231] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-11-04T18:26:35.552122] [INFO ] [REMOTE ] DLB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:26:36.167067] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-11. [2025-11-04T18:26:36.710412] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T18:26:36.847587] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T18:26:36.848638] [INFO ] [REMOTE ] DKL: Full fetch complete (27299 records). [2025-11-04T18:26:36.848735] [DEBUG] [VALIDATOR ] DKL: Validating primary data. [2025-11-04T18:26:36.972147] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-01. [2025-11-04T18:26:37.228991] [DEBUG] [VALIDATOR ] DKL: All expected trading days are present. [2025-11-04T18:26:37.229091] [DEBUG] [VALIDATOR ] DKL: Primary data validation passed. [2025-11-04T18:26:37.229676] [INFO ] [MAIN ] DKL: Fetched primary data is valid. [2025-11-04T18:26:37.229749] [DEBUG] [LOCAL ] DKL: Saving primary data. [2025-11-04T18:26:37.230735] [DEBUG] [LOCAL ] DKL: Trimmed 27299 records to 15337 (keeping 200 days of data) [2025-11-04T18:26:37.324510] [DEBUG] [LOCAL ] DKL: Primary data saved (15337 records). [2025-11-04T18:26:37.324668] [INFO ] [MAIN ] DKL: Full history saved. [2025-11-04T18:26:37.324956] [INFO ] [MAIN ] Update Progress: 626/2367 symbols completed. DKL took 24.0s. Estimated remaining: 4h 0m 45s. Rate: 7.2 symbols/min [2025-11-04T18:26:37.494464] [INFO ] [MAIN ] DLNG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:26:37.502309] [DEBUG] [LOCAL ] DLNG: Loaded 5800 existing records from primary storage. [2025-11-04T18:26:37.502517] [INFO ] [MAIN ] DLNG: Loaded 5800 existing records. [2025-11-04T18:26:37.677635] [WARN ] [MAIN ] DLNG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:26:37.677893] [INFO ] [MAIN ] DLNG: Wiping existing primary data before full refresh. [2025-11-04T18:26:37.677941] [INFO ] [LOCAL ] DLNG: Starting deletion of primary data. [2025-11-04T18:26:37.678106] [DEBUG] [LOCAL ] DLNG: Deleted primary data file. [2025-11-04T18:26:37.678164] [INFO ] [MAIN ] DLNG: Starting full primary refresh. [2025-11-04T18:26:37.777051] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-10. [2025-11-04T18:26:38.041209] [INFO ] [REMOTE ] DLNG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:26:38.131856] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T18:26:38.567044] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-01. [2025-11-04T18:26:39.060036] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-11-04T18:26:39.320592] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T18:26:39.367027] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-02. [2025-11-04T18:26:39.804745] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-11-04T18:26:40.167030] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-05. [2025-11-04T18:26:40.967053] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-03. [2025-11-04T18:26:41.385235] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T18:26:41.767067] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-11. [2025-11-04T18:26:41.951396] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-11-04T18:26:42.567041] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-06. [2025-11-04T18:26:42.781074] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-11 (1.01 seconds). [2025-11-04T18:26:42.782972] [INFO ] [REMOTE ] DK: Full fetch complete (70807 records). [2025-11-04T18:26:42.783101] [DEBUG] [VALIDATOR ] DK: Validating primary data. [2025-11-04T18:26:43.163513] [DEBUG] [VALIDATOR ] DK: All expected trading days are present. [2025-11-04T18:26:43.163589] [DEBUG] [VALIDATOR ] DK: Primary data validation passed. [2025-11-04T18:26:43.164363] [INFO ] [MAIN ] DK: Fetched primary data is valid. [2025-11-04T18:26:43.164426] [DEBUG] [LOCAL ] DK: Saving primary data. [2025-11-04T18:26:43.165749] [DEBUG] [LOCAL ] DK: Trimmed 70807 records to 49057 (keeping 200 days of data) [2025-11-04T18:26:43.370490] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-04. [2025-11-04T18:26:43.455802] [DEBUG] [LOCAL ] DK: Primary data saved (49057 records). [2025-11-04T18:26:43.455926] [INFO ] [MAIN ] DK: Full history saved. [2025-11-04T18:26:43.456001] [INFO ] [MAIN ] Update Progress: 627/2367 symbols completed. DK took 33.4s. Estimated remaining: 4h 0m 31s. Rate: 7.2 symbols/min [2025-11-04T18:26:43.630734] [INFO ] [MAIN ] DLR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:26:43.630992] [DEBUG] [LOCAL ] DLR: No existing data file found in primary storage. [2025-11-04T18:26:43.631050] [WARN ] [MAIN ] DLR: No existing data found, performing full refresh instead. [2025-11-04T18:26:43.631163] [INFO ] [MAIN ] DLR: Wiping existing primary data before full refresh. [2025-11-04T18:26:43.631207] [INFO ] [LOCAL ] DLR: Starting deletion of primary data. [2025-11-04T18:26:43.631247] [DEBUG] [LOCAL ] DLR: No primary data file found to delete. [2025-11-04T18:26:43.631279] [INFO ] [MAIN ] DLR: Starting full primary refresh. [2025-11-04T18:26:43.996739] [INFO ] [REMOTE ] DLR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:26:44.032741] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-11-04T18:26:44.034421] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T18:26:44.167012] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-07. [2025-11-04T18:26:44.226942] [DEBUG] [REMOTE ] DKS: Error fetching primary data for 2025-07: 'API call returned an empty response.' [2025-11-04T18:26:44.227167] [WARN ] [REMOTE ] DKS: No data returned for month 2025-07 [2025-11-04T18:26:44.227257] [FATAL] [REMOTE ] DKS: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:26:44.227355] [FATAL] [MAIN ] DKS: Failed to fetch primary data. [2025-11-04T18:26:44.227422] [WARN ] [MAIN ] DKS: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:26:44.227490] [INFO ] [LOCAL ] DKS: Starting deletion of primary data. [2025-11-04T18:26:44.227551] [DEBUG] [LOCAL ] DKS: No primary data file found to delete. [2025-11-04T18:26:44.227587] [INFO ] [LOCAL ] DKS: Starting deletion of derivative data. [2025-11-04T18:26:44.227753] [DEBUG] [LOCAL ] DKS: Deleted derivative data file. [2025-11-04T18:26:44.227855] [INFO ] [MAIN ] Update Progress: 628/2367 symbols completed. DKS took 27.9s. Estimated remaining: 4h 0m 2s. Rate: 7.2 symbols/min [2025-11-04T18:26:44.397718] [INFO ] [MAIN ] DLX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:26:44.424550] [DEBUG] [LOCAL ] DLX: Loaded 27628 existing records from primary storage. [2025-11-04T18:26:44.424821] [INFO ] [MAIN ] DLX: Loaded 27628 existing records. [2025-11-04T18:26:44.598506] [WARN ] [MAIN ] DLX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:26:44.598756] [INFO ] [MAIN ] DLX: Wiping existing primary data before full refresh. [2025-11-04T18:26:44.598809] [INFO ] [LOCAL ] DLX: Starting deletion of primary data. [2025-11-04T18:26:44.599008] [DEBUG] [LOCAL ] DLX: Deleted primary data file. [2025-11-04T18:26:44.599065] [INFO ] [MAIN ] DLX: Starting full primary refresh. [2025-11-04T18:26:44.949545] [INFO ] [REMOTE ] DLX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:26:44.967004] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-01. [2025-11-04T18:26:45.767100] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-05. [2025-11-04T18:26:46.016705] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-11-04T18:26:46.262148] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-11-04T18:26:46.567115] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-02. [2025-11-04T18:26:47.367020] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-01. [2025-11-04T18:26:47.536339] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T18:26:48.167115] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-02. [2025-11-04T18:26:48.747218] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T18:26:48.967055] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-02. [2025-11-04T18:26:49.617519] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-11-04T18:26:49.767066] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-03. [2025-11-04T18:26:50.064376] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T18:26:50.567016] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-06. [2025-11-04T18:26:50.677183] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T18:26:50.853133] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-06 (0.29 seconds). [2025-11-04T18:26:51.367049] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-03. [2025-11-04T18:26:52.167085] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-03. [2025-11-04T18:26:52.710806] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T18:26:52.967055] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-07. [2025-11-04T18:26:53.045056] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-11-04T18:26:53.292748] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-11-04T18:26:53.767045] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-04. [2025-11-04T18:26:54.567046] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-08. [2025-11-04T18:26:54.800422] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-11-04T18:26:54.944181] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-11-04T18:26:55.367048] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-04. [2025-11-04T18:26:56.167048] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-05. [2025-11-04T18:26:56.948666] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-11-04T18:26:56.967008] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-09. [2025-11-04T18:26:57.152566] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T18:26:57.364974] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-11-04T18:26:57.767081] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-10. [2025-11-04T18:26:58.090033] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-10 (0.32 seconds). [2025-11-04T18:26:58.567044] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-05. [2025-11-04T18:26:59.367053] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-06. [2025-11-04T18:27:00.168118] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-11. [2025-11-04T18:27:00.180221] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-11-04T18:27:00.192605] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T18:27:00.708689] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T18:27:00.709497] [INFO ] [REMOTE ] DLNG: Full fetch complete (10573 records). [2025-11-04T18:27:00.709564] [DEBUG] [VALIDATOR ] DLNG: Validating primary data. [2025-11-04T18:27:00.972222] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-07. [2025-11-04T18:27:01.085953] [DEBUG] [VALIDATOR ] DLNG: All expected trading days are present. [2025-11-04T18:27:01.086036] [DEBUG] [VALIDATOR ] DLNG: Primary data validation passed. [2025-11-04T18:27:01.086408] [INFO ] [MAIN ] DLNG: Fetched primary data is valid. [2025-11-04T18:27:01.086446] [DEBUG] [LOCAL ] DLNG: Saving primary data. [2025-11-04T18:27:01.087081] [DEBUG] [LOCAL ] DLNG: Trimmed 10573 records to 5785 (keeping 200 days of data) [2025-11-04T18:27:01.121895] [DEBUG] [LOCAL ] DLNG: Primary data saved (5785 records). [2025-11-04T18:27:01.122041] [INFO ] [MAIN ] DLNG: Full history saved. [2025-11-04T18:27:01.122128] [INFO ] [MAIN ] Update Progress: 629/2367 symbols completed. DLNG took 23.8s. Estimated remaining: 4h 0m 17s. Rate: 7.2 symbols/min [2025-11-04T18:27:01.310585] [INFO ] [MAIN ] DLY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:27:01.310794] [DEBUG] [LOCAL ] DLY: No existing data file found in primary storage. [2025-11-04T18:27:01.310836] [WARN ] [MAIN ] DLY: No existing data found, performing full refresh instead. [2025-11-04T18:27:01.310950] [INFO ] [MAIN ] DLY: Wiping existing primary data before full refresh. [2025-11-04T18:27:01.310990] [INFO ] [LOCAL ] DLY: Starting deletion of primary data. [2025-11-04T18:27:01.311030] [DEBUG] [LOCAL ] DLY: No primary data file found to delete. [2025-11-04T18:27:01.311061] [INFO ] [MAIN ] DLY: Starting full primary refresh. [2025-11-04T18:27:01.671792] [INFO ] [REMOTE ] DLY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:27:01.767054] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-01. [2025-11-04T18:27:01.987934] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T18:27:02.567104] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-06. [2025-11-04T18:27:02.750347] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T18:27:03.367061] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-02. [2025-11-04T18:27:03.972058] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-11-04T18:27:04.003080] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T18:27:04.167105] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-04. [2025-11-04T18:27:04.967054] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-07. [2025-11-04T18:27:05.421264] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T18:27:05.767071] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-03. [2025-11-04T18:27:06.516869] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-11-04T18:27:06.547511] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-11-04T18:27:06.567039] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-04. [2025-11-04T18:27:07.281320] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-11-04T18:27:07.367080] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-05. [2025-11-04T18:27:08.090054] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-11-04T18:27:08.167018] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-06. [2025-11-04T18:27:08.852609] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-11-04T18:27:08.967114] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-07. [2025-11-04T18:27:09.706704] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T18:27:09.767054] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-08. [2025-11-04T18:27:10.424120] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-11-04T18:27:10.567050] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-09. [2025-11-04T18:27:11.221090] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-11-04T18:27:11.367066] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-08. [2025-11-04T18:27:12.167109] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-10. [2025-11-04T18:27:12.820456] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T18:27:12.967032] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-09. [2025-11-04T18:27:13.307358] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T18:27:13.767058] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-05. [2025-11-04T18:27:14.330572] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T18:27:14.567043] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-08. [2025-11-04T18:27:14.729139] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T18:27:15.367017] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-11. [2025-11-04T18:27:15.534632] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-11-04T18:27:16.080851] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T18:27:16.082246] [INFO ] [REMOTE ] DLY: Full fetch complete (29384 records). [2025-11-04T18:27:16.082318] [DEBUG] [VALIDATOR ] DLY: Validating primary data. [2025-11-04T18:27:16.172144] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-10. [2025-11-04T18:27:16.444893] [DEBUG] [VALIDATOR ] DLY: All expected trading days are present. [2025-11-04T18:27:16.445000] [DEBUG] [VALIDATOR ] DLY: Primary data validation passed. [2025-11-04T18:27:16.445501] [INFO ] [MAIN ] DLY: Fetched primary data is valid. [2025-11-04T18:27:16.445566] [DEBUG] [LOCAL ] DLY: Saving primary data. [2025-11-04T18:27:16.446291] [DEBUG] [LOCAL ] DLY: Trimmed 29384 records to 20902 (keeping 200 days of data) [2025-11-04T18:27:16.569316] [DEBUG] [LOCAL ] DLY: Primary data saved (20902 records). [2025-11-04T18:27:16.569438] [INFO ] [MAIN ] DLY: Full history saved. [2025-11-04T18:27:16.569504] [INFO ] [MAIN ] Update Progress: 630/2367 symbols completed. DLY took 15.4s. Estimated remaining: 4h 0m 29s. Rate: 7.2 symbols/min [2025-11-04T18:27:16.732853] [INFO ] [MAIN ] DMA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:27:16.739891] [DEBUG] [LOCAL ] DMA: Loaded 4675 existing records from primary storage. [2025-11-04T18:27:16.740080] [INFO ] [MAIN ] DMA: Loaded 4675 existing records. [2025-11-04T18:27:16.910928] [WARN ] [MAIN ] DMA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:27:16.911125] [INFO ] [MAIN ] DMA: Wiping existing primary data before full refresh. [2025-11-04T18:27:16.911171] [INFO ] [LOCAL ] DMA: Starting deletion of primary data. [2025-11-04T18:27:16.911292] [DEBUG] [LOCAL ] DMA: Deleted primary data file. [2025-11-04T18:27:16.911327] [INFO ] [MAIN ] DMA: Starting full primary refresh. [2025-11-04T18:27:16.972284] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-06. [2025-11-04T18:27:17.300633] [INFO ] [REMOTE ] DMA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:27:17.563920] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T18:27:17.767011] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-09. [2025-11-04T18:27:17.960714] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-11-04T18:27:18.540330] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-11-04T18:27:18.567025] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-11. [2025-11-04T18:27:19.367077] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-01. [2025-11-04T18:27:19.759935] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T18:27:19.761837] [INFO ] [REMOTE ] DLR: Full fetch complete (74169 records). [2025-11-04T18:27:19.761932] [DEBUG] [VALIDATOR ] DLR: Validating primary data. [2025-11-04T18:27:19.866567] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-11-04T18:27:20.151696] [DEBUG] [VALIDATOR ] DLR: All expected trading days are present. [2025-11-04T18:27:20.151791] [DEBUG] [VALIDATOR ] DLR: Primary data validation passed. [2025-11-04T18:27:20.152592] [INFO ] [MAIN ] DLR: Fetched primary data is valid. [2025-11-04T18:27:20.152634] [DEBUG] [LOCAL ] DLR: Saving primary data. [2025-11-04T18:27:20.153978] [DEBUG] [LOCAL ] DLR: Trimmed 74169 records to 50480 (keeping 200 days of data) [2025-11-04T18:27:20.207759] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-02. [2025-11-04T18:27:20.460559] [DEBUG] [LOCAL ] DLR: Primary data saved (50480 records). [2025-11-04T18:27:20.460715] [INFO ] [MAIN ] DLR: Full history saved. [2025-11-04T18:27:20.460794] [INFO ] [MAIN ] Update Progress: 631/2367 symbols completed. DLR took 37.0s. Estimated remaining: 4h 0m 8s. Rate: 7.2 symbols/min [2025-11-04T18:27:20.626277] [INFO ] [MAIN ] DMB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:27:20.630487] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-11-04T18:27:20.641609] [DEBUG] [LOCAL ] DMB: Loaded 10357 existing records from primary storage. [2025-11-04T18:27:20.641862] [INFO ] [MAIN ] DMB: Loaded 10357 existing records. [2025-11-04T18:27:20.820788] [WARN ] [MAIN ] DMB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:27:20.821015] [INFO ] [MAIN ] DMB: Wiping existing primary data before full refresh. [2025-11-04T18:27:20.821066] [INFO ] [LOCAL ] DMB: Starting deletion of primary data. [2025-11-04T18:27:20.821239] [DEBUG] [LOCAL ] DMB: Deleted primary data file. [2025-11-04T18:27:20.821294] [INFO ] [MAIN ] DMB: Starting full primary refresh. [2025-11-04T18:27:20.968381] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-03. [2025-11-04T18:27:21.172959] [INFO ] [REMOTE ] DMB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:27:21.443517] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-11-04T18:27:21.767081] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-01. [2025-11-04T18:27:22.430992] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-11-04T18:27:22.567074] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-07. [2025-11-04T18:27:23.367015] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-10. [2025-11-04T18:27:23.429014] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-11-04T18:27:24.167012] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-08. [2025-11-04T18:27:24.230450] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-11-04T18:27:24.967009] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-02. [2025-11-04T18:27:25.072084] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-11-04T18:27:25.465762] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-11-04T18:27:25.767050] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-03. [2025-11-04T18:27:26.129061] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-11-04T18:27:26.567012] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-04. [2025-11-04T18:27:27.088292] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-11-04T18:27:27.367037] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-05. [2025-11-04T18:27:27.849224] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-11-04T18:27:28.167054] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-04. [2025-11-04T18:27:28.510815] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-11-04T18:27:28.967054] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-11. [2025-11-04T18:27:29.766247] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-11 (0.80 seconds). [2025-11-04T18:27:29.768396] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-06. [2025-11-04T18:27:29.769639] [INFO ] [REMOTE ] DLX: Full fetch complete (41650 records). [2025-11-04T18:27:29.769764] [DEBUG] [VALIDATOR ] DLX: Validating primary data. [2025-11-04T18:27:30.148898] [DEBUG] [VALIDATOR ] DLX: All expected trading days are present. [2025-11-04T18:27:30.148979] [DEBUG] [VALIDATOR ] DLX: Primary data validation passed. [2025-11-04T18:27:30.149537] [INFO ] [MAIN ] DLX: Fetched primary data is valid. [2025-11-04T18:27:30.149580] [DEBUG] [LOCAL ] DLX: Saving primary data. [2025-11-04T18:27:30.150570] [DEBUG] [LOCAL ] DLX: Trimmed 41650 records to 27577 (keeping 200 days of data) [2025-11-04T18:27:30.318256] [DEBUG] [LOCAL ] DLX: Primary data saved (27577 records). [2025-11-04T18:27:30.318385] [INFO ] [MAIN ] DLX: Full history saved. [2025-11-04T18:27:30.318462] [INFO ] [MAIN ] Update Progress: 632/2367 symbols completed. DLX took 46.1s. Estimated remaining: 4h 0m 4s. Rate: 7.2 symbols/min [2025-11-04T18:27:30.449831] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-11-04T18:27:30.518295] [INFO ] [MAIN ] DMO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:27:30.529349] [DEBUG] [LOCAL ] DMO: Loaded 9225 existing records from primary storage. [2025-11-04T18:27:30.529539] [INFO ] [MAIN ] DMO: Loaded 9225 existing records. [2025-11-04T18:27:30.572142] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-07. [2025-11-04T18:27:30.706695] [WARN ] [MAIN ] DMO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:27:30.706930] [INFO ] [MAIN ] DMO: Wiping existing primary data before full refresh. [2025-11-04T18:27:30.706978] [INFO ] [LOCAL ] DMO: Starting deletion of primary data. [2025-11-04T18:27:30.707190] [DEBUG] [LOCAL ] DMO: Deleted primary data file. [2025-11-04T18:27:30.707245] [INFO ] [MAIN ] DMO: Starting full primary refresh. [2025-11-04T18:27:31.058942] [INFO ] [REMOTE ] DMO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:27:31.200206] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-11-04T18:27:31.367019] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-05. [2025-11-04T18:27:31.876547] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-11-04T18:27:32.167025] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-01. [2025-11-04T18:27:32.830259] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-11-04T18:27:32.967102] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-06. [2025-11-04T18:27:33.281469] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-11-04T18:27:33.767039] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-07. [2025-11-04T18:27:34.131268] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-11-04T18:27:34.567089] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-09. [2025-11-04T18:27:35.367054] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-08. [2025-11-04T18:27:35.664567] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T18:27:35.807748] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T18:27:36.167082] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-10. [2025-11-04T18:27:36.967079] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-02. [2025-11-04T18:27:37.217518] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-11-04T18:27:37.444043] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T18:27:37.767050] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-03. [2025-11-04T18:27:38.370255] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-11-04T18:27:38.567065] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-04. [2025-11-04T18:27:39.240734] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-11-04T18:27:39.367079] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-05. [2025-11-04T18:27:39.847927] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-11-04T18:27:40.167056] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-06. [2025-11-04T18:27:40.665817] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-11-04T18:27:40.967049] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-07. [2025-11-04T18:27:41.501451] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-11-04T18:27:41.767062] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-11. [2025-11-04T18:27:42.567004] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-09. [2025-11-04T18:27:42.652679] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T18:27:42.654213] [INFO ] [REMOTE ] DLB: Full fetch complete (47454 records). [2025-11-04T18:27:42.654284] [DEBUG] [VALIDATOR ] DLB: Validating primary data. [2025-11-04T18:27:43.030751] [DEBUG] [VALIDATOR ] DLB: All expected trading days are present. [2025-11-04T18:27:43.030820] [DEBUG] [VALIDATOR ] DLB: Primary data validation passed. [2025-11-04T18:27:43.031466] [INFO ] [MAIN ] DLB: Fetched primary data is valid. [2025-11-04T18:27:43.031505] [DEBUG] [LOCAL ] DLB: Saving primary data. [2025-11-04T18:27:43.032562] [DEBUG] [LOCAL ] DLB: Trimmed 47454 records to 32391 (keeping 200 days of data) [2025-11-04T18:27:43.127879] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T18:27:43.227283] [DEBUG] [LOCAL ] DLB: Primary data saved (32391 records). [2025-11-04T18:27:43.227410] [INFO ] [MAIN ] DLB: Full history saved. [2025-11-04T18:27:43.227485] [INFO ] [MAIN ] Update Progress: 633/2367 symbols completed. DLB took 1m 8s. Estimated remaining: 4h 0m 9s. Rate: 7.2 symbols/min [2025-11-04T18:27:43.376055] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-08. [2025-11-04T18:27:43.394473] [INFO ] [MAIN ] DNA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:27:43.437133] [DEBUG] [LOCAL ] DNA: Loaded 48104 existing records from primary storage. [2025-11-04T18:27:43.437412] [INFO ] [MAIN ] DNA: Loaded 48104 existing records. [2025-11-04T18:27:43.609664] [WARN ] [MAIN ] DNA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:27:43.609891] [INFO ] [MAIN ] DNA: Wiping existing primary data before full refresh. [2025-11-04T18:27:43.609941] [INFO ] [LOCAL ] DNA: Starting deletion of primary data. [2025-11-04T18:27:43.610128] [DEBUG] [LOCAL ] DNA: Deleted primary data file. [2025-11-04T18:27:43.610183] [INFO ] [MAIN ] DNA: Starting full primary refresh. [2025-11-04T18:27:43.957535] [INFO ] [REMOTE ] DNA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:27:44.000260] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-11-04T18:27:44.167030] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-01. [2025-11-04T18:27:44.967045] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-10. [2025-11-04T18:27:45.024328] [DEBUG] [REMOTE ] DMA: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-11-04T18:27:45.024527] [WARN ] [REMOTE ] DMA: No data returned for month 2025-10 [2025-11-04T18:27:45.024612] [FATAL] [REMOTE ] DMA: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:27:45.024747] [FATAL] [MAIN ] DMA: Failed to fetch primary data. [2025-11-04T18:27:45.024821] [WARN ] [MAIN ] DMA: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:27:45.024891] [INFO ] [LOCAL ] DMA: Starting deletion of primary data. [2025-11-04T18:27:45.024952] [DEBUG] [LOCAL ] DMA: No primary data file found to delete. [2025-11-04T18:27:45.024990] [INFO ] [LOCAL ] DMA: Starting deletion of derivative data. [2025-11-04T18:27:45.025147] [DEBUG] [LOCAL ] DMA: Deleted derivative data file. [2025-11-04T18:27:45.025226] [INFO ] [MAIN ] Update Progress: 634/2367 symbols completed. DMA took 28.5s. Estimated remaining: 3h 59m 43s. Rate: 7.2 symbols/min [2025-11-04T18:27:45.189496] [INFO ] [MAIN ] DNOW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:27:45.228535] [DEBUG] [LOCAL ] DNOW: Loaded 43552 existing records from primary storage. [2025-11-04T18:27:45.228894] [INFO ] [MAIN ] DNOW: Loaded 43552 existing records. [2025-11-04T18:27:45.405622] [WARN ] [MAIN ] DNOW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:27:45.405887] [INFO ] [MAIN ] DNOW: Wiping existing primary data before full refresh. [2025-11-04T18:27:45.405994] [INFO ] [LOCAL ] DNOW: Starting deletion of primary data. [2025-11-04T18:27:45.406206] [DEBUG] [LOCAL ] DNOW: Deleted primary data file. [2025-11-04T18:27:45.406261] [INFO ] [MAIN ] DNOW: Starting full primary refresh. [2025-11-04T18:27:45.768282] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-08. [2025-11-04T18:27:45.768213] [INFO ] [REMOTE ] DNOW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:27:45.788630] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-11-04T18:27:46.313111] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-11-04T18:27:46.567045] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-09. [2025-11-04T18:27:47.124497] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T18:27:47.367055] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-09. [2025-11-04T18:27:47.885003] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-11-04T18:27:48.167044] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-02. [2025-11-04T18:27:48.967121] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-01. [2025-11-04T18:27:49.715816] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-11-04T18:27:49.767002] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-10. [2025-11-04T18:27:50.079369] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-11-04T18:27:50.145533] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-11-04T18:27:50.567075] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-03. [2025-11-04T18:27:51.367046] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-10. [2025-11-04T18:27:51.858728] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-11-04T18:27:51.859024] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-11-04T18:27:52.167062] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-11. [2025-11-04T18:27:52.665881] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T18:27:52.667056] [INFO ] [REMOTE ] DMB: Full fetch complete (14456 records). [2025-11-04T18:27:52.667142] [DEBUG] [VALIDATOR ] DMB: Validating primary data. [2025-11-04T18:27:52.972183] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-11. [2025-11-04T18:27:53.034109] [DEBUG] [VALIDATOR ] DMB: All expected trading days are present. [2025-11-04T18:27:53.034190] [DEBUG] [VALIDATOR ] DMB: Primary data validation passed. [2025-11-04T18:27:53.034533] [INFO ] [MAIN ] DMB: Fetched primary data is valid. [2025-11-04T18:27:53.034572] [DEBUG] [LOCAL ] DMB: Saving primary data. [2025-11-04T18:27:53.035169] [DEBUG] [LOCAL ] DMB: Trimmed 14456 records to 10378 (keeping 200 days of data) [2025-11-04T18:27:53.100734] [DEBUG] [LOCAL ] DMB: Primary data saved (10378 records). [2025-11-04T18:27:53.100902] [INFO ] [MAIN ] DMB: Full history saved. [2025-11-04T18:27:53.101019] [INFO ] [MAIN ] Update Progress: 635/2367 symbols completed. DMB took 32.6s. Estimated remaining: 3h 59m 34s. Rate: 7.2 symbols/min [2025-11-04T18:27:53.270945] [INFO ] [MAIN ] DNP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:27:53.301157] [DEBUG] [LOCAL ] DNP: Loaded 32716 existing records from primary storage. [2025-11-04T18:27:53.301429] [INFO ] [MAIN ] DNP: Loaded 32716 existing records. [2025-11-04T18:27:53.482439] [WARN ] [MAIN ] DNP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:27:53.482686] [INFO ] [MAIN ] DNP: Wiping existing primary data before full refresh. [2025-11-04T18:27:53.482742] [INFO ] [LOCAL ] DNP: Starting deletion of primary data. [2025-11-04T18:27:53.482941] [DEBUG] [LOCAL ] DNP: Deleted primary data file. [2025-11-04T18:27:53.482995] [INFO ] [MAIN ] DNP: Starting full primary refresh. [2025-11-04T18:27:53.659192] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-11 (0.69 seconds). [2025-11-04T18:27:53.722250] [INFO ] [REMOTE ] DMO: Full fetch complete (14134 records). [2025-11-04T18:27:53.767304] [DEBUG] [VALIDATOR ] DMO: Validating primary data. [2025-11-04T18:27:53.767083] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-04. [2025-11-04T18:27:54.028785] [INFO ] [REMOTE ] DNP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:27:54.274259] [DEBUG] [VALIDATOR ] DMO: All expected trading days are present. [2025-11-04T18:27:54.274341] [DEBUG] [VALIDATOR ] DMO: Primary data validation passed. [2025-11-04T18:27:54.274714] [INFO ] [MAIN ] DMO: Fetched primary data is valid. [2025-11-04T18:27:54.274775] [DEBUG] [LOCAL ] DMO: Saving primary data. [2025-11-04T18:27:54.275281] [DEBUG] [LOCAL ] DMO: Trimmed 14134 records to 9199 (keeping 200 days of data) [2025-11-04T18:27:54.330567] [DEBUG] [LOCAL ] DMO: Primary data saved (9199 records). [2025-11-04T18:27:54.330736] [INFO ] [MAIN ] DMO: Full history saved. [2025-11-04T18:27:54.330829] [INFO ] [MAIN ] Update Progress: 636/2367 symbols completed. DMO took 24.0s. Estimated remaining: 3h 59m 6s. Rate: 7.2 symbols/min [2025-11-04T18:27:54.514406] [INFO ] [MAIN ] DOC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:27:54.563945] [DEBUG] [LOCAL ] DOC: Loaded 53554 existing records from primary storage. [2025-11-04T18:27:54.564259] [INFO ] [MAIN ] DOC: Loaded 53554 existing records. [2025-11-04T18:27:54.575396] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-02. [2025-11-04T18:27:54.738670] [WARN ] [MAIN ] DOC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:27:54.738904] [INFO ] [MAIN ] DOC: Wiping existing primary data before full refresh. [2025-11-04T18:27:54.738952] [INFO ] [LOCAL ] DOC: Starting deletion of primary data. [2025-11-04T18:27:54.739133] [DEBUG] [LOCAL ] DOC: Deleted primary data file. [2025-11-04T18:27:54.739169] [INFO ] [MAIN ] DOC: Starting full primary refresh. [2025-11-04T18:27:55.085780] [INFO ] [REMOTE ] DOC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:27:55.152889] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T18:27:55.367057] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-05. [2025-11-04T18:27:55.640217] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T18:27:56.167046] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-03. [2025-11-04T18:27:56.626964] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T18:27:56.967033] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-06. [2025-11-04T18:27:57.416485] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-11-04T18:27:57.767037] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-04. [2025-11-04T18:27:58.266089] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T18:27:58.567049] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-01. [2025-11-04T18:27:58.903163] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T18:27:59.367097] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-01. [2025-11-04T18:27:59.838421] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T18:28:00.167099] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-05. [2025-11-04T18:28:00.967050] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-02. [2025-11-04T18:28:01.072860] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-11-04T18:28:01.400732] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T18:28:01.767068] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-06. [2025-11-04T18:28:01.820005] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T18:28:02.567047] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-07. [2025-11-04T18:28:02.770349] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T18:28:03.367095] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-03. [2025-11-04T18:28:04.080003] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T18:28:04.167017] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-07. [2025-11-04T18:28:04.471610] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T18:28:04.967042] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-08. [2025-11-04T18:28:05.198353] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T18:28:05.767037] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-02. [2025-11-04T18:28:06.386520] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T18:28:06.567082] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-08. [2025-11-04T18:28:07.030874] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T18:28:07.367069] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-09. [2025-11-04T18:28:07.712857] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-11-04T18:28:08.167084] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-09. [2025-11-04T18:28:08.863359] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T18:28:08.967022] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-10. [2025-11-04T18:28:09.239760] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-11-04T18:28:09.767029] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-03. [2025-11-04T18:28:10.559817] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T18:28:10.570358] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-10. [2025-11-04T18:28:11.323367] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T18:28:11.367027] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-04. [2025-11-04T18:28:11.877853] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T18:28:12.167070] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-11. [2025-11-04T18:28:12.708359] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T18:28:12.967066] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-05. [2025-11-04T18:28:13.361425] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T18:28:13.362877] [INFO ] [REMOTE ] DNA: Full fetch complete (70585 records). [2025-11-04T18:28:13.362978] [DEBUG] [VALIDATOR ] DNA: Validating primary data. [2025-11-04T18:28:13.363839] [FATAL] [VALIDATOR ] DNA: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:28:13.363950] [FATAL] [MAIN ] DNA: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:28:13.364010] [WARN ] [MAIN ] DNA: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:28:13.364078] [INFO ] [LOCAL ] DNA: Starting deletion of primary data. [2025-11-04T18:28:13.364142] [DEBUG] [LOCAL ] DNA: No primary data file found to delete. [2025-11-04T18:28:13.364180] [INFO ] [LOCAL ] DNA: Starting deletion of derivative data. [2025-11-04T18:28:13.364368] [DEBUG] [LOCAL ] DNA: Deleted derivative data file. [2025-11-04T18:28:13.364460] [INFO ] [MAIN ] Update Progress: 637/2367 symbols completed. DNA took 30.1s. Estimated remaining: 3h 59m 27s. Rate: 7.2 symbols/min [2025-11-04T18:28:13.767028] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-11. [2025-11-04T18:28:14.350055] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-11-04T18:28:14.567042] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-04. [2025-11-04T18:28:14.883523] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T18:28:14.885307] [INFO ] [REMOTE ] DNOW: Full fetch complete (63905 records). [2025-11-04T18:28:14.885392] [DEBUG] [VALIDATOR ] DNOW: Validating primary data. [2025-11-04T18:28:15.289554] [DEBUG] [VALIDATOR ] DNOW: All expected trading days are present. [2025-11-04T18:28:15.289694] [DEBUG] [VALIDATOR ] DNOW: Primary data validation passed. [2025-11-04T18:28:15.290663] [INFO ] [MAIN ] DNOW: Fetched primary data is valid. [2025-11-04T18:28:15.290732] [DEBUG] [LOCAL ] DNOW: Saving primary data. [2025-11-04T18:28:15.292036] [DEBUG] [LOCAL ] DNOW: Trimmed 63905 records to 43528 (keeping 200 days of data) [2025-11-04T18:28:15.375059] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-06. [2025-11-04T18:28:15.547347] [DEBUG] [LOCAL ] DNOW: Primary data saved (43528 records). [2025-11-04T18:28:15.547465] [INFO ] [MAIN ] DNOW: Full history saved. [2025-11-04T18:28:15.547541] [INFO ] [MAIN ] Update Progress: 638/2367 symbols completed. DNOW took 30.5s. Estimated remaining: 3h 59m 2s. Rate: 7.2 symbols/min [2025-11-04T18:28:15.681247] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-11-04T18:28:16.167073] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-05. [2025-11-04T18:28:16.675474] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T18:28:16.967032] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-07. [2025-11-04T18:28:17.056277] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T18:28:17.767037] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-06. [2025-11-04T18:28:18.403561] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T18:28:18.572179] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-08. [2025-11-04T18:28:18.585611] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-11-04T18:28:19.367040] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-07. [2025-11-04T18:28:20.070526] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-11-04T18:28:20.167011] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-09. [2025-11-04T18:28:20.325441] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T18:28:20.967080] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-08. [2025-11-04T18:28:21.614874] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-11-04T18:28:21.767008] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-10. [2025-11-04T18:28:21.828216] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-11-04T18:28:22.567042] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-09. [2025-11-04T18:28:23.246442] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-11-04T18:28:23.367006] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-11. [2025-11-04T18:28:23.556811] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T18:28:24.167034] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-10. [2025-11-04T18:28:24.416079] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T18:28:24.418594] [INFO ] [REMOTE ] DOC: Full fetch complete (77242 records). [2025-11-04T18:28:24.418896] [DEBUG] [VALIDATOR ] DOC: Validating primary data. [2025-11-04T18:28:24.419976] [FATAL] [VALIDATOR ] DOC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:28:24.420079] [FATAL] [MAIN ] DOC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:28:24.420150] [WARN ] [MAIN ] DOC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:28:24.420232] [INFO ] [LOCAL ] DOC: Starting deletion of primary data. [2025-11-04T18:28:24.420323] [DEBUG] [LOCAL ] DOC: No primary data file found to delete. [2025-11-04T18:28:24.420368] [INFO ] [LOCAL ] DOC: Starting deletion of derivative data. [2025-11-04T18:28:24.420537] [DEBUG] [LOCAL ] DOC: Deleted derivative data file. [2025-11-04T18:28:24.420623] [INFO ] [MAIN ] Update Progress: 639/2367 symbols completed. DOC took 30.1s. Estimated remaining: 3h 58m 55s. Rate: 7.2 symbols/min [2025-11-04T18:28:25.139419] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T18:28:25.147364] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-11. [2025-11-04T18:28:26.177181] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T18:28:26.179076] [INFO ] [REMOTE ] DNP: Full fetch complete (48327 records). [2025-11-04T18:28:26.179178] [DEBUG] [VALIDATOR ] DNP: Validating primary data. [2025-11-04T18:28:26.573630] [DEBUG] [VALIDATOR ] DNP: All expected trading days are present. [2025-11-04T18:28:26.573743] [DEBUG] [VALIDATOR ] DNP: Primary data validation passed. [2025-11-04T18:28:26.574370] [INFO ] [MAIN ] DNP: Fetched primary data is valid. [2025-11-04T18:28:26.574411] [DEBUG] [LOCAL ] DNP: Saving primary data. [2025-11-04T18:28:26.575419] [DEBUG] [LOCAL ] DNP: Trimmed 48327 records to 32734 (keeping 200 days of data) [2025-11-04T18:28:26.772049] [DEBUG] [LOCAL ] DNP: Primary data saved (32734 records). [2025-11-04T18:28:26.772290] [INFO ] [MAIN ] DNP: Full history saved. [2025-11-04T18:28:26.772384] [INFO ] [MAIN ] Update Progress: 640/2367 symbols completed. DNP took 33.7s. Estimated remaining: 3h 58m 31s. Rate: 7.2 symbols/min [2025-11-04T18:28:27.472610] [INFO ] [MAIN ] DOLE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:28:27.477219] [INFO ] [MAIN ] DOCN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:28:27.496995] [INFO ] [MAIN ] DOCS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:28:27.518458] [INFO ] [MAIN ] DOUG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:28:27.583968] [DEBUG] [LOCAL ] DOUG: Loaded 33447 existing records from primary storage. [2025-11-04T18:28:27.584268] [INFO ] [MAIN ] DOUG: Loaded 33447 existing records. [2025-11-04T18:28:27.713904] [DEBUG] [LOCAL ] DOLE: Loaded 39218 existing records from primary storage. [2025-11-04T18:28:27.725703] [INFO ] [MAIN ] DOLE: Loaded 39218 existing records. [2025-11-04T18:28:27.731861] [DEBUG] [LOCAL ] DOCN: Loaded 51164 existing records from primary storage. [2025-11-04T18:28:27.746970] [INFO ] [MAIN ] DOCN: Loaded 51164 existing records. [2025-11-04T18:28:27.740908] [DEBUG] [LOCAL ] DOCS: Loaded 49914 existing records from primary storage. [2025-11-04T18:28:27.762582] [INFO ] [MAIN ] DOCS: Loaded 49914 existing records. [2025-11-04T18:28:28.148722] [WARN ] [MAIN ] DOUG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:28:28.180267] [INFO ] [MAIN ] DOUG: Wiping existing primary data before full refresh. [2025-11-04T18:28:28.246856] [INFO ] [LOCAL ] DOUG: Starting deletion of primary data. [2025-11-04T18:28:28.280618] [WARN ] [MAIN ] DOCN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:28:28.291595] [DEBUG] [LOCAL ] DOUG: Deleted primary data file. [2025-11-04T18:28:28.317579] [WARN ] [MAIN ] DOCS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:28:28.317948] [INFO ] [MAIN ] DOUG: Starting full primary refresh. [2025-11-04T18:28:28.323244] [INFO ] [MAIN ] DOCS: Wiping existing primary data before full refresh. [2025-11-04T18:28:28.323511] [INFO ] [LOCAL ] DOCS: Starting deletion of primary data. [2025-11-04T18:28:28.317886] [INFO ] [MAIN ] DOCN: Wiping existing primary data before full refresh. [2025-11-04T18:28:28.373171] [DEBUG] [LOCAL ] DOCS: Deleted primary data file. [2025-11-04T18:28:28.378597] [INFO ] [LOCAL ] DOCN: Starting deletion of primary data. [2025-11-04T18:28:28.387553] [INFO ] [MAIN ] DOCS: Starting full primary refresh. [2025-11-04T18:28:28.461587] [DEBUG] [LOCAL ] DOCN: Deleted primary data file. [2025-11-04T18:28:28.507047] [INFO ] [MAIN ] DOCN: Starting full primary refresh. [2025-11-04T18:28:28.532581] [WARN ] [MAIN ] DOLE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:28:28.643040] [INFO ] [MAIN ] DOLE: Wiping existing primary data before full refresh. [2025-11-04T18:28:28.662561] [INFO ] [LOCAL ] DOLE: Starting deletion of primary data. [2025-11-04T18:28:28.795557] [DEBUG] [LOCAL ] DOLE: Deleted primary data file. [2025-11-04T18:28:28.874246] [INFO ] [MAIN ] DOLE: Starting full primary refresh. [2025-11-04T18:28:29.417049] [INFO ] [REMOTE ] DOUG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:28:29.443522] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-01. [2025-11-04T18:28:29.580707] [INFO ] [REMOTE ] DOCS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:28:29.660932] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-01. [2025-11-04T18:28:29.755794] [INFO ] [REMOTE ] DOCN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:28:29.770999] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-01. [2025-11-04T18:28:29.897781] [INFO ] [REMOTE ] DOLE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:28:29.898122] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-01. [2025-11-04T18:28:30.693932] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T18:28:30.701138] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-02. [2025-11-04T18:28:31.044141] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T18:28:31.054783] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-02. [2025-11-04T18:28:31.196224] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T18:28:31.205930] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-02. [2025-11-04T18:28:31.356966] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-11-04T18:28:31.366991] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-02. [2025-11-04T18:28:31.679057] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-11-04T18:28:32.158246] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T18:28:32.166988] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-03. [2025-11-04T18:28:32.225397] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T18:28:32.331620] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T18:28:32.967065] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-03. [2025-11-04T18:28:33.191058] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-11-04T18:28:33.767024] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-03. [2025-11-04T18:28:34.124202] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-11-04T18:28:34.567042] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-03. [2025-11-04T18:28:34.730409] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T18:28:35.367081] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-04. [2025-11-04T18:28:35.886819] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T18:28:36.167021] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-04. [2025-11-04T18:28:36.271124] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-11-04T18:28:36.967087] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-04. [2025-11-04T18:28:37.621964] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T18:28:37.767052] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-04. [2025-11-04T18:28:38.431249] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-11-04T18:28:38.567003] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-05. [2025-11-04T18:28:38.881945] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-11-04T18:28:39.367119] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-05. [2025-11-04T18:28:39.809361] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-11-04T18:28:40.167043] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-06. [2025-11-04T18:28:40.282955] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T18:28:40.967144] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-05. [2025-11-04T18:28:41.738022] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-11-04T18:28:41.767098] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-06. [2025-11-04T18:28:42.493379] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-11-04T18:28:42.567038] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-06. [2025-11-04T18:28:42.855302] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T18:28:43.367044] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-07. [2025-11-04T18:28:43.852669] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-11-04T18:28:44.167048] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-07. [2025-11-04T18:28:44.612312] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:28:44.967045] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-08. [2025-11-04T18:28:45.767064] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-05. [2025-11-04T18:28:45.796301] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-11-04T18:28:46.188194] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T18:28:46.567051] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-07. [2025-11-04T18:28:46.982213] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T18:28:47.367053] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-06. [2025-11-04T18:28:47.449128] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-11-04T18:28:48.167110] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-08. [2025-11-04T18:28:48.385058] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T18:28:48.967118] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-07. [2025-11-04T18:28:49.649177] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-11-04T18:28:49.767039] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-09. [2025-11-04T18:28:50.061741] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-11-04T18:28:50.567080] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-09. [2025-11-04T18:28:51.151861] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T18:28:51.367030] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-10. [2025-11-04T18:28:51.933522] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-11-04T18:28:52.167128] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-10. [2025-11-04T18:28:52.894454] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-11-04T18:28:52.967018] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-11. [2025-11-04T18:28:53.414228] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T18:28:53.767045] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-08. [2025-11-04T18:28:54.208005] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T18:28:54.210012] [INFO ] [REMOTE ] DOCN: Full fetch complete (73449 records). [2025-11-04T18:28:54.210111] [DEBUG] [VALIDATOR ] DOCN: Validating primary data. [2025-11-04T18:28:54.593800] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-11. [2025-11-04T18:28:54.616933] [DEBUG] [VALIDATOR ] DOCN: All expected trading days are present. [2025-11-04T18:28:54.618215] [DEBUG] [VALIDATOR ] DOCN: Primary data validation passed. [2025-11-04T18:28:54.619106] [INFO ] [MAIN ] DOCN: Fetched primary data is valid. [2025-11-04T18:28:54.619261] [DEBUG] [LOCAL ] DOCN: Saving primary data. [2025-11-04T18:28:54.620827] [DEBUG] [LOCAL ] DOCN: Trimmed 73449 records to 51235 (keeping 200 days of data) [2025-11-04T18:28:54.930100] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T18:28:54.954128] [DEBUG] [LOCAL ] DOCN: Primary data saved (51235 records). [2025-11-04T18:28:54.954270] [INFO ] [MAIN ] DOCN: Full history saved. [2025-11-04T18:28:54.954363] [INFO ] [MAIN ] Update Progress: 641/2367 symbols completed. DOCN took 28.2s. Estimated remaining: 3h 59m 16s. Rate: 7.2 symbols/min [2025-11-04T18:28:55.128591] [INFO ] [MAIN ] DOV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:28:55.171736] [DEBUG] [LOCAL ] DOV: Loaded 45492 existing records from primary storage. [2025-11-04T18:28:55.172068] [INFO ] [MAIN ] DOV: Loaded 45492 existing records. [2025-11-04T18:28:55.355256] [WARN ] [MAIN ] DOV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:28:55.355492] [INFO ] [MAIN ] DOV: Wiping existing primary data before full refresh. [2025-11-04T18:28:55.355545] [INFO ] [LOCAL ] DOV: Starting deletion of primary data. [2025-11-04T18:28:55.355823] [DEBUG] [LOCAL ] DOV: Deleted primary data file. [2025-11-04T18:28:55.355881] [INFO ] [MAIN ] DOV: Starting full primary refresh. [2025-11-04T18:28:55.377489] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-08. [2025-11-04T18:28:55.742286] [INFO ] [REMOTE ] DOV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:28:55.743053] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T18:28:55.745541] [INFO ] [REMOTE ] DOCS: Full fetch complete (73525 records). [2025-11-04T18:28:55.745634] [DEBUG] [VALIDATOR ] DOCS: Validating primary data. [2025-11-04T18:28:56.133368] [DEBUG] [VALIDATOR ] DOCS: All expected trading days are present. [2025-11-04T18:28:56.133445] [DEBUG] [VALIDATOR ] DOCS: Primary data validation passed. [2025-11-04T18:28:56.134240] [INFO ] [MAIN ] DOCS: Fetched primary data is valid. [2025-11-04T18:28:56.134302] [DEBUG] [LOCAL ] DOCS: Saving primary data. [2025-11-04T18:28:56.135775] [DEBUG] [LOCAL ] DOCS: Trimmed 73525 records to 49868 (keeping 200 days of data) [2025-11-04T18:28:56.185224] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-01. [2025-11-04T18:28:56.423570] [DEBUG] [LOCAL ] DOCS: Primary data saved (49868 records). [2025-11-04T18:28:56.423716] [INFO ] [MAIN ] DOCS: Full history saved. [2025-11-04T18:28:56.423813] [INFO ] [MAIN ] Update Progress: 642/2367 symbols completed. DOCS took 29.7s. Estimated remaining: 3h 58m 50s. Rate: 7.2 symbols/min [2025-11-04T18:28:56.595192] [INFO ] [MAIN ] DOW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:28:56.644962] [DEBUG] [LOCAL ] DOW: Loaded 53733 existing records from primary storage. [2025-11-04T18:28:56.645502] [INFO ] [MAIN ] DOW: Loaded 53733 existing records. [2025-11-04T18:28:56.827481] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T18:28:56.863613] [WARN ] [MAIN ] DOW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:28:56.864116] [INFO ] [MAIN ] DOW: Wiping existing primary data before full refresh. [2025-11-04T18:28:56.864164] [INFO ] [LOCAL ] DOW: Starting deletion of primary data. [2025-11-04T18:28:56.864355] [DEBUG] [LOCAL ] DOW: Deleted primary data file. [2025-11-04T18:28:56.864410] [INFO ] [MAIN ] DOW: Starting full primary refresh. [2025-11-04T18:28:56.971966] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-09. [2025-11-04T18:28:57.229198] [INFO ] [REMOTE ] DOW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:28:57.591071] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T18:28:57.767060] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-02. [2025-11-04T18:28:58.121261] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T18:28:58.567042] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-09. [2025-11-04T18:28:58.823069] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T18:28:59.367056] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-03. [2025-11-04T18:28:59.545688] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T18:29:00.167053] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-10. [2025-11-04T18:29:00.708684] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T18:29:00.967044] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-04. [2025-11-04T18:29:01.294114] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-11-04T18:29:01.767048] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-11. [2025-11-04T18:29:02.351735] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T18:29:02.535332] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T18:29:02.536780] [INFO ] [REMOTE ] DOUG: Full fetch complete (49068 records). [2025-11-04T18:29:02.536878] [DEBUG] [VALIDATOR ] DOUG: Validating primary data. [2025-11-04T18:29:02.572134] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-10. [2025-11-04T18:29:02.910840] [DEBUG] [VALIDATOR ] DOUG: All expected trading days are present. [2025-11-04T18:29:02.910914] [DEBUG] [VALIDATOR ] DOUG: Primary data validation passed. [2025-11-04T18:29:02.911516] [INFO ] [MAIN ] DOUG: Fetched primary data is valid. [2025-11-04T18:29:02.911555] [DEBUG] [LOCAL ] DOUG: Saving primary data. [2025-11-04T18:29:02.912519] [DEBUG] [LOCAL ] DOUG: Trimmed 49068 records to 33369 (keeping 200 days of data) [2025-11-04T18:29:03.092283] [DEBUG] [LOCAL ] DOUG: Primary data saved (33369 records). [2025-11-04T18:29:03.092415] [INFO ] [MAIN ] DOUG: Full history saved. [2025-11-04T18:29:03.092491] [INFO ] [MAIN ] Update Progress: 643/2367 symbols completed. DOUG took 36.3s. Estimated remaining: 3h 58m 37s. Rate: 7.2 symbols/min [2025-11-04T18:29:03.257370] [INFO ] [MAIN ] DPG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:29:03.271707] [DEBUG] [LOCAL ] DPG: Loaded 12995 existing records from primary storage. [2025-11-04T18:29:03.271922] [INFO ] [MAIN ] DPG: Loaded 12995 existing records. [2025-11-04T18:29:03.371452] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-05. [2025-11-04T18:29:03.448272] [WARN ] [MAIN ] DPG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:29:03.448508] [INFO ] [MAIN ] DPG: Wiping existing primary data before full refresh. [2025-11-04T18:29:03.448557] [INFO ] [LOCAL ] DPG: Starting deletion of primary data. [2025-11-04T18:29:03.448733] [DEBUG] [LOCAL ] DPG: Deleted primary data file. [2025-11-04T18:29:03.448793] [INFO ] [MAIN ] DPG: Starting full primary refresh. [2025-11-04T18:29:03.804190] [INFO ] [REMOTE ] DPG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:29:03.837482] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T18:29:04.167066] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-11. [2025-11-04T18:29:04.447375] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-11-04T18:29:04.967078] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-06. [2025-11-04T18:29:05.203910] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T18:29:05.206299] [INFO ] [REMOTE ] DOLE: Full fetch complete (54962 records). [2025-11-04T18:29:05.206461] [DEBUG] [VALIDATOR ] DOLE: Validating primary data. [2025-11-04T18:29:05.630476] [DEBUG] [VALIDATOR ] DOLE: All expected trading days are present. [2025-11-04T18:29:05.630602] [DEBUG] [VALIDATOR ] DOLE: Primary data validation passed. [2025-11-04T18:29:05.631489] [INFO ] [MAIN ] DOLE: Fetched primary data is valid. [2025-11-04T18:29:05.631554] [DEBUG] [LOCAL ] DOLE: Saving primary data. [2025-11-04T18:29:05.633020] [DEBUG] [LOCAL ] DOLE: Trimmed 54962 records to 39184 (keeping 200 days of data) [2025-11-04T18:29:05.767066] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-01. [2025-11-04T18:29:05.893249] [DEBUG] [LOCAL ] DOLE: Primary data saved (39184 records). [2025-11-04T18:29:05.893484] [INFO ] [MAIN ] DOLE: Full history saved. [2025-11-04T18:29:05.893606] [INFO ] [MAIN ] Update Progress: 644/2367 symbols completed. DOLE took 39.1s. Estimated remaining: 3h 58m 14s. Rate: 7.2 symbols/min [2025-11-04T18:29:06.074788] [INFO ] [MAIN ] DQ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:29:06.118863] [DEBUG] [LOCAL ] DQ: Loaded 43820 existing records from primary storage. [2025-11-04T18:29:06.119383] [INFO ] [MAIN ] DQ: Loaded 43820 existing records. [2025-11-04T18:29:06.299739] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T18:29:06.350340] [WARN ] [MAIN ] DQ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:29:06.351573] [INFO ] [MAIN ] DQ: Wiping existing primary data before full refresh. [2025-11-04T18:29:06.351725] [INFO ] [LOCAL ] DQ: Starting deletion of primary data. [2025-11-04T18:29:06.351991] [DEBUG] [LOCAL ] DQ: Deleted primary data file. [2025-11-04T18:29:06.352051] [INFO ] [MAIN ] DQ: Starting full primary refresh. [2025-11-04T18:29:06.572178] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-07. [2025-11-04T18:29:06.672568] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-11-04T18:29:06.757526] [INFO ] [REMOTE ] DQ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:29:07.367064] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-01. [2025-11-04T18:29:08.008185] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T18:29:08.167021] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-08. [2025-11-04T18:29:08.485329] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T18:29:08.967048] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-02. [2025-11-04T18:29:09.277568] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T18:29:09.385346] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-11-04T18:29:09.767025] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-01. [2025-11-04T18:29:10.567050] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-02. [2025-11-04T18:29:11.367047] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-09. [2025-11-04T18:29:11.405696] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-11-04T18:29:11.657703] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T18:29:12.167046] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-03. [2025-11-04T18:29:12.584062] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T18:29:12.611411] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-11-04T18:29:12.967058] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-03. [2025-11-04T18:29:13.767043] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-10. [2025-11-04T18:29:14.265601] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T18:29:14.567062] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-04. [2025-11-04T18:29:15.164932] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T18:29:15.367065] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-11. [2025-11-04T18:29:15.936815] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-11-04T18:29:16.167038] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-05. [2025-11-04T18:29:16.599707] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T18:29:16.601457] [INFO ] [REMOTE ] DOV: Full fetch complete (66489 records). [2025-11-04T18:29:16.601544] [DEBUG] [VALIDATOR ] DOV: Validating primary data. [2025-11-04T18:29:16.981771] [DEBUG] [VALIDATOR ] DOV: All expected trading days are present. [2025-11-04T18:29:16.978476] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-04. [2025-11-04T18:29:16.982832] [DEBUG] [VALIDATOR ] DOV: Primary data validation passed. [2025-11-04T18:29:16.983597] [INFO ] [MAIN ] DOV: Fetched primary data is valid. [2025-11-04T18:29:16.983687] [DEBUG] [LOCAL ] DOV: Saving primary data. [2025-11-04T18:29:16.985005] [DEBUG] [LOCAL ] DOV: Trimmed 66489 records to 45440 (keeping 200 days of data) [2025-11-04T18:29:17.277174] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T18:29:17.291111] [DEBUG] [LOCAL ] DOV: Primary data saved (45440 records). [2025-11-04T18:29:17.291198] [INFO ] [MAIN ] DOV: Full history saved. [2025-11-04T18:29:17.291268] [INFO ] [MAIN ] Update Progress: 645/2367 symbols completed. DOV took 22.3s. Estimated remaining: 3h 58m 14s. Rate: 7.2 symbols/min [2025-11-04T18:29:17.462786] [INFO ] [MAIN ] DRD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:29:17.497365] [DEBUG] [LOCAL ] DRD: Loaded 37782 existing records from primary storage. [2025-11-04T18:29:17.497696] [INFO ] [MAIN ] DRD: Loaded 37782 existing records. [2025-11-04T18:29:17.686988] [WARN ] [MAIN ] DRD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:29:17.687231] [INFO ] [MAIN ] DRD: Wiping existing primary data before full refresh. [2025-11-04T18:29:17.687277] [INFO ] [LOCAL ] DRD: Starting deletion of primary data. [2025-11-04T18:29:17.687448] [DEBUG] [LOCAL ] DRD: Deleted primary data file. [2025-11-04T18:29:17.687483] [INFO ] [MAIN ] DRD: Starting full primary refresh. [2025-11-04T18:29:17.770368] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-02. [2025-11-04T18:29:18.031718] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-11-04T18:29:18.093515] [INFO ] [REMOTE ] DRD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:29:18.567037] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-01. [2025-11-04T18:29:19.208153] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-11-04T18:29:19.367032] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-03. [2025-11-04T18:29:19.545375] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T18:29:20.167040] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-05. [2025-11-04T18:29:20.805459] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T18:29:20.840470] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-11-04T18:29:20.967040] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-06. [2025-11-04T18:29:21.505056] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-11-04T18:29:21.767030] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-04. [2025-11-04T18:29:22.567044] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-06. [2025-11-04T18:29:23.367006] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-07. [2025-11-04T18:29:23.441457] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-11-04T18:29:23.646971] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T18:29:24.034158] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T18:29:24.167047] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-08. [2025-11-04T18:29:24.733761] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-11-04T18:29:24.967050] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-07. [2025-11-04T18:29:25.767031] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-05. [2025-11-04T18:29:26.229237] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T18:29:26.567096] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-02. [2025-11-04T18:29:27.306513] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T18:29:27.367040] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-08. [2025-11-04T18:29:27.531041] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T18:29:28.167050] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-06. [2025-11-04T18:29:28.480894] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T18:29:28.967124] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-09. [2025-11-04T18:29:29.775814] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-09. [2025-11-04T18:29:29.798678] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-11-04T18:29:30.064908] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T18:29:30.515090] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-11-04T18:29:30.567033] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-07. [2025-11-04T18:29:31.367098] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-10. [2025-11-04T18:29:32.167044] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-03. [2025-11-04T18:29:32.378455] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-11-04T18:29:32.601353] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T18:29:32.967125] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-10. [2025-11-04T18:29:33.518518] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-11-04T18:29:33.586612] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T18:29:33.767023] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-11. [2025-11-04T18:29:34.351814] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-11 (0.58 seconds). [2025-11-04T18:29:34.352796] [INFO ] [REMOTE ] DPG: Full fetch complete (18861 records). [2025-11-04T18:29:34.352867] [DEBUG] [VALIDATOR ] DPG: Validating primary data. [2025-11-04T18:29:34.572920] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-04. [2025-11-04T18:29:34.734368] [DEBUG] [VALIDATOR ] DPG: All expected trading days are present. [2025-11-04T18:29:34.734492] [DEBUG] [VALIDATOR ] DPG: Primary data validation passed. [2025-11-04T18:29:34.735186] [INFO ] [MAIN ] DPG: Fetched primary data is valid. [2025-11-04T18:29:34.735280] [DEBUG] [LOCAL ] DPG: Saving primary data. [2025-11-04T18:29:34.736289] [DEBUG] [LOCAL ] DPG: Trimmed 18861 records to 12966 (keeping 200 days of data) [2025-11-04T18:29:34.826172] [DEBUG] [LOCAL ] DPG: Primary data saved (12966 records). [2025-11-04T18:29:34.826279] [INFO ] [MAIN ] DPG: Full history saved. [2025-11-04T18:29:34.826360] [INFO ] [MAIN ] Update Progress: 646/2367 symbols completed. DPG took 31.7s. Estimated remaining: 3h 58m 30s. Rate: 7.2 symbols/min [2025-11-04T18:29:35.039116] [INFO ] [MAIN ] DRH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:29:35.084920] [DEBUG] [LOCAL ] DRH: Loaded 50035 existing records from primary storage. [2025-11-04T18:29:35.085339] [INFO ] [MAIN ] DRH: Loaded 50035 existing records. [2025-11-04T18:29:35.306491] [WARN ] [MAIN ] DRH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:29:35.306739] [INFO ] [MAIN ] DRH: Wiping existing primary data before full refresh. [2025-11-04T18:29:35.306835] [INFO ] [LOCAL ] DRH: Starting deletion of primary data. [2025-11-04T18:29:35.307028] [DEBUG] [LOCAL ] DRH: Deleted primary data file. [2025-11-04T18:29:35.307085] [INFO ] [MAIN ] DRH: Starting full primary refresh. [2025-11-04T18:29:35.373945] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-11. [2025-11-04T18:29:35.759924] [INFO ] [REMOTE ] DRH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:29:35.871433] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T18:29:36.167037] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-08. [2025-11-04T18:29:36.589386] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T18:29:36.591163] [INFO ] [REMOTE ] DQ: Full fetch complete (64116 records). [2025-11-04T18:29:36.591243] [DEBUG] [VALIDATOR ] DQ: Validating primary data. [2025-11-04T18:29:36.967355] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-05. [2025-11-04T18:29:36.968588] [DEBUG] [VALIDATOR ] DQ: All expected trading days are present. [2025-11-04T18:29:36.968689] [DEBUG] [VALIDATOR ] DQ: Primary data validation passed. [2025-11-04T18:29:36.969429] [INFO ] [MAIN ] DQ: Fetched primary data is valid. [2025-11-04T18:29:36.969595] [DEBUG] [LOCAL ] DQ: Saving primary data. [2025-11-04T18:29:36.970886] [DEBUG] [LOCAL ] DQ: Trimmed 64116 records to 43833 (keeping 200 days of data) [2025-11-04T18:29:37.212950] [DEBUG] [LOCAL ] DQ: Primary data saved (43833 records). [2025-11-04T18:29:37.213066] [INFO ] [MAIN ] DQ: Full history saved. [2025-11-04T18:29:37.213140] [INFO ] [MAIN ] Update Progress: 647/2367 symbols completed. DQ took 31.3s. Estimated remaining: 3h 58m 6s. Rate: 7.2 symbols/min [2025-11-04T18:29:37.378076] [INFO ] [MAIN ] DRI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:29:37.419130] [DEBUG] [LOCAL ] DRI: Loaded 47343 existing records from primary storage. [2025-11-04T18:29:37.419414] [INFO ] [MAIN ] DRI: Loaded 47343 existing records. [2025-11-04T18:29:37.596000] [WARN ] [MAIN ] DRI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:29:37.596199] [INFO ] [MAIN ] DRI: Wiping existing primary data before full refresh. [2025-11-04T18:29:37.596256] [INFO ] [LOCAL ] DRI: Starting deletion of primary data. [2025-11-04T18:29:37.596457] [DEBUG] [LOCAL ] DRI: Deleted primary data file. [2025-11-04T18:29:37.596513] [INFO ] [MAIN ] DRI: Starting full primary refresh. [2025-11-04T18:29:37.790077] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-01. [2025-11-04T18:29:37.950223] [INFO ] [REMOTE ] DRI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:29:37.967490] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-11-04T18:29:38.104598] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-11-04T18:29:38.567079] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-09. [2025-11-04T18:29:39.305998] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-11-04T18:29:39.367021] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-02. [2025-11-04T18:29:40.125875] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-11-04T18:29:40.167001] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-10. [2025-11-04T18:29:40.419920] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T18:29:40.967081] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-01. [2025-11-04T18:29:41.767072] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-03. [2025-11-04T18:29:41.873132] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-11-04T18:29:42.186762] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-11-04T18:29:42.567043] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-06. [2025-11-04T18:29:42.967138] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T18:29:43.367067] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-02. [2025-11-04T18:29:43.749616] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T18:29:44.167029] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-11. [2025-11-04T18:29:44.650752] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T18:29:44.967142] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-04. [2025-11-04T18:29:45.765078] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-11 (1.60 seconds). [2025-11-04T18:29:45.767038] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-03. [2025-11-04T18:29:45.768242] [INFO ] [REMOTE ] DOW: Full fetch complete (77911 records). [2025-11-04T18:29:45.768830] [DEBUG] [VALIDATOR ] DOW: Validating primary data. [2025-11-04T18:29:46.173484] [DEBUG] [VALIDATOR ] DOW: All expected trading days are present. [2025-11-04T18:29:46.173621] [DEBUG] [VALIDATOR ] DOW: Primary data validation passed. [2025-11-04T18:29:46.174569] [INFO ] [MAIN ] DOW: Fetched primary data is valid. [2025-11-04T18:29:46.174673] [DEBUG] [LOCAL ] DOW: Saving primary data. [2025-11-04T18:29:46.191284] [DEBUG] [LOCAL ] DOW: Trimmed 77911 records to 53733 (keeping 200 days of data) [2025-11-04T18:29:46.205578] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T18:29:46.535124] [DEBUG] [LOCAL ] DOW: Primary data saved (53733 records). [2025-11-04T18:29:46.535266] [INFO ] [MAIN ] DOW: Full history saved. [2025-11-04T18:29:46.535350] [INFO ] [MAIN ] Update Progress: 648/2367 symbols completed. DOW took 50.1s. Estimated remaining: 3h 58m 1s. Rate: 7.2 symbols/min [2025-11-04T18:29:46.572306] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-07. [2025-11-04T18:29:46.705356] [INFO ] [MAIN ] DRLL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:29:46.714701] [DEBUG] [LOCAL ] DRLL: Loaded 8351 existing records from primary storage. [2025-11-04T18:29:46.714920] [INFO ] [MAIN ] DRLL: Loaded 8351 existing records. [2025-11-04T18:29:46.888378] [WARN ] [MAIN ] DRLL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:29:46.888587] [INFO ] [MAIN ] DRLL: Wiping existing primary data before full refresh. [2025-11-04T18:29:46.888635] [INFO ] [LOCAL ] DRLL: Starting deletion of primary data. [2025-11-04T18:29:46.888857] [DEBUG] [LOCAL ] DRLL: Deleted primary data file. [2025-11-04T18:29:46.888912] [INFO ] [MAIN ] DRLL: Starting full primary refresh. [2025-11-04T18:29:47.244618] [INFO ] [REMOTE ] DRLL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:29:47.293382] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T18:29:47.367014] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-01. [2025-11-04T18:29:47.645559] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T18:29:47.980268] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-11-04T18:29:48.167101] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-05. [2025-11-04T18:29:48.967127] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-08. [2025-11-04T18:29:49.488997] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:29:49.767050] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-02. [2025-11-04T18:29:50.076368] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T18:29:50.188859] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-11-04T18:29:50.567084] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-03. [2025-11-04T18:29:51.190476] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-11-04T18:29:51.367090] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-09. [2025-11-04T18:29:52.167090] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-04. [2025-11-04T18:29:52.620553] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T18:29:52.708109] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-11-04T18:29:52.967046] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-06. [2025-11-04T18:29:53.767045] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-10. [2025-11-04T18:29:54.171566] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T18:29:54.567151] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-04. [2025-11-04T18:29:54.963844] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T18:29:55.367066] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-11. [2025-11-04T18:29:55.967413] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T18:29:56.167078] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-07. [2025-11-04T18:29:56.638741] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-11 (1.27 seconds). [2025-11-04T18:29:56.640234] [INFO ] [REMOTE ] DRD: Full fetch complete (52995 records). [2025-11-04T18:29:56.640311] [DEBUG] [VALIDATOR ] DRD: Validating primary data. [2025-11-04T18:29:56.972189] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-05. [2025-11-04T18:29:57.026691] [DEBUG] [VALIDATOR ] DRD: All expected trading days are present. [2025-11-04T18:29:57.026818] [DEBUG] [VALIDATOR ] DRD: Primary data validation passed. [2025-11-04T18:29:57.027546] [INFO ] [MAIN ] DRD: Fetched primary data is valid. [2025-11-04T18:29:57.027590] [DEBUG] [LOCAL ] DRD: Saving primary data. [2025-11-04T18:29:57.028801] [DEBUG] [LOCAL ] DRD: Trimmed 52995 records to 37786 (keeping 200 days of data) [2025-11-04T18:29:57.253977] [DEBUG] [LOCAL ] DRD: Primary data saved (37786 records). [2025-11-04T18:29:57.254273] [INFO ] [MAIN ] DRD: Full history saved. [2025-11-04T18:29:57.254394] [INFO ] [MAIN ] Update Progress: 649/2367 symbols completed. DRD took 40.0s. Estimated remaining: 3h 57m 59s. Rate: 7.2 symbols/min [2025-11-04T18:29:57.432023] [INFO ] [MAIN ] DSL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:29:57.432219] [DEBUG] [LOCAL ] DSL: No existing data file found in primary storage. [2025-11-04T18:29:57.432332] [WARN ] [MAIN ] DSL: No existing data found, performing full refresh instead. [2025-11-04T18:29:57.432479] [INFO ] [MAIN ] DSL: Wiping existing primary data before full refresh. [2025-11-04T18:29:57.432543] [INFO ] [LOCAL ] DSL: Starting deletion of primary data. [2025-11-04T18:29:57.432631] [DEBUG] [LOCAL ] DSL: No primary data file found to delete. [2025-11-04T18:29:57.432839] [INFO ] [MAIN ] DSL: Starting full primary refresh. [2025-11-04T18:29:57.561583] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-11-04T18:29:57.636874] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T18:29:57.772098] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-05. [2025-11-04T18:29:57.837878] [INFO ] [REMOTE ] DSL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:29:58.567052] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-08. [2025-11-04T18:29:59.056384] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T18:29:59.367039] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-06. [2025-11-04T18:29:59.783463] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T18:30:00.167081] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-01. [2025-11-04T18:30:00.774557] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-11-04T18:30:00.967052] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-09. [2025-11-04T18:30:01.319976] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T18:30:01.767034] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-06. [2025-11-04T18:30:02.208191] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-11-04T18:30:02.311460] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-11-04T18:30:02.567059] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-10. [2025-11-04T18:30:03.367061] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-07. [2025-11-04T18:30:03.880956] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T18:30:04.167060] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-11. [2025-11-04T18:30:04.696447] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T18:30:04.967093] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-08. [2025-11-04T18:30:05.324353] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T18:30:05.325911] [INFO ] [REMOTE ] DRH: Full fetch complete (71749 records). [2025-11-04T18:30:05.326004] [DEBUG] [VALIDATOR ] DRH: Validating primary data. [2025-11-04T18:30:05.326672] [FATAL] [VALIDATOR ] DRH: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:30:05.329724] [FATAL] [MAIN ] DRH: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:30:05.329831] [WARN ] [MAIN ] DRH: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:30:05.329923] [INFO ] [LOCAL ] DRH: Starting deletion of primary data. [2025-11-04T18:30:05.330031] [DEBUG] [LOCAL ] DRH: No primary data file found to delete. [2025-11-04T18:30:05.330086] [INFO ] [LOCAL ] DRH: Starting deletion of derivative data. [2025-11-04T18:30:05.330213] [DEBUG] [LOCAL ] DRH: Deleted derivative data file. [2025-11-04T18:30:05.330323] [INFO ] [MAIN ] Update Progress: 650/2367 symbols completed. DRH took 30.5s. Estimated remaining: 3h 57m 50s. Rate: 7.2 symbols/min [2025-11-04T18:30:05.497095] [INFO ] [MAIN ] DSM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:30:05.511116] [DEBUG] [LOCAL ] DSM: Loaded 13679 existing records from primary storage. [2025-11-04T18:30:05.511333] [INFO ] [MAIN ] DSM: Loaded 13679 existing records. [2025-11-04T18:30:05.684124] [WARN ] [MAIN ] DSM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:30:05.684371] [INFO ] [MAIN ] DSM: Wiping existing primary data before full refresh. [2025-11-04T18:30:05.684430] [INFO ] [LOCAL ] DSM: Starting deletion of primary data. [2025-11-04T18:30:05.684579] [DEBUG] [LOCAL ] DSM: Deleted primary data file. [2025-11-04T18:30:05.684615] [INFO ] [MAIN ] DSM: Starting full primary refresh. [2025-11-04T18:30:05.773049] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-07. [2025-11-04T18:30:06.035494] [INFO ] [REMOTE ] DSM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:30:06.303191] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-11-04T18:30:06.458836] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T18:30:06.567026] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-09. [2025-11-04T18:30:07.367060] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-02. [2025-11-04T18:30:07.997025] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T18:30:08.167079] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-08. [2025-11-04T18:30:08.279991] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T18:30:08.764193] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-11-04T18:30:08.967046] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-03. [2025-11-04T18:30:09.767015] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-09. [2025-11-04T18:30:09.908351] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-11-04T18:30:10.223948] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-11-04T18:30:10.567083] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-10. [2025-11-04T18:30:11.367068] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-04. [2025-11-04T18:30:11.974524] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T18:30:12.167027] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-11. [2025-11-04T18:30:12.500459] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T18:30:12.555936] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-11 (0.39 seconds). [2025-11-04T18:30:12.557961] [INFO ] [REMOTE ] DRI: Full fetch complete (69555 records). [2025-11-04T18:30:12.558096] [DEBUG] [VALIDATOR ] DRI: Validating primary data. [2025-11-04T18:30:12.945037] [DEBUG] [VALIDATOR ] DRI: All expected trading days are present. [2025-11-04T18:30:12.945119] [DEBUG] [VALIDATOR ] DRI: Primary data validation passed. [2025-11-04T18:30:12.945928] [INFO ] [MAIN ] DRI: Fetched primary data is valid. [2025-11-04T18:30:12.945989] [DEBUG] [LOCAL ] DRI: Saving primary data. [2025-11-04T18:30:12.947262] [DEBUG] [LOCAL ] DRI: Trimmed 69555 records to 47402 (keeping 200 days of data) [2025-11-04T18:30:13.000712] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-01. [2025-11-04T18:30:13.239564] [DEBUG] [LOCAL ] DRI: Primary data saved (47402 records). [2025-11-04T18:30:13.239727] [INFO ] [MAIN ] DRI: Full history saved. [2025-11-04T18:30:13.239846] [INFO ] [MAIN ] Update Progress: 651/2367 symbols completed. DRI took 36.0s. Estimated remaining: 3h 57m 40s. Rate: 7.2 symbols/min [2025-11-04T18:30:13.403358] [INFO ] [MAIN ] DSMC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:30:13.407317] [DEBUG] [LOCAL ] DSMC: Loaded 1735 existing records from primary storage. [2025-11-04T18:30:13.407453] [INFO ] [MAIN ] DSMC: Loaded 1735 existing records. [2025-11-04T18:30:13.582117] [WARN ] [MAIN ] DSMC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:30:13.582268] [INFO ] [MAIN ] DSMC: Wiping existing primary data before full refresh. [2025-11-04T18:30:13.582311] [INFO ] [LOCAL ] DSMC: Starting deletion of primary data. [2025-11-04T18:30:13.582430] [DEBUG] [LOCAL ] DSMC: Deleted primary data file. [2025-11-04T18:30:13.582467] [INFO ] [MAIN ] DSMC: Starting full primary refresh. [2025-11-04T18:30:13.773179] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-05. [2025-11-04T18:30:13.931056] [INFO ] [REMOTE ] DSMC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:30:13.943487] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-11-04T18:30:14.516319] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-11-04T18:30:14.566999] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-02. [2025-11-04T18:30:15.330530] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T18:30:15.367000] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-03. [2025-11-04T18:30:16.167168] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-10. [2025-11-04T18:30:16.170785] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-11-04T18:30:16.748560] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-11-04T18:30:16.967038] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-06. [2025-11-04T18:30:17.767021] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-11. [2025-11-04T18:30:17.833965] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T18:30:18.262432] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T18:30:18.263639] [INFO ] [REMOTE ] DRLL: Full fetch complete (14274 records). [2025-11-04T18:30:18.263785] [DEBUG] [VALIDATOR ] DRLL: Validating primary data. [2025-11-04T18:30:18.572194] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-01. [2025-11-04T18:30:18.636830] [DEBUG] [VALIDATOR ] DRLL: All expected trading days are present. [2025-11-04T18:30:18.636905] [DEBUG] [VALIDATOR ] DRLL: Primary data validation passed. [2025-11-04T18:30:18.637215] [INFO ] [MAIN ] DRLL: Fetched primary data is valid. [2025-11-04T18:30:18.637252] [DEBUG] [LOCAL ] DRLL: Saving primary data. [2025-11-04T18:30:18.637836] [DEBUG] [LOCAL ] DRLL: Trimmed 14274 records to 8335 (keeping 200 days of data) [2025-11-04T18:30:18.687485] [DEBUG] [LOCAL ] DRLL: Primary data saved (8335 records). [2025-11-04T18:30:18.687616] [INFO ] [MAIN ] DRLL: Full history saved. [2025-11-04T18:30:18.687746] [INFO ] [MAIN ] Update Progress: 652/2367 symbols completed. DRLL took 32.2s. Estimated remaining: 3h 57m 24s. Rate: 7.2 symbols/min [2025-11-04T18:30:18.863506] [INFO ] [MAIN ] DSU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:30:18.887814] [DEBUG] [LOCAL ] DSU: Loaded 22360 existing records from primary storage. [2025-11-04T18:30:18.888049] [INFO ] [MAIN ] DSU: Loaded 22360 existing records. [2025-11-04T18:30:19.063163] [WARN ] [MAIN ] DSU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:30:19.065825] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-01 (0.49 seconds). [2025-11-04T18:30:19.066952] [INFO ] [MAIN ] DSU: Wiping existing primary data before full refresh. [2025-11-04T18:30:19.067009] [INFO ] [LOCAL ] DSU: Starting deletion of primary data. [2025-11-04T18:30:19.067206] [DEBUG] [LOCAL ] DSU: Deleted primary data file. [2025-11-04T18:30:19.067273] [INFO ] [MAIN ] DSU: Starting full primary refresh. [2025-11-04T18:30:19.377509] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-02. [2025-11-04T18:30:19.418114] [INFO ] [REMOTE ] DSU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:30:19.694393] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T18:30:20.167022] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-04. [2025-11-04T18:30:20.888972] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-11-04T18:30:20.967128] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-01. [2025-11-04T18:30:21.767043] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-07. [2025-11-04T18:30:21.870336] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T18:30:22.567056] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-02. [2025-11-04T18:30:22.860510] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-11-04T18:30:23.311357] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-11-04T18:30:23.367004] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-03. [2025-11-04T18:30:24.060197] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-11-04T18:30:24.167011] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-04. [2025-11-04T18:30:24.967052] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-03. [2025-11-04T18:30:24.987176] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T18:30:25.294917] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-11-04T18:30:25.767087] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-08. [2025-11-04T18:30:26.567029] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-05. [2025-11-04T18:30:26.677914] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-11-04T18:30:27.280910] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-11-04T18:30:27.367021] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-06. [2025-11-04T18:30:28.020065] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-11-04T18:30:28.167032] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-04. [2025-11-04T18:30:28.495449] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-11-04T18:30:28.967098] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-05. [2025-11-04T18:30:29.686828] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-11-04T18:30:29.767040] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-06. [2025-11-04T18:30:30.468231] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T18:30:30.567080] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-07. [2025-11-04T18:30:31.367066] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-07. [2025-11-04T18:30:31.404610] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-11-04T18:30:31.837243] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-11-04T18:30:32.167125] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-08. [2025-11-04T18:30:32.789057] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-11-04T18:30:32.967042] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-09. [2025-11-04T18:30:33.574731] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-11-04T18:30:33.767056] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-05. [2025-11-04T18:30:34.155526] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-11-04T18:30:34.567066] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-09. [2025-11-04T18:30:35.367041] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-08. [2025-11-04T18:30:35.475329] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-11-04T18:30:36.088553] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-11-04T18:30:36.167074] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-10. [2025-11-04T18:30:36.967013] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-10. [2025-11-04T18:30:37.302571] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T18:30:37.767037] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-11. [2025-11-04T18:30:37.790607] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T18:30:38.535450] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T18:30:38.536972] [INFO ] [REMOTE ] DSL: Full fetch complete (42376 records). [2025-11-04T18:30:38.537045] [DEBUG] [VALIDATOR ] DSL: Validating primary data. [2025-11-04T18:30:38.572147] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-06. [2025-11-04T18:30:38.913275] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-11-04T18:30:38.914619] [DEBUG] [VALIDATOR ] DSL: All expected trading days are present. [2025-11-04T18:30:38.914705] [DEBUG] [VALIDATOR ] DSL: Primary data validation passed. [2025-11-04T18:30:38.915252] [INFO ] [MAIN ] DSL: Fetched primary data is valid. [2025-11-04T18:30:38.915354] [DEBUG] [LOCAL ] DSL: Saving primary data. [2025-11-04T18:30:38.916185] [DEBUG] [LOCAL ] DSL: Trimmed 42376 records to 28826 (keeping 200 days of data) [2025-11-04T18:30:39.081550] [DEBUG] [LOCAL ] DSL: Primary data saved (28826 records). [2025-11-04T18:30:39.081701] [INFO ] [MAIN ] DSL: Full history saved. [2025-11-04T18:30:39.081782] [INFO ] [MAIN ] Update Progress: 653/2367 symbols completed. DSL took 41.8s. Estimated remaining: 3h 57m 48s. Rate: 7.2 symbols/min [2025-11-04T18:30:39.271359] [INFO ] [MAIN ] DSX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:30:39.292459] [DEBUG] [LOCAL ] DSX: Loaded 22929 existing records from primary storage. [2025-11-04T18:30:39.292705] [INFO ] [MAIN ] DSX: Loaded 22929 existing records. [2025-11-04T18:30:39.372266] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-07. [2025-11-04T18:30:39.469745] [WARN ] [MAIN ] DSX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:30:39.469980] [INFO ] [MAIN ] DSX: Wiping existing primary data before full refresh. [2025-11-04T18:30:39.470028] [INFO ] [LOCAL ] DSX: Starting deletion of primary data. [2025-11-04T18:30:39.470203] [DEBUG] [LOCAL ] DSX: Deleted primary data file. [2025-11-04T18:30:39.470244] [INFO ] [MAIN ] DSX: Starting full primary refresh. [2025-11-04T18:30:39.720512] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-11-04T18:30:39.829853] [INFO ] [REMOTE ] DSX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:30:40.167070] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-11. [2025-11-04T18:30:40.733481] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T18:30:40.734480] [INFO ] [REMOTE ] DSM: Full fetch complete (24315 records). [2025-11-04T18:30:40.734548] [DEBUG] [VALIDATOR ] DSM: Validating primary data. [2025-11-04T18:30:40.972374] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-08. [2025-11-04T18:30:41.102982] [DEBUG] [VALIDATOR ] DSM: All expected trading days are present. [2025-11-04T18:30:41.103054] [DEBUG] [VALIDATOR ] DSM: Primary data validation passed. [2025-11-04T18:30:41.103485] [INFO ] [MAIN ] DSM: Fetched primary data is valid. [2025-11-04T18:30:41.103523] [DEBUG] [LOCAL ] DSM: Saving primary data. [2025-11-04T18:30:41.104115] [DEBUG] [LOCAL ] DSM: Trimmed 24315 records to 13618 (keeping 200 days of data) [2025-11-04T18:30:41.178922] [DEBUG] [LOCAL ] DSM: Primary data saved (13618 records). [2025-11-04T18:30:41.179043] [INFO ] [MAIN ] DSM: Full history saved. [2025-11-04T18:30:41.179117] [INFO ] [MAIN ] Update Progress: 654/2367 symbols completed. DSM took 35.8s. Estimated remaining: 3h 57m 23s. Rate: 7.2 symbols/min [2025-11-04T18:30:41.351669] [INFO ] [MAIN ] DT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:30:41.348750] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-11-04T18:30:41.351865] [DEBUG] [LOCAL ] DT: No existing data file found in primary storage. [2025-11-04T18:30:41.351933] [WARN ] [MAIN ] DT: No existing data found, performing full refresh instead. [2025-11-04T18:30:41.352068] [INFO ] [MAIN ] DT: Wiping existing primary data before full refresh. [2025-11-04T18:30:41.353304] [INFO ] [LOCAL ] DT: Starting deletion of primary data. [2025-11-04T18:30:41.353362] [DEBUG] [LOCAL ] DT: No primary data file found to delete. [2025-11-04T18:30:41.353396] [INFO ] [MAIN ] DT: Starting full primary refresh. [2025-11-04T18:30:41.701918] [INFO ] [REMOTE ] DT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:30:41.766998] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-01. [2025-11-04T18:30:42.567090] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-01. [2025-11-04T18:30:43.367062] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-09. [2025-11-04T18:30:43.419528] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-11-04T18:30:43.571394] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T18:30:43.691419] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-11-04T18:30:44.167049] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-09. [2025-11-04T18:30:44.967050] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-02. [2025-11-04T18:30:44.995612] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-11-04T18:30:45.767046] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-02. [2025-11-04T18:30:46.195020] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T18:30:46.567016] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-03. [2025-11-04T18:30:46.703737] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-11-04T18:30:47.367023] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-03. [2025-11-04T18:30:47.837111] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T18:30:48.167001] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-10. [2025-11-04T18:30:48.335866] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-11-04T18:30:48.569589] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-11-04T18:30:48.967009] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-11. [2025-11-04T18:30:49.328429] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T18:30:49.329231] [INFO ] [REMOTE ] DSMC: Full fetch complete (2926 records). [2025-11-04T18:30:49.329307] [DEBUG] [VALIDATOR ] DSMC: Validating primary data. [2025-11-04T18:30:49.689555] [DEBUG] [VALIDATOR ] DSMC: All expected trading days are present. [2025-11-04T18:30:49.689627] [DEBUG] [VALIDATOR ] DSMC: Primary data validation passed. [2025-11-04T18:30:49.689763] [INFO ] [MAIN ] DSMC: Fetched primary data is valid. [2025-11-04T18:30:49.689810] [DEBUG] [LOCAL ] DSMC: Saving primary data. [2025-11-04T18:30:49.690125] [DEBUG] [LOCAL ] DSMC: Trimmed 2926 records to 1739 (keeping 200 days of data) [2025-11-04T18:30:49.700640] [DEBUG] [LOCAL ] DSMC: Primary data saved (1739 records). [2025-11-04T18:30:49.700723] [INFO ] [MAIN ] DSMC: Full history saved. [2025-11-04T18:30:49.700787] [INFO ] [MAIN ] Update Progress: 655/2367 symbols completed. DSMC took 36.5s. Estimated remaining: 3h 57m 15s. Rate: 7.2 symbols/min [2025-11-04T18:30:49.772264] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-04. [2025-11-04T18:30:49.867341] [INFO ] [MAIN ] DTB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:30:49.874288] [DEBUG] [LOCAL ] DTB: Loaded 4815 existing records from primary storage. [2025-11-04T18:30:49.874446] [INFO ] [MAIN ] DTB: Loaded 4815 existing records. [2025-11-04T18:30:50.046420] [WARN ] [MAIN ] DTB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:30:50.046635] [INFO ] [MAIN ] DTB: Wiping existing primary data before full refresh. [2025-11-04T18:30:50.046715] [INFO ] [LOCAL ] DTB: Starting deletion of primary data. [2025-11-04T18:30:50.046988] [DEBUG] [LOCAL ] DTB: Deleted primary data file. [2025-11-04T18:30:50.047044] [INFO ] [MAIN ] DTB: Starting full primary refresh. [2025-11-04T18:30:50.447337] [INFO ] [REMOTE ] DTB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:30:50.567063] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-10. [2025-11-04T18:30:51.180846] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-11-04T18:30:51.375804] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-04. [2025-11-04T18:30:51.381016] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-11-04T18:30:52.167061] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-11. [2025-11-04T18:30:52.195990] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T18:30:52.850521] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T18:30:52.851628] [INFO ] [REMOTE ] DSU: Full fetch complete (32490 records). [2025-11-04T18:30:52.851742] [DEBUG] [VALIDATOR ] DSU: Validating primary data. [2025-11-04T18:30:52.972221] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-01. [2025-11-04T18:30:53.239042] [DEBUG] [VALIDATOR ] DSU: All expected trading days are present. [2025-11-04T18:30:53.239117] [DEBUG] [VALIDATOR ] DSU: Primary data validation passed. [2025-11-04T18:30:53.239588] [INFO ] [MAIN ] DSU: Fetched primary data is valid. [2025-11-04T18:30:53.239627] [DEBUG] [LOCAL ] DSU: Saving primary data. [2025-11-04T18:30:53.240383] [DEBUG] [LOCAL ] DSU: Trimmed 32490 records to 22325 (keeping 200 days of data) [2025-11-04T18:30:53.368368] [DEBUG] [LOCAL ] DSU: Primary data saved (22325 records). [2025-11-04T18:30:53.368456] [INFO ] [MAIN ] DSU: Full history saved. [2025-11-04T18:30:53.368533] [INFO ] [MAIN ] Update Progress: 656/2367 symbols completed. DSU took 34.7s. Estimated remaining: 3h 56m 55s. Rate: 7.2 symbols/min [2025-11-04T18:30:53.473399] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-11-04T18:30:53.541153] [INFO ] [MAIN ] DTE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:30:53.583466] [DEBUG] [LOCAL ] DTE: Loaded 46907 existing records from primary storage. [2025-11-04T18:30:53.583791] [INFO ] [MAIN ] DTE: Loaded 46907 existing records. [2025-11-04T18:30:53.768795] [WARN ] [MAIN ] DTE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:30:53.768877] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-02. [2025-11-04T18:30:53.769953] [INFO ] [MAIN ] DTE: Wiping existing primary data before full refresh. [2025-11-04T18:30:53.770028] [INFO ] [LOCAL ] DTE: Starting deletion of primary data. [2025-11-04T18:30:53.770260] [DEBUG] [LOCAL ] DTE: Deleted primary data file. [2025-11-04T18:30:53.770316] [INFO ] [MAIN ] DTE: Starting full primary refresh. [2025-11-04T18:30:54.124515] [INFO ] [REMOTE ] DTE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:30:54.226838] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-11-04T18:30:54.567075] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-01. [2025-11-04T18:30:55.367040] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-05. [2025-11-04T18:30:55.918759] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T18:30:56.167081] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-05. [2025-11-04T18:30:56.420875] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T18:30:56.967030] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-03. [2025-11-04T18:30:57.323256] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-11-04T18:30:57.670678] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-11-04T18:30:57.767081] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-06. [2025-11-04T18:30:58.530210] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-11-04T18:30:58.567022] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-07. [2025-11-04T18:30:59.367062] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-06. [2025-11-04T18:30:59.581730] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T18:31:00.167047] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-04. [2025-11-04T18:31:00.583662] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-11-04T18:31:00.913665] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-11-04T18:31:00.967037] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-07. [2025-11-04T18:31:01.767073] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-02. [2025-11-04T18:31:02.391817] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-11-04T18:31:02.567059] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-08. [2025-11-04T18:31:03.055900] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T18:31:03.313955] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T18:31:03.367038] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-09. [2025-11-04T18:31:04.053539] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-11-04T18:31:04.167036] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-08. [2025-11-04T18:31:04.967077] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-05. [2025-11-04T18:31:05.434356] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-11-04T18:31:05.506264] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T18:31:05.767078] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-09. [2025-11-04T18:31:06.567058] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-06. [2025-11-04T18:31:06.994826] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-11-04T18:31:07.206785] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T18:31:07.367067] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-07. [2025-11-04T18:31:07.719445] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-11-04T18:31:08.167111] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-03. [2025-11-04T18:31:08.967089] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-08. [2025-11-04T18:31:09.361034] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-11-04T18:31:09.518569] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-11-04T18:31:09.767041] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-04. [2025-11-04T18:31:10.567095] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-09. [2025-11-04T18:31:11.060892] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-11-04T18:31:11.091439] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T18:31:11.367037] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-10. [2025-11-04T18:31:11.739941] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-11-04T18:31:12.167055] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-11. [2025-11-04T18:31:12.539738] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-11 (0.37 seconds). [2025-11-04T18:31:12.540462] [INFO ] [REMOTE ] DTB: Full fetch complete (6413 records). [2025-11-04T18:31:12.540563] [DEBUG] [VALIDATOR ] DTB: Validating primary data. [2025-11-04T18:31:12.917380] [DEBUG] [VALIDATOR ] DTB: All expected trading days are present. [2025-11-04T18:31:12.917496] [DEBUG] [VALIDATOR ] DTB: Primary data validation passed. [2025-11-04T18:31:12.917704] [INFO ] [MAIN ] DTB: Fetched primary data is valid. [2025-11-04T18:31:12.917766] [DEBUG] [LOCAL ] DTB: Saving primary data. [2025-11-04T18:31:12.918187] [DEBUG] [LOCAL ] DTB: Trimmed 6413 records to 4821 (keeping 200 days of data) [2025-11-04T18:31:12.947675] [DEBUG] [LOCAL ] DTB: Primary data saved (4821 records). [2025-11-04T18:31:12.947819] [INFO ] [MAIN ] DTB: Full history saved. [2025-11-04T18:31:12.947913] [INFO ] [MAIN ] Update Progress: 657/2367 symbols completed. DTB took 23.2s. Estimated remaining: 3h 57m 16s. Rate: 7.2 symbols/min [2025-11-04T18:31:12.977507] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-05. [2025-11-04T18:31:13.123897] [INFO ] [MAIN ] DTF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:31:13.124085] [DEBUG] [LOCAL ] DTF: No existing data file found in primary storage. [2025-11-04T18:31:13.124123] [WARN ] [MAIN ] DTF: No existing data found, performing full refresh instead. [2025-11-04T18:31:13.124229] [INFO ] [MAIN ] DTF: Wiping existing primary data before full refresh. [2025-11-04T18:31:13.124270] [INFO ] [LOCAL ] DTF: Starting deletion of primary data. [2025-11-04T18:31:13.124310] [DEBUG] [LOCAL ] DTF: No primary data file found to delete. [2025-11-04T18:31:13.124342] [INFO ] [MAIN ] DTF: Starting full primary refresh. [2025-11-04T18:31:13.491833] [INFO ] [REMOTE ] DTF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:31:13.767039] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-10. [2025-11-04T18:31:14.491180] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-11-04T18:31:14.567024] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-10. [2025-11-04T18:31:14.637916] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-11-04T18:31:15.367053] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-11. [2025-11-04T18:31:16.092973] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-11-04T18:31:16.108359] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T18:31:16.109802] [INFO ] [REMOTE ] DSX: Full fetch complete (34736 records). [2025-11-04T18:31:16.109911] [DEBUG] [VALIDATOR ] DSX: Validating primary data. [2025-11-04T18:31:16.175754] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-11. [2025-11-04T18:31:16.486936] [DEBUG] [VALIDATOR ] DSX: All expected trading days are present. [2025-11-04T18:31:16.487019] [DEBUG] [VALIDATOR ] DSX: Primary data validation passed. [2025-11-04T18:31:16.487543] [INFO ] [MAIN ] DSX: Fetched primary data is valid. [2025-11-04T18:31:16.487583] [DEBUG] [LOCAL ] DSX: Saving primary data. [2025-11-04T18:31:16.488411] [DEBUG] [LOCAL ] DSX: Trimmed 34736 records to 22834 (keeping 200 days of data) [2025-11-04T18:31:16.612864] [DEBUG] [LOCAL ] DSX: Primary data saved (22834 records). [2025-11-04T18:31:16.612987] [INFO ] [MAIN ] DSX: Full history saved. [2025-11-04T18:31:16.613061] [INFO ] [MAIN ] Update Progress: 658/2367 symbols completed. DSX took 37.5s. Estimated remaining: 3h 56m 56s. Rate: 7.2 symbols/min [2025-11-04T18:31:16.778265] [INFO ] [MAIN ] DTG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:31:16.785762] [DEBUG] [LOCAL ] DTG: Loaded 5537 existing records from primary storage. [2025-11-04T18:31:16.785953] [INFO ] [MAIN ] DTG: Loaded 5537 existing records. [2025-11-04T18:31:16.961727] [WARN ] [MAIN ] DTG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:31:16.961998] [INFO ] [MAIN ] DTG: Wiping existing primary data before full refresh. [2025-11-04T18:31:16.962052] [INFO ] [LOCAL ] DTG: Starting deletion of primary data. [2025-11-04T18:31:16.962206] [DEBUG] [LOCAL ] DTG: Deleted primary data file. [2025-11-04T18:31:16.962260] [INFO ] [MAIN ] DTG: Starting full primary refresh. [2025-11-04T18:31:16.973733] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-01. [2025-11-04T18:31:17.322127] [INFO ] [REMOTE ] DTG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:31:17.352040] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T18:31:17.353991] [INFO ] [REMOTE ] DT: Full fetch complete (76874 records). [2025-11-04T18:31:17.354084] [DEBUG] [VALIDATOR ] DT: Validating primary data. [2025-11-04T18:31:17.509004] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-11-04T18:31:17.779948] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-02. [2025-11-04T18:31:17.785174] [DEBUG] [VALIDATOR ] DT: All expected trading days are present. [2025-11-04T18:31:17.785264] [DEBUG] [VALIDATOR ] DT: Primary data validation passed. [2025-11-04T18:31:17.786294] [INFO ] [MAIN ] DT: Fetched primary data is valid. [2025-11-04T18:31:17.786471] [DEBUG] [LOCAL ] DT: Saving primary data. [2025-11-04T18:31:17.788037] [DEBUG] [LOCAL ] DT: Trimmed 76874 records to 52907 (keeping 200 days of data) [2025-11-04T18:31:18.107079] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-11-04T18:31:18.125823] [DEBUG] [LOCAL ] DT: Primary data saved (52907 records). [2025-11-04T18:31:18.125960] [INFO ] [MAIN ] DT: Full history saved. [2025-11-04T18:31:18.126033] [INFO ] [MAIN ] Update Progress: 659/2367 symbols completed. DT took 36.9s. Estimated remaining: 3h 56m 30s. Rate: 7.2 symbols/min [2025-11-04T18:31:18.294268] [INFO ] [MAIN ] DTK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:31:18.294456] [DEBUG] [LOCAL ] DTK: No existing data file found in primary storage. [2025-11-04T18:31:18.294503] [WARN ] [MAIN ] DTK: No existing data found, performing full refresh instead. [2025-11-04T18:31:18.294623] [INFO ] [MAIN ] DTK: Wiping existing primary data before full refresh. [2025-11-04T18:31:18.294701] [INFO ] [LOCAL ] DTK: Starting deletion of primary data. [2025-11-04T18:31:18.294754] [DEBUG] [LOCAL ] DTK: No primary data file found to delete. [2025-11-04T18:31:18.294786] [INFO ] [MAIN ] DTK: Starting full primary refresh. [2025-11-04T18:31:18.572211] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-01. [2025-11-04T18:31:18.641703] [INFO ] [REMOTE ] DTK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:31:19.098782] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T18:31:19.367040] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-06. [2025-11-04T18:31:20.167041] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-02. [2025-11-04T18:31:20.580243] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-11-04T18:31:20.619408] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-11-04T18:31:20.967055] [INFO ] [REMOTE ] DTK: Fetching primary data for 2025-01. [2025-11-04T18:31:21.204867] [DEBUG] [REMOTE ] DTK: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:31:21.205201] [WARN ] [REMOTE ] DTK: No data returned for month 2025-01 [2025-11-04T18:31:21.205287] [FATAL] [REMOTE ] DTK: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:31:21.205366] [FATAL] [MAIN ] DTK: Failed to fetch primary data. [2025-11-04T18:31:21.205428] [WARN ] [MAIN ] DTK: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:31:21.205495] [INFO ] [LOCAL ] DTK: Starting deletion of primary data. [2025-11-04T18:31:21.205592] [DEBUG] [LOCAL ] DTK: No primary data file found to delete. [2025-11-04T18:31:21.205635] [INFO ] [LOCAL ] DTK: Starting deletion of derivative data. [2025-11-04T18:31:21.205719] [DEBUG] [LOCAL ] DTK: No derivative data file found to delete. [2025-11-04T18:31:21.205771] [INFO ] [MAIN ] Update Progress: 660/2367 symbols completed. DTK took 3.1s. Estimated remaining: 3h 56m 8s. Rate: 7.2 symbols/min [2025-11-04T18:31:21.382461] [INFO ] [MAIN ] DTM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:31:21.382680] [DEBUG] [LOCAL ] DTM: No existing data file found in primary storage. [2025-11-04T18:31:21.382735] [WARN ] [MAIN ] DTM: No existing data found, performing full refresh instead. [2025-11-04T18:31:21.382861] [INFO ] [MAIN ] DTM: Wiping existing primary data before full refresh. [2025-11-04T18:31:21.382921] [INFO ] [LOCAL ] DTM: Starting deletion of primary data. [2025-11-04T18:31:21.383007] [DEBUG] [LOCAL ] DTM: No primary data file found to delete. [2025-11-04T18:31:21.383057] [INFO ] [MAIN ] DTM: Starting full primary refresh. [2025-11-04T18:31:21.776564] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-03. [2025-11-04T18:31:21.783633] [INFO ] [REMOTE ] DTM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:31:22.065791] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-11-04T18:31:22.567046] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-01. [2025-11-04T18:31:23.367034] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-07. [2025-11-04T18:31:23.911502] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T18:31:24.167122] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-04. [2025-11-04T18:31:24.474721] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-11-04T18:31:24.798580] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-11-04T18:31:24.967067] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-03. [2025-11-04T18:31:25.344258] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-11-04T18:31:25.767066] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-02. [2025-11-04T18:31:26.567079] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-04. [2025-11-04T18:31:26.836797] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T18:31:27.055015] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-11-04T18:31:27.367042] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-05. [2025-11-04T18:31:27.746195] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-11-04T18:31:28.167062] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-03. [2025-11-04T18:31:28.967062] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-08. [2025-11-04T18:31:29.656942] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-11-04T18:31:29.767045] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-04. [2025-11-04T18:31:30.289288] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-11-04T18:31:30.567055] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-05. [2025-11-04T18:31:31.047509] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T18:31:31.066782] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-11-04T18:31:31.367030] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-06. [2025-11-04T18:31:31.733008] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-11-04T18:31:32.167053] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-07. [2025-11-04T18:31:32.636170] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-11-04T18:31:32.967054] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-05. [2025-11-04T18:31:33.767093] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-09. [2025-11-04T18:31:34.269432] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T18:31:34.567049] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-08. [2025-11-04T18:31:35.038175] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-11-04T18:31:35.212515] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-11-04T18:31:35.367033] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-10. [2025-11-04T18:31:36.167085] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-06. [2025-11-04T18:31:36.746493] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T18:31:36.967047] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-09. [2025-11-04T18:31:37.147854] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T18:31:37.349681] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-11-04T18:31:37.767099] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-11. [2025-11-04T18:31:38.268775] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T18:31:38.271601] [INFO ] [REMOTE ] DTE: Full fetch complete (68926 records). [2025-11-04T18:31:38.271828] [DEBUG] [VALIDATOR ] DTE: Validating primary data. [2025-11-04T18:31:38.572379] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-10. [2025-11-04T18:31:38.690474] [DEBUG] [VALIDATOR ] DTE: All expected trading days are present. [2025-11-04T18:31:38.690558] [DEBUG] [VALIDATOR ] DTE: Primary data validation passed. [2025-11-04T18:31:38.691465] [INFO ] [MAIN ] DTE: Fetched primary data is valid. [2025-11-04T18:31:38.691529] [DEBUG] [LOCAL ] DTE: Saving primary data. [2025-11-04T18:31:38.692957] [DEBUG] [LOCAL ] DTE: Trimmed 68926 records to 46921 (keeping 200 days of data) [2025-11-04T18:31:38.993854] [DEBUG] [LOCAL ] DTE: Primary data saved (46921 records). [2025-11-04T18:31:38.994044] [INFO ] [MAIN ] DTE: Full history saved. [2025-11-04T18:31:38.994157] [INFO ] [MAIN ] Update Progress: 661/2367 symbols completed. DTE took 45.6s. Estimated remaining: 3h 56m 24s. Rate: 7.2 symbols/min [2025-11-04T18:31:39.128287] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-11-04T18:31:39.214190] [INFO ] [MAIN ] DTW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:31:39.221345] [DEBUG] [LOCAL ] DTW: Loaded 4549 existing records from primary storage. [2025-11-04T18:31:39.221550] [INFO ] [MAIN ] DTW: Loaded 4549 existing records. [2025-11-04T18:31:39.374192] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-11. [2025-11-04T18:31:39.422288] [WARN ] [MAIN ] DTW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:31:39.422505] [INFO ] [MAIN ] DTW: Wiping existing primary data before full refresh. [2025-11-04T18:31:39.422555] [INFO ] [LOCAL ] DTW: Starting deletion of primary data. [2025-11-04T18:31:39.422710] [DEBUG] [LOCAL ] DTW: Deleted primary data file. [2025-11-04T18:31:39.422766] [INFO ] [MAIN ] DTW: Starting full primary refresh. [2025-11-04T18:31:39.801728] [INFO ] [REMOTE ] DTW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:31:39.801870] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T18:31:39.803258] [INFO ] [REMOTE ] DTG: Full fetch complete (8296 records). [2025-11-04T18:31:39.803336] [DEBUG] [VALIDATOR ] DTG: Validating primary data. [2025-11-04T18:31:39.803805] [FATAL] [VALIDATOR ] DTG: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:31:39.803882] [FATAL] [MAIN ] DTG: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:31:39.803952] [WARN ] [MAIN ] DTG: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:31:39.804020] [INFO ] [LOCAL ] DTG: Starting deletion of primary data. [2025-11-04T18:31:39.804083] [DEBUG] [LOCAL ] DTG: No primary data file found to delete. [2025-11-04T18:31:39.804117] [INFO ] [LOCAL ] DTG: Starting deletion of derivative data. [2025-11-04T18:31:39.804281] [DEBUG] [LOCAL ] DTG: Deleted derivative data file. [2025-11-04T18:31:39.804367] [INFO ] [MAIN ] Update Progress: 662/2367 symbols completed. DTG took 23.2s. Estimated remaining: 3h 55m 56s. Rate: 7.2 symbols/min [2025-11-04T18:31:39.978632] [INFO ] [MAIN ] DUK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:31:40.030535] [DEBUG] [LOCAL ] DUK: Loaded 53416 existing records from primary storage. [2025-11-04T18:31:40.030890] [INFO ] [MAIN ] DUK: Loaded 53416 existing records. [2025-11-04T18:31:40.182124] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-06. [2025-11-04T18:31:40.222307] [WARN ] [MAIN ] DUK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:31:40.222514] [INFO ] [MAIN ] DUK: Wiping existing primary data before full refresh. [2025-11-04T18:31:40.222577] [INFO ] [LOCAL ] DUK: Starting deletion of primary data. [2025-11-04T18:31:40.222865] [DEBUG] [LOCAL ] DUK: Deleted primary data file. [2025-11-04T18:31:40.222922] [INFO ] [MAIN ] DUK: Starting full primary refresh. [2025-11-04T18:31:40.610521] [INFO ] [REMOTE ] DUK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:31:40.612895] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-11-04T18:31:40.967031] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-07. [2025-11-04T18:31:41.245704] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-07 (0.28 seconds). [2025-11-04T18:31:41.767013] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-08. [2025-11-04T18:31:42.070224] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-08 (0.30 seconds). [2025-11-04T18:31:42.567036] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-01. [2025-11-04T18:31:43.105497] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-11-04T18:31:43.367021] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-02. [2025-11-04T18:31:43.769205] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-11-04T18:31:44.167079] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-01. [2025-11-04T18:31:44.967084] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-03. [2025-11-04T18:31:45.389000] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T18:31:45.677937] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-11-04T18:31:45.767068] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-07. [2025-11-04T18:31:46.567063] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-04. [2025-11-04T18:31:46.623329] [DEBUG] [REMOTE ] DTW: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-11-04T18:31:46.623667] [WARN ] [REMOTE ] DTW: No data returned for month 2025-04 [2025-11-04T18:31:46.623773] [FATAL] [REMOTE ] DTW: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:31:46.623884] [FATAL] [MAIN ] DTW: Failed to fetch primary data. [2025-11-04T18:31:46.623967] [WARN ] [MAIN ] DTW: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:31:46.624052] [INFO ] [LOCAL ] DTW: Starting deletion of primary data. [2025-11-04T18:31:46.624127] [DEBUG] [LOCAL ] DTW: No primary data file found to delete. [2025-11-04T18:31:46.624165] [INFO ] [LOCAL ] DTW: Starting deletion of derivative data. [2025-11-04T18:31:46.624290] [DEBUG] [LOCAL ] DTW: Deleted derivative data file. [2025-11-04T18:31:46.624373] [INFO ] [MAIN ] Update Progress: 663/2367 symbols completed. DTW took 7.6s. Estimated remaining: 3h 55m 44s. Rate: 7.2 symbols/min [2025-11-04T18:31:46.801269] [INFO ] [MAIN ] DUKB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:31:46.809807] [DEBUG] [LOCAL ] DUKB: Loaded 5898 existing records from primary storage. [2025-11-04T18:31:46.810223] [INFO ] [MAIN ] DUKB: Loaded 5898 existing records. [2025-11-04T18:31:46.995473] [WARN ] [MAIN ] DUKB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:31:46.995759] [INFO ] [MAIN ] DUKB: Wiping existing primary data before full refresh. [2025-11-04T18:31:46.995860] [INFO ] [LOCAL ] DUKB: Starting deletion of primary data. [2025-11-04T18:31:46.996033] [DEBUG] [LOCAL ] DUKB: Deleted primary data file. [2025-11-04T18:31:46.996132] [INFO ] [MAIN ] DUKB: Starting full primary refresh. [2025-11-04T18:31:47.347353] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-11-04T18:31:47.390375] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-02. [2025-11-04T18:31:47.396459] [INFO ] [REMOTE ] DUKB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:31:48.167088] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-08. [2025-11-04T18:31:48.935429] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-11-04T18:31:48.967007] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-03. [2025-11-04T18:31:49.357443] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T18:31:49.767043] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-01. [2025-11-04T18:31:50.382487] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T18:31:50.398867] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-11-04T18:31:50.567037] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-04. [2025-11-04T18:31:51.367043] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-09. [2025-11-04T18:31:51.785343] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-11-04T18:31:51.883242] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T18:31:52.167116] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-10. [2025-11-04T18:31:52.521251] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-10 (0.35 seconds). [2025-11-04T18:31:52.967062] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-11. [2025-11-04T18:31:53.527546] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-11 (0.56 seconds). [2025-11-04T18:31:53.528278] [INFO ] [REMOTE ] DTF: Full fetch complete (1712 records). [2025-11-04T18:31:53.528348] [DEBUG] [VALIDATOR ] DTF: Validating primary data. [2025-11-04T18:31:53.711872] [WARN ] [VALIDATOR ] DTF: Validation failed - insufficient historical data. Have 197 unique days, need 200 days [2025-11-04T18:31:53.712086] [FATAL] [MAIN ] DTF: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:31:53.712153] [WARN ] [MAIN ] DTF: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:31:53.712195] [INFO ] [LOCAL ] DTF: Starting deletion of primary data. [2025-11-04T18:31:53.712247] [DEBUG] [LOCAL ] DTF: No primary data file found to delete. [2025-11-04T18:31:53.712282] [INFO ] [LOCAL ] DTF: Starting deletion of derivative data. [2025-11-04T18:31:53.712320] [DEBUG] [LOCAL ] DTF: No derivative data file found to delete. [2025-11-04T18:31:53.712374] [INFO ] [MAIN ] Update Progress: 664/2367 symbols completed. DTF took 40.8s. Estimated remaining: 3h 55m 33s. Rate: 7.2 symbols/min [2025-11-04T18:31:53.776498] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-09. [2025-11-04T18:31:53.890000] [INFO ] [MAIN ] DV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:31:53.890179] [DEBUG] [LOCAL ] DV: No existing data file found in primary storage. [2025-11-04T18:31:53.890216] [WARN ] [MAIN ] DV: No existing data found, performing full refresh instead. [2025-11-04T18:31:53.890319] [INFO ] [MAIN ] DV: Wiping existing primary data before full refresh. [2025-11-04T18:31:53.890359] [INFO ] [LOCAL ] DV: Starting deletion of primary data. [2025-11-04T18:31:53.890444] [DEBUG] [LOCAL ] DV: No primary data file found to delete. [2025-11-04T18:31:53.890482] [INFO ] [MAIN ] DV: Starting full primary refresh. [2025-11-04T18:31:54.258000] [INFO ] [REMOTE ] DV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:31:54.567055] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-02. [2025-11-04T18:31:54.927915] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-11-04T18:31:54.960510] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:31:55.367069] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-03. [2025-11-04T18:31:55.874193] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-11-04T18:31:56.167094] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-01. [2025-11-04T18:31:56.967047] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-05. [2025-11-04T18:31:57.722109] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-11-04T18:31:57.767024] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-02. [2025-11-04T18:31:58.567051] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-04. [2025-11-04T18:31:58.664763] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-11-04T18:31:58.756822] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T18:31:59.059444] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-11-04T18:31:59.367060] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-03. [2025-11-04T18:32:00.167051] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-05. [2025-11-04T18:32:00.808666] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T18:32:00.849328] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-11-04T18:32:00.967035] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-10. [2025-11-04T18:32:01.767049] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-06. [2025-11-04T18:32:02.411924] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T18:32:02.567076] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-06. [2025-11-04T18:32:03.036964] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-11-04T18:32:03.216687] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T18:32:03.367063] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-11. [2025-11-04T18:32:04.167051] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-07. [2025-11-04T18:32:04.512574] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T18:32:04.515107] [INFO ] [REMOTE ] DTM: Full fetch complete (65327 records). [2025-11-04T18:32:04.515237] [DEBUG] [VALIDATOR ] DTM: Validating primary data. [2025-11-04T18:32:04.644342] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-11-04T18:32:04.915584] [DEBUG] [VALIDATOR ] DTM: All expected trading days are present. [2025-11-04T18:32:04.915690] [DEBUG] [VALIDATOR ] DTM: Primary data validation passed. [2025-11-04T18:32:04.916467] [INFO ] [MAIN ] DTM: Fetched primary data is valid. [2025-11-04T18:32:04.916509] [DEBUG] [LOCAL ] DTM: Saving primary data. [2025-11-04T18:32:04.917789] [DEBUG] [LOCAL ] DTM: Trimmed 65327 records to 43518 (keeping 200 days of data) [2025-11-04T18:32:04.970827] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-07. [2025-11-04T18:32:05.184343] [DEBUG] [LOCAL ] DTM: Primary data saved (43518 records). [2025-11-04T18:32:05.184468] [INFO ] [MAIN ] DTM: Full history saved. [2025-11-04T18:32:05.184537] [INFO ] [MAIN ] Update Progress: 665/2367 symbols completed. DTM took 44.0s. Estimated remaining: 3h 55m 33s. Rate: 7.2 symbols/min [2025-11-04T18:32:05.349223] [INFO ] [MAIN ] DVA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:32:05.349404] [DEBUG] [LOCAL ] DVA: No existing data file found in primary storage. [2025-11-04T18:32:05.349440] [WARN ] [MAIN ] DVA: No existing data found, performing full refresh instead. [2025-11-04T18:32:05.349547] [INFO ] [MAIN ] DVA: Wiping existing primary data before full refresh. [2025-11-04T18:32:05.349587] [INFO ] [LOCAL ] DVA: Starting deletion of primary data. [2025-11-04T18:32:05.349629] [DEBUG] [LOCAL ] DVA: No primary data file found to delete. [2025-11-04T18:32:05.349697] [INFO ] [MAIN ] DVA: Starting full primary refresh. [2025-11-04T18:32:05.701942] [INFO ] [REMOTE ] DVA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:32:05.767024] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-01. [2025-11-04T18:32:06.567086] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-04. [2025-11-04T18:32:06.728876] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-11-04T18:32:06.764707] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T18:32:07.367032] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-02. [2025-11-04T18:32:08.174298] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-08. [2025-11-04T18:32:08.175951] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-11-04T18:32:08.432475] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T18:32:08.553731] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-11-04T18:32:08.967017] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-05. [2025-11-04T18:32:09.767055] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-08. [2025-11-04T18:32:10.159928] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T18:32:10.567021] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-03. [2025-11-04T18:32:11.060517] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T18:32:11.367042] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-09. [2025-11-04T18:32:11.846633] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T18:32:12.167041] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-04. [2025-11-04T18:32:12.667089] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T18:32:12.967037] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-06. [2025-11-04T18:32:13.603558] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T18:32:13.767034] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-10. [2025-11-04T18:32:14.087263] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T18:32:14.567034] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-05. [2025-11-04T18:32:15.367005] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-09. [2025-11-04T18:32:15.561422] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-11-04T18:32:15.695716] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T18:32:15.870285] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-11-04T18:32:16.167050] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-07. [2025-11-04T18:32:16.967051] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-06. [2025-11-04T18:32:17.417374] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:32:17.767042] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-10. [2025-11-04T18:32:18.148916] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-11-04T18:32:18.196386] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-11-04T18:32:18.567054] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-11. [2025-11-04T18:32:18.947893] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T18:32:18.948687] [INFO ] [REMOTE ] DUKB: Full fetch complete (8869 records). [2025-11-04T18:32:18.948756] [DEBUG] [VALIDATOR ] DUKB: Validating primary data. [2025-11-04T18:32:19.348927] [DEBUG] [VALIDATOR ] DUKB: All expected trading days are present. [2025-11-04T18:32:19.349017] [DEBUG] [VALIDATOR ] DUKB: Primary data validation passed. [2025-11-04T18:32:19.349257] [INFO ] [MAIN ] DUKB: Fetched primary data is valid. [2025-11-04T18:32:19.349296] [DEBUG] [LOCAL ] DUKB: Saving primary data. [2025-11-04T18:32:19.349784] [DEBUG] [LOCAL ] DUKB: Trimmed 8869 records to 5909 (keeping 200 days of data) [2025-11-04T18:32:19.367712] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-07. [2025-11-04T18:32:19.388683] [DEBUG] [LOCAL ] DUKB: Primary data saved (5909 records). [2025-11-04T18:32:19.388973] [INFO ] [MAIN ] DUKB: Full history saved. [2025-11-04T18:32:19.389091] [INFO ] [MAIN ] Update Progress: 666/2367 symbols completed. DUKB took 32.8s. Estimated remaining: 3h 55m 39s. Rate: 7.2 symbols/min [2025-11-04T18:32:19.580686] [INFO ] [MAIN ] DVN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:32:19.632581] [DEBUG] [LOCAL ] DVN: Loaded 53699 existing records from primary storage. [2025-11-04T18:32:19.632945] [INFO ] [MAIN ] DVN: Loaded 53699 existing records. [2025-11-04T18:32:19.813906] [WARN ] [MAIN ] DVN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:32:19.814147] [INFO ] [MAIN ] DVN: Wiping existing primary data before full refresh. [2025-11-04T18:32:19.814195] [INFO ] [LOCAL ] DVN: Starting deletion of primary data. [2025-11-04T18:32:19.814378] [DEBUG] [LOCAL ] DVN: Deleted primary data file. [2025-11-04T18:32:19.814417] [INFO ] [MAIN ] DVN: Starting full primary refresh. [2025-11-04T18:32:20.169102] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-11. [2025-11-04T18:32:20.172752] [INFO ] [REMOTE ] DVN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:32:20.551711] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T18:32:20.636951] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T18:32:20.638975] [INFO ] [REMOTE ] DUK: Full fetch complete (77554 records). [2025-11-04T18:32:20.639123] [DEBUG] [VALIDATOR ] DUK: Validating primary data. [2025-11-04T18:32:20.972221] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-08. [2025-11-04T18:32:21.028795] [DEBUG] [VALIDATOR ] DUK: All expected trading days are present. [2025-11-04T18:32:21.028862] [DEBUG] [VALIDATOR ] DUK: Primary data validation passed. [2025-11-04T18:32:21.029739] [INFO ] [MAIN ] DUK: Fetched primary data is valid. [2025-11-04T18:32:21.029807] [DEBUG] [LOCAL ] DUK: Saving primary data. [2025-11-04T18:32:21.031362] [DEBUG] [LOCAL ] DUK: Trimmed 77554 records to 53416 (keeping 200 days of data) [2025-11-04T18:32:21.355027] [DEBUG] [LOCAL ] DUK: Primary data saved (53416 records). [2025-11-04T18:32:21.355166] [INFO ] [MAIN ] DUK: Full history saved. [2025-11-04T18:32:21.355244] [INFO ] [MAIN ] Update Progress: 667/2367 symbols completed. DUK took 41.6s. Estimated remaining: 3h 55m 15s. Rate: 7.2 symbols/min [2025-11-04T18:32:21.526422] [INFO ] [MAIN ] DX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:32:21.574908] [DEBUG] [LOCAL ] DX: Loaded 52476 existing records from primary storage. [2025-11-04T18:32:21.575236] [INFO ] [MAIN ] DX: Loaded 52476 existing records. [2025-11-04T18:32:21.752570] [WARN ] [MAIN ] DX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:32:21.752800] [INFO ] [MAIN ] DX: Wiping existing primary data before full refresh. [2025-11-04T18:32:21.752848] [INFO ] [LOCAL ] DX: Starting deletion of primary data. [2025-11-04T18:32:21.753066] [DEBUG] [LOCAL ] DX: Deleted primary data file. [2025-11-04T18:32:21.753122] [INFO ] [MAIN ] DX: Starting full primary refresh. [2025-11-04T18:32:21.773603] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-08. [2025-11-04T18:32:22.115841] [INFO ] [REMOTE ] DX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:32:22.181438] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T18:32:22.567041] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-01. [2025-11-04T18:32:23.222880] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T18:32:23.367067] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-09. [2025-11-04T18:32:24.177467] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-01. [2025-11-04T18:32:24.201249] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-11-04T18:32:24.385207] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T18:32:24.967082] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-10. [2025-11-04T18:32:25.701213] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-11-04T18:32:25.767023] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-02. [2025-11-04T18:32:26.183322] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T18:32:26.567033] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-02. [2025-11-04T18:32:26.990105] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T18:32:27.367111] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-09. [2025-11-04T18:32:27.931242] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T18:32:28.167073] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-03. [2025-11-04T18:32:28.584529] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T18:32:28.967086] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-11. [2025-11-04T18:32:29.597865] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-11-04T18:32:29.767060] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-03. [2025-11-04T18:32:29.854033] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T18:32:29.855618] [INFO ] [REMOTE ] DVA: Full fetch complete (59567 records). [2025-11-04T18:32:29.855806] [DEBUG] [VALIDATOR ] DVA: Validating primary data. [2025-11-04T18:32:29.856493] [FATAL] [VALIDATOR ] DVA: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:32:29.856588] [FATAL] [MAIN ] DVA: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:32:29.856632] [WARN ] [MAIN ] DVA: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:32:29.856723] [INFO ] [LOCAL ] DVA: Starting deletion of primary data. [2025-11-04T18:32:29.856862] [DEBUG] [LOCAL ] DVA: No primary data file found to delete. [2025-11-04T18:32:29.856936] [INFO ] [LOCAL ] DVA: Starting deletion of derivative data. [2025-11-04T18:32:29.857030] [DEBUG] [LOCAL ] DVA: No derivative data file found to delete. [2025-11-04T18:32:29.857095] [INFO ] [MAIN ] Update Progress: 668/2367 symbols completed. DVA took 24.7s. Estimated remaining: 3h 55m 7s. Rate: 7.2 symbols/min [2025-11-04T18:32:30.032942] [INFO ] [MAIN ] DXC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:32:30.078133] [DEBUG] [LOCAL ] DXC: Loaded 49895 existing records from primary storage. [2025-11-04T18:32:30.078469] [INFO ] [MAIN ] DXC: Loaded 49895 existing records. [2025-11-04T18:32:30.260925] [WARN ] [MAIN ] DXC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:32:30.261142] [INFO ] [MAIN ] DXC: Wiping existing primary data before full refresh. [2025-11-04T18:32:30.261189] [INFO ] [LOCAL ] DXC: Starting deletion of primary data. [2025-11-04T18:32:30.261534] [DEBUG] [LOCAL ] DXC: Deleted primary data file. [2025-11-04T18:32:30.261590] [INFO ] [MAIN ] DXC: Starting full primary refresh. [2025-11-04T18:32:30.572190] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-04. [2025-11-04T18:32:30.625664] [INFO ] [REMOTE ] DXC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:32:31.205762] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T18:32:31.367051] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-10. [2025-11-04T18:32:32.000280] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T18:32:32.167049] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-05. [2025-11-04T18:32:32.764132] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T18:32:32.967020] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-01. [2025-11-04T18:32:33.770774] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-11. [2025-11-04T18:32:33.777579] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-11-04T18:32:34.182559] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-11-04T18:32:34.567052] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-02. [2025-11-04T18:32:35.317776] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-11 (1.55 seconds). [2025-11-04T18:32:35.319768] [INFO ] [REMOTE ] DV: Full fetch complete (73990 records). [2025-11-04T18:32:35.319887] [DEBUG] [VALIDATOR ] DV: Validating primary data. [2025-11-04T18:32:35.376678] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-04. [2025-11-04T18:32:35.731746] [DEBUG] [VALIDATOR ] DV: All expected trading days are present. [2025-11-04T18:32:35.731854] [DEBUG] [VALIDATOR ] DV: Primary data validation passed. [2025-11-04T18:32:35.732782] [INFO ] [MAIN ] DV: Fetched primary data is valid. [2025-11-04T18:32:35.732869] [DEBUG] [LOCAL ] DV: Saving primary data. [2025-11-04T18:32:35.734566] [DEBUG] [LOCAL ] DV: Trimmed 73990 records to 51219 (keeping 200 days of data) [2025-11-04T18:32:35.873560] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T18:32:36.063525] [DEBUG] [LOCAL ] DV: Primary data saved (51219 records). [2025-11-04T18:32:36.063675] [INFO ] [MAIN ] DV: Full history saved. [2025-11-04T18:32:36.063775] [INFO ] [MAIN ] Update Progress: 669/2367 symbols completed. DV took 42.4s. Estimated remaining: 3h 54m 54s. Rate: 7.2 symbols/min [2025-11-04T18:32:36.172194] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-06. [2025-11-04T18:32:36.235758] [INFO ] [MAIN ] DXYZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:32:36.235943] [DEBUG] [LOCAL ] DXYZ: No existing data file found in primary storage. [2025-11-04T18:32:36.235987] [WARN ] [MAIN ] DXYZ: No existing data found, performing full refresh instead. [2025-11-04T18:32:36.236090] [INFO ] [MAIN ] DXYZ: Wiping existing primary data before full refresh. [2025-11-04T18:32:36.236131] [INFO ] [LOCAL ] DXYZ: Starting deletion of primary data. [2025-11-04T18:32:36.236170] [DEBUG] [LOCAL ] DXYZ: No primary data file found to delete. [2025-11-04T18:32:36.236201] [INFO ] [MAIN ] DXYZ: Starting full primary refresh. [2025-11-04T18:32:36.589597] [INFO ] [REMOTE ] DXYZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:32:36.967008] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-01. [2025-11-04T18:32:37.130991] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-11-04T18:32:37.637803] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T18:32:37.767070] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-05. [2025-11-04T18:32:38.567051] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-07. [2025-11-04T18:32:38.890049] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-11-04T18:32:39.350040] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-11-04T18:32:39.367012] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-02. [2025-11-04T18:32:40.074120] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T18:32:40.167025] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-03. [2025-11-04T18:32:40.785740] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-11-04T18:32:40.967067] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-03. [2025-11-04T18:32:41.175318] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T18:32:41.767057] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-08. [2025-11-04T18:32:42.385124] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T18:32:42.567035] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-06. [2025-11-04T18:32:43.146257] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T18:32:43.367039] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-09. [2025-11-04T18:32:43.990359] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-11-04T18:32:44.167053] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-07. [2025-11-04T18:32:44.777635] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-11-04T18:32:44.967053] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-04. [2025-11-04T18:32:45.767019] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-10. [2025-11-04T18:32:45.820352] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-11-04T18:32:46.321056] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T18:32:46.567055] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-04. [2025-11-04T18:32:47.263783] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-11-04T18:32:47.367044] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-11. [2025-11-04T18:32:48.167086] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-05. [2025-11-04T18:32:48.316681] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-11-04T18:32:48.696561] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-11 (1.33 seconds). [2025-11-04T18:32:48.698638] [INFO ] [REMOTE ] DX: Full fetch complete (75990 records). [2025-11-04T18:32:48.698774] [DEBUG] [VALIDATOR ] DX: Validating primary data. [2025-11-04T18:32:48.972298] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-05. [2025-11-04T18:32:49.129689] [DEBUG] [VALIDATOR ] DX: All expected trading days are present. [2025-11-04T18:32:49.129793] [DEBUG] [VALIDATOR ] DX: Primary data validation passed. [2025-11-04T18:32:49.130627] [INFO ] [MAIN ] DX: Fetched primary data is valid. [2025-11-04T18:32:49.130704] [DEBUG] [LOCAL ] DX: Saving primary data. [2025-11-04T18:32:49.132153] [DEBUG] [LOCAL ] DX: Trimmed 75990 records to 52473 (keeping 200 days of data) [2025-11-04T18:32:49.454638] [DEBUG] [LOCAL ] DX: Primary data saved (52473 records). [2025-11-04T18:32:49.454972] [INFO ] [MAIN ] DX: Full history saved. [2025-11-04T18:32:49.455084] [INFO ] [MAIN ] Update Progress: 670/2367 symbols completed. DX took 28.1s. Estimated remaining: 3h 54m 58s. Rate: 7.2 symbols/min [2025-11-04T18:32:49.626498] [INFO ] [MAIN ] DY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:32:49.661575] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-11-04T18:32:49.696253] [DEBUG] [LOCAL ] DY: Loaded 27496 existing records from primary storage. [2025-11-04T18:32:49.696517] [INFO ] [MAIN ] DY: Loaded 27496 existing records. [2025-11-04T18:32:49.772481] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-08. [2025-11-04T18:32:49.874152] [WARN ] [MAIN ] DY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:32:49.874384] [INFO ] [MAIN ] DY: Wiping existing primary data before full refresh. [2025-11-04T18:32:49.874431] [INFO ] [LOCAL ] DY: Starting deletion of primary data. [2025-11-04T18:32:49.874622] [DEBUG] [LOCAL ] DY: Deleted primary data file. [2025-11-04T18:32:49.874735] [INFO ] [MAIN ] DY: Starting full primary refresh. [2025-11-04T18:32:50.226361] [INFO ] [REMOTE ] DY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:32:50.451377] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T18:32:50.567010] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-01. [2025-11-04T18:32:51.367009] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-06. [2025-11-04T18:32:51.393568] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-11-04T18:32:51.485784] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-11-04T18:32:52.167089] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-02. [2025-11-04T18:32:52.742708] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-11-04T18:32:52.967055] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-09. [2025-11-04T18:32:53.234626] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T18:32:53.767060] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-07. [2025-11-04T18:32:54.450335] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-11-04T18:32:54.567028] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-10. [2025-11-04T18:32:55.202196] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T18:32:55.367012] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-06. [2025-11-04T18:32:56.167027] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-03. [2025-11-04T18:32:56.280744] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-11-04T18:32:56.524346] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T18:32:56.967039] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-07. [2025-11-04T18:32:57.209744] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T18:32:57.767041] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-11. [2025-11-04T18:32:58.208628] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-11 (0.44 seconds). [2025-11-04T18:32:58.210118] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-11-04T18:32:58.222736] [INFO ] [REMOTE ] DVN: Full fetch complete (77879 records). [2025-11-04T18:32:58.222891] [DEBUG] [VALIDATOR ] DVN: Validating primary data. [2025-11-04T18:32:58.572179] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-08. [2025-11-04T18:32:58.698358] [DEBUG] [VALIDATOR ] DVN: All expected trading days are present. [2025-11-04T18:32:58.698467] [DEBUG] [VALIDATOR ] DVN: Primary data validation passed. [2025-11-04T18:32:58.699635] [INFO ] [MAIN ] DVN: Fetched primary data is valid. [2025-11-04T18:32:58.699729] [DEBUG] [LOCAL ] DVN: Saving primary data. [2025-11-04T18:32:58.701220] [DEBUG] [LOCAL ] DVN: Trimmed 77879 records to 53699 (keeping 200 days of data) [2025-11-04T18:32:59.063066] [DEBUG] [LOCAL ] DVN: Primary data saved (53699 records). [2025-11-04T18:32:59.063190] [INFO ] [MAIN ] DVN: Full history saved. [2025-11-04T18:32:59.063266] [INFO ] [MAIN ] Update Progress: 671/2367 symbols completed. DVN took 39.7s. Estimated remaining: 3h 54m 53s. Rate: 7.2 symbols/min [2025-11-04T18:32:59.268268] [INFO ] [MAIN ] E: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:32:59.302267] [DEBUG] [LOCAL ] E: Loaded 32350 existing records from primary storage. [2025-11-04T18:32:59.302530] [INFO ] [MAIN ] E: Loaded 32350 existing records. [2025-11-04T18:32:59.376778] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-04. [2025-11-04T18:32:59.515905] [WARN ] [MAIN ] E: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:32:59.516152] [INFO ] [MAIN ] E: Wiping existing primary data before full refresh. [2025-11-04T18:32:59.516233] [INFO ] [LOCAL ] E: Starting deletion of primary data. [2025-11-04T18:32:59.516435] [DEBUG] [LOCAL ] E: Deleted primary data file. [2025-11-04T18:32:59.516487] [INFO ] [MAIN ] E: Starting full primary refresh. [2025-11-04T18:32:59.963955] [INFO ] [REMOTE ] E: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:32:59.994414] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T18:33:00.167050] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-09. [2025-11-04T18:33:00.669908] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T18:33:00.967070] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-05. [2025-11-04T18:33:01.597998] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T18:33:01.767061] [INFO ] [REMOTE ] E: Fetching primary data for 2025-01. [2025-11-04T18:33:02.227796] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T18:33:02.567079] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-10. [2025-11-04T18:33:02.867223] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-11-04T18:33:03.367087] [INFO ] [REMOTE ] E: Fetching primary data for 2025-02. [2025-11-04T18:33:03.854422] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T18:33:04.167048] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-08. [2025-11-04T18:33:04.207115] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-11-04T18:33:04.967053] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-06. [2025-11-04T18:33:05.697699] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T18:33:05.767005] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-09. [2025-11-04T18:33:05.822153] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T18:33:06.567043] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-11. [2025-11-04T18:33:07.212349] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-11-04T18:33:07.367034] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-10. [2025-11-04T18:33:07.872356] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-11 (1.31 seconds). [2025-11-04T18:33:07.874381] [INFO ] [REMOTE ] DXC: Full fetch complete (70877 records). [2025-11-04T18:33:07.874468] [DEBUG] [VALIDATOR ] DXC: Validating primary data. [2025-11-04T18:33:08.172146] [INFO ] [REMOTE ] E: Fetching primary data for 2025-03. [2025-11-04T18:33:08.258706] [DEBUG] [VALIDATOR ] DXC: All expected trading days are present. [2025-11-04T18:33:08.258807] [DEBUG] [VALIDATOR ] DXC: Primary data validation passed. [2025-11-04T18:33:08.259600] [INFO ] [MAIN ] DXC: Fetched primary data is valid. [2025-11-04T18:33:08.259639] [DEBUG] [LOCAL ] DXC: Saving primary data. [2025-11-04T18:33:08.261085] [DEBUG] [LOCAL ] DXC: Trimmed 70877 records to 49917 (keeping 200 days of data) [2025-11-04T18:33:08.556438] [DEBUG] [LOCAL ] DXC: Primary data saved (49917 records). [2025-11-04T18:33:08.556594] [INFO ] [MAIN ] DXC: Full history saved. [2025-11-04T18:33:08.556714] [INFO ] [MAIN ] Update Progress: 672/2367 symbols completed. DXC took 38.7s. Estimated remaining: 3h 54m 48s. Rate: 7.2 symbols/min [2025-11-04T18:33:08.724836] [INFO ] [MAIN ] EAF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:33:08.725027] [DEBUG] [LOCAL ] EAF: No existing data file found in primary storage. [2025-11-04T18:33:08.725063] [WARN ] [MAIN ] EAF: No existing data found, performing full refresh instead. [2025-11-04T18:33:08.725172] [INFO ] [MAIN ] EAF: Wiping existing primary data before full refresh. [2025-11-04T18:33:08.725212] [INFO ] [LOCAL ] EAF: Starting deletion of primary data. [2025-11-04T18:33:08.725251] [DEBUG] [LOCAL ] EAF: No primary data file found to delete. [2025-11-04T18:33:08.725281] [INFO ] [MAIN ] EAF: Starting full primary refresh. [2025-11-04T18:33:08.970411] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-07. [2025-11-04T18:33:09.093462] [INFO ] [REMOTE ] EAF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:33:09.119367] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-11-04T18:33:09.266528] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T18:33:09.767048] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-01. [2025-11-04T18:33:09.822268] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T18:33:10.567072] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-08. [2025-11-04T18:33:11.081744] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T18:33:11.367045] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-02. [2025-11-04T18:33:11.561961] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-11-04T18:33:12.167043] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-09. [2025-11-04T18:33:12.664255] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T18:33:12.967049] [INFO ] [REMOTE ] E: Fetching primary data for 2025-04. [2025-11-04T18:33:13.358313] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T18:33:13.767071] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-10. [2025-11-04T18:33:14.266064] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T18:33:14.509903] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-11-04T18:33:14.567023] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-11. [2025-11-04T18:33:15.282396] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T18:33:15.284743] [INFO ] [REMOTE ] DY: Full fetch complete (43280 records). [2025-11-04T18:33:15.284902] [DEBUG] [VALIDATOR ] DY: Validating primary data. [2025-11-04T18:33:15.372231] [INFO ] [REMOTE ] E: Fetching primary data for 2025-05. [2025-11-04T18:33:15.656021] [DEBUG] [VALIDATOR ] DY: All expected trading days are present. [2025-11-04T18:33:15.656095] [DEBUG] [VALIDATOR ] DY: Primary data validation passed. [2025-11-04T18:33:15.656662] [INFO ] [MAIN ] DY: Fetched primary data is valid. [2025-11-04T18:33:15.656723] [DEBUG] [LOCAL ] DY: Saving primary data. [2025-11-04T18:33:15.657592] [DEBUG] [LOCAL ] DY: Trimmed 43280 records to 27557 (keeping 200 days of data) [2025-11-04T18:33:15.814415] [DEBUG] [LOCAL ] DY: Primary data saved (27557 records). [2025-11-04T18:33:15.814532] [INFO ] [MAIN ] DY: Full history saved. [2025-11-04T18:33:15.814606] [INFO ] [MAIN ] Update Progress: 673/2367 symbols completed. DY took 26.4s. Estimated remaining: 3h 54m 37s. Rate: 7.2 symbols/min [2025-11-04T18:33:15.980457] [INFO ] [MAIN ] EAI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:33:15.986375] [DEBUG] [LOCAL ] EAI: Loaded 3812 existing records from primary storage. [2025-11-04T18:33:15.986524] [INFO ] [MAIN ] EAI: Loaded 3812 existing records. [2025-11-04T18:33:16.159523] [WARN ] [MAIN ] EAI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:33:16.159794] [INFO ] [MAIN ] EAI: Wiping existing primary data before full refresh. [2025-11-04T18:33:16.159856] [INFO ] [LOCAL ] EAI: Starting deletion of primary data. [2025-11-04T18:33:16.160070] [DEBUG] [LOCAL ] EAI: Deleted primary data file. [2025-11-04T18:33:16.160137] [INFO ] [MAIN ] EAI: Starting full primary refresh. [2025-11-04T18:33:16.172145] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-03. [2025-11-04T18:33:16.526821] [INFO ] [REMOTE ] EAI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:33:16.585538] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T18:33:16.967090] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-11. [2025-11-04T18:33:17.561637] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T18:33:17.767054] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-01. [2025-11-04T18:33:18.246488] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-01 (0.48 seconds). [2025-11-04T18:33:18.567353] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-04. [2025-11-04T18:33:18.568068] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-11 (1.60 seconds). [2025-11-04T18:33:18.570689] [INFO ] [REMOTE ] DXYZ: Full fetch complete (55876 records). [2025-11-04T18:33:18.570771] [DEBUG] [VALIDATOR ] DXYZ: Validating primary data. [2025-11-04T18:33:18.964347] [DEBUG] [VALIDATOR ] DXYZ: All expected trading days are present. [2025-11-04T18:33:18.964432] [DEBUG] [VALIDATOR ] DXYZ: Primary data validation passed. [2025-11-04T18:33:18.965120] [INFO ] [MAIN ] DXYZ: Fetched primary data is valid. [2025-11-04T18:33:18.965184] [DEBUG] [LOCAL ] DXYZ: Saving primary data. [2025-11-04T18:33:18.966347] [DEBUG] [LOCAL ] DXYZ: Trimmed 55876 records to 36813 (keeping 200 days of data) [2025-11-04T18:33:19.176782] [DEBUG] [LOCAL ] DXYZ: Primary data saved (36813 records). [2025-11-04T18:33:19.176922] [INFO ] [MAIN ] DXYZ: Full history saved. [2025-11-04T18:33:19.176993] [INFO ] [MAIN ] Update Progress: 674/2367 symbols completed. DXYZ took 43.1s. Estimated remaining: 3h 54m 16s. Rate: 7.2 symbols/min [2025-11-04T18:33:19.352272] [INFO ] [MAIN ] EARN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:33:19.367058] [INFO ] [REMOTE ] E: Fetching primary data for 2025-06. [2025-11-04T18:33:19.378417] [DEBUG] [LOCAL ] EARN: Loaded 26878 existing records from primary storage. [2025-11-04T18:33:19.378722] [INFO ] [MAIN ] EARN: Loaded 26878 existing records. [2025-11-04T18:33:19.574835] [WARN ] [MAIN ] EARN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:33:19.575035] [INFO ] [MAIN ] EARN: Wiping existing primary data before full refresh. [2025-11-04T18:33:19.575082] [INFO ] [LOCAL ] EARN: Starting deletion of primary data. [2025-11-04T18:33:19.575276] [DEBUG] [LOCAL ] EARN: Deleted primary data file. [2025-11-04T18:33:19.575331] [INFO ] [MAIN ] EARN: Starting full primary refresh. [2025-11-04T18:33:19.932708] [INFO ] [REMOTE ] EARN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:33:19.962567] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T18:33:20.167058] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-01. [2025-11-04T18:33:20.422452] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-11-04T18:33:20.967089] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-05. [2025-11-04T18:33:21.271022] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-11-04T18:33:21.767101] [INFO ] [REMOTE ] E: Fetching primary data for 2025-07. [2025-11-04T18:33:22.495283] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-11-04T18:33:22.567002] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-06. [2025-11-04T18:33:22.704489] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-11-04T18:33:23.367066] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-02. [2025-11-04T18:33:23.631007] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-02 (0.26 seconds). [2025-11-04T18:33:23.665406] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T18:33:24.167065] [INFO ] [REMOTE ] E: Fetching primary data for 2025-08. [2025-11-04T18:33:24.967057] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-07. [2025-11-04T18:33:25.270918] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-11-04T18:33:25.767053] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-02. [2025-11-04T18:33:26.235054] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T18:33:26.567045] [INFO ] [REMOTE ] E: Fetching primary data for 2025-09. [2025-11-04T18:33:26.785254] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T18:33:27.367046] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-08. [2025-11-04T18:33:27.575957] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T18:33:28.167081] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-03. [2025-11-04T18:33:28.605820] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T18:33:28.967062] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-03. [2025-11-04T18:33:29.326535] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-11-04T18:33:29.510804] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-11-04T18:33:29.767053] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-09. [2025-11-04T18:33:30.567029] [INFO ] [REMOTE ] E: Fetching primary data for 2025-10. [2025-11-04T18:33:30.632273] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-11-04T18:33:31.367046] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-04. [2025-11-04T18:33:31.671540] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T18:33:31.803886] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-11-04T18:33:32.167047] [INFO ] [REMOTE ] E: Fetching primary data for 2025-11. [2025-11-04T18:33:32.967101] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-04. [2025-11-04T18:33:33.059738] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T18:33:33.061202] [INFO ] [REMOTE ] E: Full fetch complete (47493 records). [2025-11-04T18:33:33.061287] [DEBUG] [VALIDATOR ] E: Validating primary data. [2025-11-04T18:33:33.428750] [DEBUG] [VALIDATOR ] E: All expected trading days are present. [2025-11-04T18:33:33.428833] [DEBUG] [VALIDATOR ] E: Primary data validation passed. [2025-11-04T18:33:33.429489] [INFO ] [MAIN ] E: Fetched primary data is valid. [2025-11-04T18:33:33.429530] [DEBUG] [LOCAL ] E: Saving primary data. [2025-11-04T18:33:33.430547] [DEBUG] [LOCAL ] E: Trimmed 47493 records to 32244 (keeping 200 days of data) [2025-11-04T18:33:33.627840] [DEBUG] [LOCAL ] E: Primary data saved (32244 records). [2025-11-04T18:33:33.627972] [INFO ] [MAIN ] E: Full history saved. [2025-11-04T18:33:33.628050] [INFO ] [MAIN ] Update Progress: 675/2367 symbols completed. E took 34.6s. Estimated remaining: 3h 54m 23s. Rate: 7.2 symbols/min [2025-11-04T18:33:33.773211] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-10. [2025-11-04T18:33:33.798494] [INFO ] [MAIN ] EAT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:33:33.841957] [DEBUG] [LOCAL ] EAT: Loaded 49723 existing records from primary storage. [2025-11-04T18:33:33.842277] [INFO ] [MAIN ] EAT: Loaded 49723 existing records. [2025-11-04T18:33:34.014462] [WARN ] [MAIN ] EAT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:33:34.014721] [INFO ] [MAIN ] EAT: Wiping existing primary data before full refresh. [2025-11-04T18:33:34.014782] [INFO ] [LOCAL ] EAT: Starting deletion of primary data. [2025-11-04T18:33:34.015193] [DEBUG] [LOCAL ] EAT: Deleted primary data file. [2025-11-04T18:33:34.015232] [INFO ] [MAIN ] EAT: Starting full primary refresh. [2025-11-04T18:33:34.370767] [INFO ] [REMOTE ] EAT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:33:34.393851] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T18:33:34.567011] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-05. [2025-11-04T18:33:34.955340] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T18:33:35.367042] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-01. [2025-11-04T18:33:35.582020] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T18:33:36.167138] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-11. [2025-11-04T18:33:36.777865] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T18:33:36.967090] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-05. [2025-11-04T18:33:37.064957] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T18:33:37.066577] [INFO ] [REMOTE ] EAF: Full fetch complete (60619 records). [2025-11-04T18:33:37.066718] [DEBUG] [VALIDATOR ] EAF: Validating primary data. [2025-11-04T18:33:37.445799] [DEBUG] [VALIDATOR ] EAF: All expected trading days are present. [2025-11-04T18:33:37.445891] [DEBUG] [VALIDATOR ] EAF: Primary data validation passed. [2025-11-04T18:33:37.446669] [INFO ] [MAIN ] EAF: Fetched primary data is valid. [2025-11-04T18:33:37.446754] [DEBUG] [LOCAL ] EAF: Saving primary data. [2025-11-04T18:33:37.447912] [DEBUG] [LOCAL ] EAF: Trimmed 60619 records to 39747 (keeping 200 days of data) [2025-11-04T18:33:37.505206] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-11-04T18:33:37.686266] [DEBUG] [LOCAL ] EAF: Primary data saved (39747 records). [2025-11-04T18:33:37.686422] [INFO ] [MAIN ] EAF: Full history saved. [2025-11-04T18:33:37.686498] [INFO ] [MAIN ] Update Progress: 676/2367 symbols completed. EAF took 29.1s. Estimated remaining: 3h 54m 4s. Rate: 7.2 symbols/min [2025-11-04T18:33:37.772355] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-06. [2025-11-04T18:33:37.856406] [INFO ] [MAIN ] EB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:33:37.856574] [DEBUG] [LOCAL ] EB: No existing data file found in primary storage. [2025-11-04T18:33:37.856611] [WARN ] [MAIN ] EB: No existing data found, performing full refresh instead. [2025-11-04T18:33:37.856771] [INFO ] [MAIN ] EB: Wiping existing primary data before full refresh. [2025-11-04T18:33:37.856831] [INFO ] [LOCAL ] EB: Starting deletion of primary data. [2025-11-04T18:33:37.856873] [DEBUG] [LOCAL ] EB: No primary data file found to delete. [2025-11-04T18:33:37.856903] [INFO ] [MAIN ] EB: Starting full primary refresh. [2025-11-04T18:33:38.202086] [INFO ] [REMOTE ] EB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:33:38.246404] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-11-04T18:33:38.567065] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-01. [2025-11-04T18:33:39.367054] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-02. [2025-11-04T18:33:39.717588] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T18:33:40.167063] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-07. [2025-11-04T18:33:40.652935] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-11-04T18:33:40.765043] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-11-04T18:33:40.967030] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-06. [2025-11-04T18:33:41.742747] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T18:33:41.767007] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-07. [2025-11-04T18:33:42.567019] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-08. [2025-11-04T18:33:42.698993] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-11-04T18:33:42.971256] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-11-04T18:33:43.367035] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-02. [2025-11-04T18:33:44.167016] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-09. [2025-11-04T18:33:44.358164] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T18:33:44.690442] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-11-04T18:33:44.967066] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-03. [2025-11-04T18:33:45.767092] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-03. [2025-11-04T18:33:46.567100] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-10. [2025-11-04T18:33:46.829671] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-11-04T18:33:46.851200] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-10 (0.28 seconds). [2025-11-04T18:33:47.078087] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T18:33:47.367033] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-04. [2025-11-04T18:33:48.167092] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-08. [2025-11-04T18:33:48.701495] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T18:33:48.967079] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-11. [2025-11-04T18:33:49.293053] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T18:33:49.363617] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-11 (0.40 seconds). [2025-11-04T18:33:49.364352] [INFO ] [REMOTE ] EAI: Full fetch complete (5695 records). [2025-11-04T18:33:49.364419] [DEBUG] [VALIDATOR ] EAI: Validating primary data. [2025-11-04T18:33:49.724395] [DEBUG] [VALIDATOR ] EAI: All expected trading days are present. [2025-11-04T18:33:49.724473] [DEBUG] [VALIDATOR ] EAI: Primary data validation passed. [2025-11-04T18:33:49.724638] [INFO ] [MAIN ] EAI: Fetched primary data is valid. [2025-11-04T18:33:49.724721] [DEBUG] [LOCAL ] EAI: Saving primary data. [2025-11-04T18:33:49.725136] [DEBUG] [LOCAL ] EAI: Trimmed 5695 records to 3821 (keeping 200 days of data) [2025-11-04T18:33:49.748138] [DEBUG] [LOCAL ] EAI: Primary data saved (3821 records). [2025-11-04T18:33:49.748270] [INFO ] [MAIN ] EAI: Full history saved. [2025-11-04T18:33:49.748350] [INFO ] [MAIN ] Update Progress: 677/2367 symbols completed. EAI took 33.9s. Estimated remaining: 3h 54m 5s. Rate: 7.2 symbols/min [2025-11-04T18:33:49.767032] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-09. [2025-11-04T18:33:50.567095] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-05. [2025-11-04T18:33:50.629748] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-11-04T18:33:51.367106] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-10. [2025-11-04T18:33:51.548907] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-11-04T18:33:52.167152] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-06. [2025-11-04T18:33:52.223632] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-11-04T18:33:52.967145] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-04. [2025-11-04T18:33:53.237768] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T18:33:53.767121] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-07. [2025-11-04T18:33:54.380737] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-11-04T18:33:54.567050] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-11. [2025-11-04T18:33:54.851005] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T18:33:55.330043] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-11 (0.76 seconds). [2025-11-04T18:33:55.331549] [INFO ] [REMOTE ] EARN: Full fetch complete (44928 records). [2025-11-04T18:33:55.331622] [DEBUG] [VALIDATOR ] EARN: Validating primary data. [2025-11-04T18:33:55.374539] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-08. [2025-11-04T18:33:55.707238] [DEBUG] [VALIDATOR ] EARN: All expected trading days are present. [2025-11-04T18:33:55.707356] [DEBUG] [VALIDATOR ] EARN: Primary data validation passed. [2025-11-04T18:33:55.707971] [INFO ] [MAIN ] EARN: Fetched primary data is valid. [2025-11-04T18:33:55.708035] [DEBUG] [LOCAL ] EARN: Saving primary data. [2025-11-04T18:33:55.708992] [DEBUG] [LOCAL ] EARN: Trimmed 44928 records to 26879 (keeping 200 days of data) [2025-11-04T18:33:55.859842] [DEBUG] [LOCAL ] EARN: Primary data saved (26879 records). [2025-11-04T18:33:55.859963] [INFO ] [MAIN ] EARN: Full history saved. [2025-11-04T18:33:55.860038] [INFO ] [MAIN ] Update Progress: 678/2367 symbols completed. EARN took 36.7s. Estimated remaining: 3h 53m 52s. Rate: 7.2 symbols/min [2025-11-04T18:33:56.167048] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-05. [2025-11-04T18:33:56.504129] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T18:33:56.967123] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-09. [2025-11-04T18:33:57.618873] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T18:33:57.767011] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-06. [2025-11-04T18:33:57.964423] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T18:33:58.567123] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-10. [2025-11-04T18:33:58.992307] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-11-04T18:33:59.367028] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-07. [2025-11-04T18:33:59.824734] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-11-04T18:34:00.167125] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-11. [2025-11-04T18:34:01.050206] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-11-04T18:34:01.061191] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-08. [2025-11-04T18:34:01.208786] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T18:34:01.210071] [INFO ] [REMOTE ] EB: Full fetch complete (53917 records). [2025-11-04T18:34:01.210179] [DEBUG] [VALIDATOR ] EB: Validating primary data. [2025-11-04T18:34:01.210755] [FATAL] [VALIDATOR ] EB: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:34:01.210840] [FATAL] [MAIN ] EB: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:34:01.210883] [WARN ] [MAIN ] EB: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:34:01.210947] [INFO ] [LOCAL ] EB: Starting deletion of primary data. [2025-11-04T18:34:01.211011] [DEBUG] [LOCAL ] EB: No primary data file found to delete. [2025-11-04T18:34:01.211046] [INFO ] [LOCAL ] EB: Starting deletion of derivative data. [2025-11-04T18:34:01.211085] [DEBUG] [LOCAL ] EB: No derivative data file found to delete. [2025-11-04T18:34:01.211138] [INFO ] [MAIN ] Update Progress: 679/2367 symbols completed. EB took 23.5s. Estimated remaining: 3h 53m 36s. Rate: 7.2 symbols/min [2025-11-04T18:34:02.708309] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-11-04T18:34:02.718879] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-09. [2025-11-04T18:34:04.236532] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-11-04T18:34:04.247151] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-10. [2025-11-04T18:34:05.999813] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-11-04T18:34:06.015923] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-11. [2025-11-04T18:34:07.359011] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-11 (1.34 seconds). [2025-11-04T18:34:07.361717] [INFO ] [REMOTE ] EAT: Full fetch complete (72638 records). [2025-11-04T18:34:07.361897] [DEBUG] [VALIDATOR ] EAT: Validating primary data. [2025-11-04T18:34:07.768201] [DEBUG] [VALIDATOR ] EAT: All expected trading days are present. [2025-11-04T18:34:07.768319] [DEBUG] [VALIDATOR ] EAT: Primary data validation passed. [2025-11-04T18:34:07.769532] [INFO ] [MAIN ] EAT: Fetched primary data is valid. [2025-11-04T18:34:07.769607] [DEBUG] [LOCAL ] EAT: Saving primary data. [2025-11-04T18:34:07.771606] [DEBUG] [LOCAL ] EAT: Trimmed 72638 records to 49749 (keeping 200 days of data) [2025-11-04T18:34:08.100867] [DEBUG] [LOCAL ] EAT: Primary data saved (49749 records). [2025-11-04T18:34:08.101115] [INFO ] [MAIN ] EAT: Full history saved. [2025-11-04T18:34:08.101234] [INFO ] [MAIN ] Update Progress: 680/2367 symbols completed. EAT took 34.5s. Estimated remaining: 3h 53m 24s. Rate: 7.2 symbols/min [2025-11-04T18:34:08.934141] [INFO ] [MAIN ] EC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:34:08.963311] [INFO ] [MAIN ] EBS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:34:08.938409] [INFO ] [MAIN ] EBR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:34:08.994554] [INFO ] [MAIN ] EBF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:34:09.036636] [DEBUG] [LOCAL ] EBF: Loaded 19535 existing records from primary storage. [2025-11-04T18:34:09.037101] [INFO ] [MAIN ] EBF: Loaded 19535 existing records. [2025-11-04T18:34:09.191062] [DEBUG] [LOCAL ] EBR: Loaded 43152 existing records from primary storage. [2025-11-04T18:34:09.203041] [INFO ] [MAIN ] EBR: Loaded 43152 existing records. [2025-11-04T18:34:09.341030] [DEBUG] [LOCAL ] EBS: Loaded 46638 existing records from primary storage. [2025-11-04T18:34:09.368740] [WARN ] [MAIN ] EBF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:34:09.406032] [INFO ] [MAIN ] EBS: Loaded 46638 existing records. [2025-11-04T18:34:09.407331] [DEBUG] [LOCAL ] EC: Loaded 48591 existing records from primary storage. [2025-11-04T18:34:09.407626] [INFO ] [MAIN ] EBF: Wiping existing primary data before full refresh. [2025-11-04T18:34:09.457397] [INFO ] [MAIN ] EC: Loaded 48591 existing records. [2025-11-04T18:34:09.485762] [INFO ] [LOCAL ] EBF: Starting deletion of primary data. [2025-11-04T18:34:09.646499] [DEBUG] [LOCAL ] EBF: Deleted primary data file. [2025-11-04T18:34:09.709514] [INFO ] [MAIN ] EBF: Starting full primary refresh. [2025-11-04T18:34:09.807607] [WARN ] [MAIN ] EBR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:34:09.917444] [WARN ] [MAIN ] EBS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:34:09.879313] [INFO ] [MAIN ] EBR: Wiping existing primary data before full refresh. [2025-11-04T18:34:09.929046] [INFO ] [LOCAL ] EBR: Starting deletion of primary data. [2025-11-04T18:34:09.949821] [INFO ] [MAIN ] EBS: Wiping existing primary data before full refresh. [2025-11-04T18:34:09.950075] [INFO ] [LOCAL ] EBS: Starting deletion of primary data. [2025-11-04T18:34:09.955291] [DEBUG] [LOCAL ] EBR: Deleted primary data file. [2025-11-04T18:34:09.965985] [DEBUG] [LOCAL ] EBS: Deleted primary data file. [2025-11-04T18:34:09.965920] [INFO ] [MAIN ] EBR: Starting full primary refresh. [2025-11-04T18:34:09.981570] [INFO ] [MAIN ] EBS: Starting full primary refresh. [2025-11-04T18:34:10.174771] [WARN ] [MAIN ] EC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:34:10.218028] [INFO ] [MAIN ] EC: Wiping existing primary data before full refresh. [2025-11-04T18:34:10.284594] [INFO ] [LOCAL ] EC: Starting deletion of primary data. [2025-11-04T18:34:10.366631] [DEBUG] [LOCAL ] EC: Deleted primary data file. [2025-11-04T18:34:10.450208] [INFO ] [MAIN ] EC: Starting full primary refresh. [2025-11-04T18:34:11.241842] [INFO ] [REMOTE ] EBF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:34:11.337366] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-01. [2025-11-04T18:34:11.389107] [INFO ] [REMOTE ] EBS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:34:11.418677] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-01. [2025-11-04T18:34:11.483264] [INFO ] [REMOTE ] EBR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:34:11.500051] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-01. [2025-11-04T18:34:11.619616] [INFO ] [REMOTE ] EC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:34:11.620067] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-01. [2025-11-04T18:34:12.704169] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-11-04T18:34:12.708000] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-02. [2025-11-04T18:34:13.062518] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-11-04T18:34:13.073183] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-02. [2025-11-04T18:34:13.187129] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-11-04T18:34:13.196428] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-02. [2025-11-04T18:34:13.363555] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-11-04T18:34:13.368605] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-03. [2025-11-04T18:34:13.603857] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-11-04T18:34:13.615046] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-02. [2025-11-04T18:34:14.128890] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T18:34:14.138857] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-03. [2025-11-04T18:34:14.193847] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T18:34:14.201866] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-03. [2025-11-04T18:34:14.324393] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T18:34:14.330131] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-04. [2025-11-04T18:34:14.879278] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T18:34:14.890065] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-03. [2025-11-04T18:34:15.125928] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-11-04T18:34:15.132440] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-05. [2025-11-04T18:34:15.446047] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T18:34:15.456841] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-04. [2025-11-04T18:34:15.688153] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-11-04T18:34:16.006029] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T18:34:16.167059] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-06. [2025-11-04T18:34:16.280868] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T18:34:16.764853] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T18:34:16.967056] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-04. [2025-11-04T18:34:16.984077] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-11-04T18:34:17.767105] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-04. [2025-11-04T18:34:18.567044] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-05. [2025-11-04T18:34:18.785272] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-11-04T18:34:19.003839] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T18:34:19.367044] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-05. [2025-11-04T18:34:19.768563] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T18:34:20.167079] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-07. [2025-11-04T18:34:20.656071] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T18:34:20.967021] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-05. [2025-11-04T18:34:21.032249] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-11-04T18:34:21.767071] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-06. [2025-11-04T18:34:22.156385] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T18:34:22.567090] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-06. [2025-11-04T18:34:23.250019] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-11-04T18:34:23.367038] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-07. [2025-11-04T18:34:23.848017] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T18:34:24.167058] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-06. [2025-11-04T18:34:24.894615] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T18:34:24.967092] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-07. [2025-11-04T18:34:25.632375] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T18:34:25.767031] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-08. [2025-11-04T18:34:26.131141] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-11-04T18:34:26.567082] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-08. [2025-11-04T18:34:26.945198] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-11-04T18:34:27.367042] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-08. [2025-11-04T18:34:27.776834] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T18:34:28.138758] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T18:34:28.166996] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-09. [2025-11-04T18:34:28.864162] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-11-04T18:34:28.967044] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-10. [2025-11-04T18:34:29.767074] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-07. [2025-11-04T18:34:29.965326] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-11-04T18:34:30.567102] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-09. [2025-11-04T18:34:31.008843] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-11-04T18:34:31.367088] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-09. [2025-11-04T18:34:31.817723] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T18:34:32.167059] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-08. [2025-11-04T18:34:32.777538] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-11-04T18:34:32.967054] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-11. [2025-11-04T18:34:33.333471] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T18:34:33.693416] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T18:34:33.694832] [INFO ] [REMOTE ] EBF: Full fetch complete (28911 records). [2025-11-04T18:34:33.694913] [DEBUG] [VALIDATOR ] EBF: Validating primary data. [2025-11-04T18:34:33.772141] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-10. [2025-11-04T18:34:34.062100] [DEBUG] [VALIDATOR ] EBF: All expected trading days are present. [2025-11-04T18:34:34.062179] [DEBUG] [VALIDATOR ] EBF: Primary data validation passed. [2025-11-04T18:34:34.062691] [INFO ] [MAIN ] EBF: Fetched primary data is valid. [2025-11-04T18:34:34.062752] [DEBUG] [LOCAL ] EBF: Saving primary data. [2025-11-04T18:34:34.063411] [DEBUG] [LOCAL ] EBF: Trimmed 28911 records to 19548 (keeping 200 days of data) [2025-11-04T18:34:34.176356] [DEBUG] [LOCAL ] EBF: Primary data saved (19548 records). [2025-11-04T18:34:34.176532] [INFO ] [MAIN ] EBF: Full history saved. [2025-11-04T18:34:34.176721] [INFO ] [MAIN ] Update Progress: 681/2367 symbols completed. EBF took 26.1s. Estimated remaining: 3h 53m 60s. Rate: 7.2 symbols/min [2025-11-04T18:34:34.382753] [INFO ] [MAIN ] ECAT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:34:34.382939] [DEBUG] [LOCAL ] ECAT: No existing data file found in primary storage. [2025-11-04T18:34:34.383168] [WARN ] [MAIN ] ECAT: No existing data found, performing full refresh instead. [2025-11-04T18:34:34.383282] [INFO ] [MAIN ] ECAT: Wiping existing primary data before full refresh. [2025-11-04T18:34:34.383325] [INFO ] [LOCAL ] ECAT: Starting deletion of primary data. [2025-11-04T18:34:34.383367] [DEBUG] [LOCAL ] ECAT: No primary data file found to delete. [2025-11-04T18:34:34.383403] [INFO ] [MAIN ] ECAT: Starting full primary refresh. [2025-11-04T18:34:34.571944] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-09. [2025-11-04T18:34:34.815297] [INFO ] [REMOTE ] ECAT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:34:35.238582] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T18:34:35.367045] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-11. [2025-11-04T18:34:36.173077] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-10. [2025-11-04T18:34:36.175374] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-11-04T18:34:36.611152] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T18:34:36.613068] [INFO ] [REMOTE ] EC: Full fetch complete (71728 records). [2025-11-04T18:34:36.613183] [DEBUG] [VALIDATOR ] EC: Validating primary data. [2025-11-04T18:34:36.974972] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-01. [2025-11-04T18:34:37.028101] [DEBUG] [VALIDATOR ] EC: All expected trading days are present. [2025-11-04T18:34:37.028220] [DEBUG] [VALIDATOR ] EC: Primary data validation passed. [2025-11-04T18:34:37.029491] [INFO ] [MAIN ] EC: Fetched primary data is valid. [2025-11-04T18:34:37.029593] [DEBUG] [LOCAL ] EC: Saving primary data. [2025-11-04T18:34:37.031752] [DEBUG] [LOCAL ] EC: Trimmed 71728 records to 48604 (keeping 200 days of data) [2025-11-04T18:34:37.328176] [DEBUG] [LOCAL ] EC: Primary data saved (48604 records). [2025-11-04T18:34:37.328418] [INFO ] [MAIN ] EC: Full history saved. [2025-11-04T18:34:37.328691] [INFO ] [MAIN ] Update Progress: 682/2367 symbols completed. EC took 29.2s. Estimated remaining: 3h 53m 39s. Rate: 7.2 symbols/min [2025-11-04T18:34:37.510136] [INFO ] [MAIN ] ECC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:34:37.544331] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T18:34:37.587117] [DEBUG] [LOCAL ] ECC: Loaded 42872 existing records from primary storage. [2025-11-04T18:34:37.587450] [INFO ] [MAIN ] ECC: Loaded 42872 existing records. [2025-11-04T18:34:37.761547] [WARN ] [MAIN ] ECC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:34:37.761766] [INFO ] [MAIN ] ECC: Wiping existing primary data before full refresh. [2025-11-04T18:34:37.761815] [INFO ] [LOCAL ] ECC: Starting deletion of primary data. [2025-11-04T18:34:37.762131] [DEBUG] [LOCAL ] ECC: Deleted primary data file. [2025-11-04T18:34:37.762186] [INFO ] [MAIN ] ECC: Starting full primary refresh. [2025-11-04T18:34:37.773298] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-10. [2025-11-04T18:34:38.120280] [INFO ] [REMOTE ] ECC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:34:38.137326] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T18:34:38.567298] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-01. [2025-11-04T18:34:39.121509] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T18:34:39.367055] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-11. [2025-11-04T18:34:39.929907] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T18:34:40.167073] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-11. [2025-11-04T18:34:40.428775] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T18:34:40.430775] [INFO ] [REMOTE ] EBS: Full fetch complete (67586 records). [2025-11-04T18:34:40.430877] [DEBUG] [VALIDATOR ] EBS: Validating primary data. [2025-11-04T18:34:40.832058] [DEBUG] [VALIDATOR ] EBS: All expected trading days are present. [2025-11-04T18:34:40.832147] [DEBUG] [VALIDATOR ] EBS: Primary data validation passed. [2025-11-04T18:34:40.833005] [INFO ] [MAIN ] EBS: Fetched primary data is valid. [2025-11-04T18:34:40.833070] [DEBUG] [LOCAL ] EBS: Saving primary data. [2025-11-04T18:34:40.834409] [DEBUG] [LOCAL ] EBS: Trimmed 67586 records to 46688 (keeping 200 days of data) [2025-11-04T18:34:40.990856] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-02. [2025-11-04T18:34:41.101480] [DEBUG] [LOCAL ] EBS: Primary data saved (46688 records). [2025-11-04T18:34:41.101623] [INFO ] [MAIN ] EBS: Full history saved. [2025-11-04T18:34:41.101761] [INFO ] [MAIN ] Update Progress: 683/2367 symbols completed. EBS took 33.0s. Estimated remaining: 3h 53m 19s. Rate: 7.2 symbols/min [2025-11-04T18:34:41.273892] [INFO ] [MAIN ] ECCC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:34:41.274399] [DEBUG] [LOCAL ] ECCC: No existing data file found in primary storage. [2025-11-04T18:34:41.274443] [WARN ] [MAIN ] ECCC: No existing data found, performing full refresh instead. [2025-11-04T18:34:41.274551] [INFO ] [MAIN ] ECCC: Wiping existing primary data before full refresh. [2025-11-04T18:34:41.274592] [INFO ] [LOCAL ] ECCC: Starting deletion of primary data. [2025-11-04T18:34:41.274636] [DEBUG] [LOCAL ] ECCC: No primary data file found to delete. [2025-11-04T18:34:41.274716] [INFO ] [MAIN ] ECCC: Starting full primary refresh. [2025-11-04T18:34:41.392816] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T18:34:41.432853] [INFO ] [REMOTE ] EBR: Full fetch complete (62481 records). [2025-11-04T18:34:41.442001] [DEBUG] [VALIDATOR ] EBR: Validating primary data. [2025-11-04T18:34:41.772685] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-02. [2025-11-04T18:34:41.884114] [INFO ] [REMOTE ] ECCC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:34:42.083445] [DEBUG] [VALIDATOR ] EBR: All expected trading days are present. [2025-11-04T18:34:42.083636] [DEBUG] [VALIDATOR ] EBR: Primary data validation passed. [2025-11-04T18:34:42.084469] [INFO ] [MAIN ] EBR: Fetched primary data is valid. [2025-11-04T18:34:42.084519] [DEBUG] [LOCAL ] EBR: Saving primary data. [2025-11-04T18:34:42.085856] [DEBUG] [LOCAL ] EBR: Trimmed 62481 records to 43188 (keeping 200 days of data) [2025-11-04T18:34:42.318186] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T18:34:42.383225] [DEBUG] [LOCAL ] EBR: Primary data saved (43188 records). [2025-11-04T18:34:42.383405] [INFO ] [MAIN ] EBR: Full history saved. [2025-11-04T18:34:42.383525] [INFO ] [MAIN ] Update Progress: 684/2367 symbols completed. EBR took 34.3s. Estimated remaining: 3h 52m 54s. Rate: 7.2 symbols/min [2025-11-04T18:34:42.557991] [INFO ] [MAIN ] ECCU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:34:42.562793] [DEBUG] [LOCAL ] ECCU: Loaded 2030 existing records from primary storage. [2025-11-04T18:34:42.563026] [INFO ] [MAIN ] ECCU: Loaded 2030 existing records. [2025-11-04T18:34:42.574740] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-01. [2025-11-04T18:34:42.761479] [WARN ] [MAIN ] ECCU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:34:42.762125] [INFO ] [MAIN ] ECCU: Wiping existing primary data before full refresh. [2025-11-04T18:34:42.762357] [INFO ] [LOCAL ] ECCU: Starting deletion of primary data. [2025-11-04T18:34:42.781455] [DEBUG] [LOCAL ] ECCU: Deleted primary data file. [2025-11-04T18:34:42.781756] [INFO ] [MAIN ] ECCU: Starting full primary refresh. [2025-11-04T18:34:42.820456] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T18:34:43.177592] [INFO ] [REMOTE ] ECCU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:34:43.259885] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T18:34:43.367045] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-03. [2025-11-04T18:34:44.167128] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-03. [2025-11-04T18:34:44.514519] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T18:34:44.967016] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-04. [2025-11-04T18:34:45.165146] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-11-04T18:34:45.767054] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-01. [2025-11-04T18:34:46.129526] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-11-04T18:34:46.328492] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T18:34:46.567065] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-02. [2025-11-04T18:34:46.769056] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-02 (0.20 seconds). [2025-11-04T18:34:47.367053] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-05. [2025-11-04T18:34:48.167048] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-02. [2025-11-04T18:34:48.600539] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-11-04T18:34:48.683546] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:34:48.967096] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-04. [2025-11-04T18:34:49.767075] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-03. [2025-11-04T18:34:50.089886] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T18:34:50.169395] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T18:34:50.567097] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-03. [2025-11-04T18:34:50.830038] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-03 (0.26 seconds). [2025-11-04T18:34:51.367141] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-06. [2025-11-04T18:34:52.167119] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-05. [2025-11-04T18:34:52.594378] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-11-04T18:34:52.967049] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-04. [2025-11-04T18:34:53.045467] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T18:34:53.221355] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-04 (0.25 seconds). [2025-11-04T18:34:53.767065] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-07. [2025-11-04T18:34:54.567089] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-06. [2025-11-04T18:34:55.151476] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-11-04T18:34:55.367015] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-05. [2025-11-04T18:34:55.487211] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-11-04T18:34:55.767459] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-11-04T18:34:56.167054] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-06. [2025-11-04T18:34:56.405449] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-06 (0.24 seconds). [2025-11-04T18:34:56.967035] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-07. [2025-11-04T18:34:57.244017] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-07 (0.28 seconds). [2025-11-04T18:34:57.767034] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-04. [2025-11-04T18:34:58.230157] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-11-04T18:34:58.567046] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-05. [2025-11-04T18:34:58.881948] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-11-04T18:34:59.367067] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-06. [2025-11-04T18:34:59.636029] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-11-04T18:35:00.167053] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-07. [2025-11-04T18:35:00.967053] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-07. [2025-11-04T18:35:01.416171] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-11-04T18:35:01.684253] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-11-04T18:35:01.767074] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-08. [2025-11-04T18:35:02.078097] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-08 (0.31 seconds). [2025-11-04T18:35:02.567085] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-08. [2025-11-04T18:35:03.367088] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-08. [2025-11-04T18:35:03.529936] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-11-04T18:35:03.611345] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-08 (0.24 seconds). [2025-11-04T18:35:04.167039] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-09. [2025-11-04T18:35:04.410439] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-09 (0.24 seconds). [2025-11-04T18:35:04.967075] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-09. [2025-11-04T18:35:05.233602] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-09 (0.27 seconds). [2025-11-04T18:35:05.767012] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-10. [2025-11-04T18:35:06.060792] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-10 (0.29 seconds). [2025-11-04T18:35:06.567048] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-08. [2025-11-04T18:35:07.367043] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-10. [2025-11-04T18:35:07.741206] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-11-04T18:35:08.138322] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-11-04T18:35:08.167000] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-09. [2025-11-04T18:35:08.967069] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-09. [2025-11-04T18:35:09.466141] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T18:35:09.744590] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-11-04T18:35:09.767055] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-10. [2025-11-04T18:35:10.567023] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-11. [2025-11-04T18:35:11.137838] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T18:35:11.138467] [INFO ] [REMOTE ] ECCU: Full fetch complete (3905 records). [2025-11-04T18:35:11.138549] [DEBUG] [VALIDATOR ] ECCU: Validating primary data. [2025-11-04T18:35:11.369110] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-11. [2025-11-04T18:35:11.412576] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-11-04T18:35:11.591611] [DEBUG] [VALIDATOR ] ECCU: All expected trading days are present. [2025-11-04T18:35:11.591720] [DEBUG] [VALIDATOR ] ECCU: Primary data validation passed. [2025-11-04T18:35:11.591870] [INFO ] [MAIN ] ECCU: Fetched primary data is valid. [2025-11-04T18:35:11.591927] [DEBUG] [LOCAL ] ECCU: Saving primary data. [2025-11-04T18:35:11.592276] [DEBUG] [LOCAL ] ECCU: Trimmed 3905 records to 2012 (keeping 200 days of data) [2025-11-04T18:35:11.604595] [DEBUG] [LOCAL ] ECCU: Primary data saved (2012 records). [2025-11-04T18:35:11.604706] [INFO ] [MAIN ] ECCU: Full history saved. [2025-11-04T18:35:11.604790] [INFO ] [MAIN ] Update Progress: 685/2367 symbols completed. ECCU took 29.2s. Estimated remaining: 3h 53m 37s. Rate: 7.2 symbols/min [2025-11-04T18:35:11.776948] [INFO ] [MAIN ] ECCW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:35:11.777133] [DEBUG] [LOCAL ] ECCW: No existing data file found in primary storage. [2025-11-04T18:35:11.777170] [WARN ] [MAIN ] ECCW: No existing data found, performing full refresh instead. [2025-11-04T18:35:11.777277] [INFO ] [MAIN ] ECCW: Wiping existing primary data before full refresh. [2025-11-04T18:35:11.777317] [INFO ] [LOCAL ] ECCW: Starting deletion of primary data. [2025-11-04T18:35:11.777356] [DEBUG] [LOCAL ] ECCW: No primary data file found to delete. [2025-11-04T18:35:11.777386] [INFO ] [MAIN ] ECCW: Starting full primary refresh. [2025-11-04T18:35:11.855230] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-11 (0.49 seconds). [2025-11-04T18:35:11.880395] [INFO ] [REMOTE ] ECCC: Full fetch complete (1314 records). [2025-11-04T18:35:11.882891] [DEBUG] [VALIDATOR ] ECCC: Validating primary data. [2025-11-04T18:35:11.883579] [FATAL] [VALIDATOR ] ECCC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:35:11.894176] [FATAL] [MAIN ] ECCC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:35:11.904462] [WARN ] [MAIN ] ECCC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:35:11.933416] [INFO ] [LOCAL ] ECCC: Starting deletion of primary data. [2025-11-04T18:35:11.958169] [DEBUG] [LOCAL ] ECCC: No primary data file found to delete. [2025-11-04T18:35:11.959922] [INFO ] [LOCAL ] ECCC: Starting deletion of derivative data. [2025-11-04T18:35:11.994268] [DEBUG] [LOCAL ] ECCC: No derivative data file found to delete. [2025-11-04T18:35:11.994369] [INFO ] [MAIN ] Update Progress: 686/2367 symbols completed. ECCC took 30.9s. Estimated remaining: 3h 53m 9s. Rate: 7.2 symbols/min [2025-11-04T18:35:12.174234] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-10. [2025-11-04T18:35:12.254093] [INFO ] [REMOTE ] ECCW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:35:12.346430] [INFO ] [MAIN ] ECCX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:35:12.346622] [DEBUG] [LOCAL ] ECCX: No existing data file found in primary storage. [2025-11-04T18:35:12.346696] [WARN ] [MAIN ] ECCX: No existing data found, performing full refresh instead. [2025-11-04T18:35:12.346835] [INFO ] [MAIN ] ECCX: Wiping existing primary data before full refresh. [2025-11-04T18:35:12.346878] [INFO ] [LOCAL ] ECCX: Starting deletion of primary data. [2025-11-04T18:35:12.346920] [DEBUG] [LOCAL ] ECCX: No primary data file found to delete. [2025-11-04T18:35:12.346953] [INFO ] [MAIN ] ECCX: Starting full primary refresh. [2025-11-04T18:35:12.704715] [INFO ] [REMOTE ] ECCX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:35:12.967057] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-01. [2025-11-04T18:35:13.401737] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T18:35:13.410162] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-11-04T18:35:13.767072] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-01. [2025-11-04T18:35:14.310459] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-11-04T18:35:14.567080] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-02. [2025-11-04T18:35:14.799900] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-02 (0.23 seconds). [2025-11-04T18:35:15.367038] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-11. [2025-11-04T18:35:16.162260] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-11 (0.80 seconds). [2025-11-04T18:35:16.163872] [INFO ] [REMOTE ] ECAT: Full fetch complete (43056 records). [2025-11-04T18:35:16.163960] [DEBUG] [VALIDATOR ] ECAT: Validating primary data. [2025-11-04T18:35:16.175324] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-02. [2025-11-04T18:35:16.538532] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-11-04T18:35:16.538434] [DEBUG] [VALIDATOR ] ECAT: All expected trading days are present. [2025-11-04T18:35:16.539563] [DEBUG] [VALIDATOR ] ECAT: Primary data validation passed. [2025-11-04T18:35:16.540203] [INFO ] [MAIN ] ECAT: Fetched primary data is valid. [2025-11-04T18:35:16.540268] [DEBUG] [LOCAL ] ECAT: Saving primary data. [2025-11-04T18:35:16.541163] [DEBUG] [LOCAL ] ECAT: Trimmed 43056 records to 29296 (keeping 200 days of data) [2025-11-04T18:35:16.718990] [DEBUG] [LOCAL ] ECAT: Primary data saved (29296 records). [2025-11-04T18:35:16.719411] [INFO ] [MAIN ] ECAT: Full history saved. [2025-11-04T18:35:16.719513] [INFO ] [MAIN ] Update Progress: 687/2367 symbols completed. ECAT took 42.5s. Estimated remaining: 3h 52m 52s. Rate: 7.2 symbols/min [2025-11-04T18:35:16.887744] [INFO ] [MAIN ] ECG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:35:16.919237] [DEBUG] [LOCAL ] ECG: Loaded 32047 existing records from primary storage. [2025-11-04T18:35:16.919576] [INFO ] [MAIN ] ECG: Loaded 32047 existing records. [2025-11-04T18:35:16.972204] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-03. [2025-11-04T18:35:17.096290] [WARN ] [MAIN ] ECG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:35:17.096515] [INFO ] [MAIN ] ECG: Wiping existing primary data before full refresh. [2025-11-04T18:35:17.096598] [INFO ] [LOCAL ] ECG: Starting deletion of primary data. [2025-11-04T18:35:17.097144] [DEBUG] [LOCAL ] ECG: Deleted primary data file. [2025-11-04T18:35:17.097202] [INFO ] [MAIN ] ECG: Starting full primary refresh. [2025-11-04T18:35:17.344612] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-11-04T18:35:17.453543] [INFO ] [REMOTE ] ECG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:35:17.767053] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-04. [2025-11-04T18:35:18.054071] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-04 (0.29 seconds). [2025-11-04T18:35:18.567038] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-01. [2025-11-04T18:35:19.367049] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-03. [2025-11-04T18:35:19.648121] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-03 (0.28 seconds). [2025-11-04T18:35:19.725966] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T18:35:20.167105] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-04. [2025-11-04T18:35:20.427774] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-04 (0.26 seconds). [2025-11-04T18:35:20.967041] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-05. [2025-11-04T18:35:21.283714] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-11-04T18:35:21.767120] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-02. [2025-11-04T18:35:22.567150] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-06. [2025-11-04T18:35:22.812384] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T18:35:22.823779] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-06 (0.26 seconds). [2025-11-04T18:35:23.367051] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-07. [2025-11-04T18:35:23.836133] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-11-04T18:35:24.167042] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-11. [2025-11-04T18:35:24.967099] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-08. [2025-11-04T18:35:25.219874] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-08 (0.25 seconds). [2025-11-04T18:35:25.285343] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T18:35:25.286816] [INFO ] [REMOTE ] ECC: Full fetch complete (61480 records). [2025-11-04T18:35:25.286923] [DEBUG] [VALIDATOR ] ECC: Validating primary data. [2025-11-04T18:35:25.287608] [FATAL] [VALIDATOR ] ECC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:35:25.287716] [FATAL] [MAIN ] ECC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:35:25.287759] [WARN ] [MAIN ] ECC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:35:25.287796] [INFO ] [LOCAL ] ECC: Starting deletion of primary data. [2025-11-04T18:35:25.287850] [DEBUG] [LOCAL ] ECC: No primary data file found to delete. [2025-11-04T18:35:25.287884] [INFO ] [LOCAL ] ECC: Starting deletion of derivative data. [2025-11-04T18:35:25.288006] [DEBUG] [LOCAL ] ECC: Deleted derivative data file. [2025-11-04T18:35:25.288075] [INFO ] [MAIN ] Update Progress: 688/2367 symbols completed. ECC took 48.0s. Estimated remaining: 3h 52m 44s. Rate: 7.2 symbols/min [2025-11-04T18:35:25.453431] [INFO ] [MAIN ] ECL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:35:25.494683] [DEBUG] [LOCAL ] ECL: Loaded 46700 existing records from primary storage. [2025-11-04T18:35:25.494975] [INFO ] [MAIN ] ECL: Loaded 46700 existing records. [2025-11-04T18:35:25.667694] [WARN ] [MAIN ] ECL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:35:25.667921] [INFO ] [MAIN ] ECL: Wiping existing primary data before full refresh. [2025-11-04T18:35:25.667968] [INFO ] [LOCAL ] ECL: Starting deletion of primary data. [2025-11-04T18:35:25.668464] [DEBUG] [LOCAL ] ECL: Deleted primary data file. [2025-11-04T18:35:25.668502] [INFO ] [MAIN ] ECL: Starting full primary refresh. [2025-11-04T18:35:25.772073] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-05. [2025-11-04T18:35:26.025058] [INFO ] [REMOTE ] ECL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:35:26.140793] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-11-04T18:35:26.567069] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-06. [2025-11-04T18:35:26.803063] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-06 (0.24 seconds). [2025-11-04T18:35:27.367075] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-09. [2025-11-04T18:35:27.618719] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-09 (0.25 seconds). [2025-11-04T18:35:28.167041] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-03. [2025-11-04T18:35:28.967041] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-10. [2025-11-04T18:35:29.211427] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-10 (0.24 seconds). [2025-11-04T18:35:29.266524] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T18:35:29.767033] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-04. [2025-11-04T18:35:30.567064] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-01. [2025-11-04T18:35:30.877926] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-11-04T18:35:31.367041] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-05. [2025-11-04T18:35:32.040108] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T18:35:32.167053] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-07. [2025-11-04T18:35:32.263153] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T18:35:32.390942] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-07 (0.22 seconds). [2025-11-04T18:35:32.967013] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-08. [2025-11-04T18:35:33.215028] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-08 (0.25 seconds). [2025-11-04T18:35:33.767047] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-11. [2025-11-04T18:35:34.193554] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T18:35:34.194371] [INFO ] [REMOTE ] ECCW: Full fetch complete (1417 records). [2025-11-04T18:35:34.194454] [DEBUG] [VALIDATOR ] ECCW: Validating primary data. [2025-11-04T18:35:34.194986] [FATAL] [VALIDATOR ] ECCW: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:35:34.195108] [FATAL] [MAIN ] ECCW: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:35:34.195202] [WARN ] [MAIN ] ECCW: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:35:34.195290] [INFO ] [LOCAL ] ECCW: Starting deletion of primary data. [2025-11-04T18:35:34.195384] [DEBUG] [LOCAL ] ECCW: No primary data file found to delete. [2025-11-04T18:35:34.195425] [INFO ] [LOCAL ] ECCW: Starting deletion of derivative data. [2025-11-04T18:35:34.195468] [DEBUG] [LOCAL ] ECCW: No derivative data file found to delete. [2025-11-04T18:35:34.195522] [INFO ] [MAIN ] Update Progress: 689/2367 symbols completed. ECCW took 22.6s. Estimated remaining: 3h 52m 37s. Rate: 7.2 symbols/min [2025-11-04T18:35:34.397636] [INFO ] [MAIN ] ECO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:35:34.417858] [DEBUG] [LOCAL ] ECO: Loaded 18332 existing records from primary storage. [2025-11-04T18:35:34.418064] [INFO ] [MAIN ] ECO: Loaded 18332 existing records. [2025-11-04T18:35:34.574408] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-02. [2025-11-04T18:35:34.596576] [WARN ] [MAIN ] ECO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:35:34.596864] [INFO ] [MAIN ] ECO: Wiping existing primary data before full refresh. [2025-11-04T18:35:34.596981] [INFO ] [LOCAL ] ECO: Starting deletion of primary data. [2025-11-04T18:35:34.597175] [DEBUG] [LOCAL ] ECO: Deleted primary data file. [2025-11-04T18:35:34.597233] [INFO ] [MAIN ] ECO: Starting full primary refresh. [2025-11-04T18:35:34.948288] [INFO ] [REMOTE ] ECO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:35:35.367048] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-09. [2025-11-04T18:35:35.604062] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-09 (0.24 seconds). [2025-11-04T18:35:35.920012] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T18:35:36.167038] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-01. [2025-11-04T18:35:36.967055] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-06. [2025-11-04T18:35:37.332319] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T18:35:37.767112] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-02. [2025-11-04T18:35:37.820117] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T18:35:38.526699] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T18:35:38.567033] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-03. [2025-11-04T18:35:39.324233] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T18:35:39.367014] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-04. [2025-11-04T18:35:40.062758] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T18:35:40.167034] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-05. [2025-11-04T18:35:40.967028] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-07. [2025-11-04T18:35:40.982247] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T18:35:41.767044] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-10. [2025-11-04T18:35:42.021509] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-11-04T18:35:42.075967] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-10 (0.31 seconds). [2025-11-04T18:35:42.567042] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-11. [2025-11-04T18:35:42.905191] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T18:35:42.905956] [INFO ] [REMOTE ] ECCX: Full fetch complete (1423 records). [2025-11-04T18:35:42.906027] [DEBUG] [VALIDATOR ] ECCX: Validating primary data. [2025-11-04T18:35:42.906436] [FATAL] [VALIDATOR ] ECCX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:35:42.906523] [FATAL] [MAIN ] ECCX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:35:42.906567] [WARN ] [MAIN ] ECCX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:35:42.906606] [INFO ] [LOCAL ] ECCX: Starting deletion of primary data. [2025-11-04T18:35:42.906686] [DEBUG] [LOCAL ] ECCX: No primary data file found to delete. [2025-11-04T18:35:42.906777] [INFO ] [LOCAL ] ECCX: Starting deletion of derivative data. [2025-11-04T18:35:42.906841] [DEBUG] [LOCAL ] ECCX: No derivative data file found to delete. [2025-11-04T18:35:42.906897] [INFO ] [MAIN ] Update Progress: 690/2367 symbols completed. ECCX took 30.9s. Estimated remaining: 3h 52m 30s. Rate: 7.2 symbols/min [2025-11-04T18:35:43.123502] [INFO ] [MAIN ] ECVT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:35:43.164081] [DEBUG] [LOCAL ] ECVT: Loaded 42536 existing records from primary storage. [2025-11-04T18:35:43.164431] [INFO ] [MAIN ] ECVT: Loaded 42536 existing records. [2025-11-04T18:35:43.367614] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-08. [2025-11-04T18:35:43.395485] [WARN ] [MAIN ] ECVT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:35:43.396390] [INFO ] [MAIN ] ECVT: Wiping existing primary data before full refresh. [2025-11-04T18:35:43.396439] [INFO ] [LOCAL ] ECVT: Starting deletion of primary data. [2025-11-04T18:35:43.396675] [DEBUG] [LOCAL ] ECVT: Deleted primary data file. [2025-11-04T18:35:43.396768] [INFO ] [MAIN ] ECVT: Starting full primary refresh. [2025-11-04T18:35:43.748853] [INFO ] [REMOTE ] ECVT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:35:44.167113] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-06. [2025-11-04T18:35:44.490618] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-11-04T18:35:44.971817] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-09. [2025-11-04T18:35:44.973721] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T18:35:45.767080] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-03. [2025-11-04T18:35:45.854331] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T18:35:46.567111] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-07. [2025-11-04T18:35:47.117547] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-11-04T18:35:47.364518] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-11-04T18:35:47.368091] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-04. [2025-11-04T18:35:48.167110] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-01. [2025-11-04T18:35:48.929852] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T18:35:48.967003] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-05. [2025-11-04T18:35:49.368080] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T18:35:49.767030] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-10. [2025-11-04T18:35:50.442940] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T18:35:50.567020] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-02. [2025-11-04T18:35:50.842320] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-11-04T18:35:51.367049] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-08. [2025-11-04T18:35:51.588389] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T18:35:52.077101] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T18:35:52.167174] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-03. [2025-11-04T18:35:52.967048] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-09. [2025-11-04T18:35:53.305467] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T18:35:53.743319] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-11-04T18:35:53.767014] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-11. [2025-11-04T18:35:54.567043] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-06. [2025-11-04T18:35:54.759184] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T18:35:54.760835] [INFO ] [REMOTE ] ECG: Full fetch complete (50725 records). [2025-11-04T18:35:54.760912] [DEBUG] [VALIDATOR ] ECG: Validating primary data. [2025-11-04T18:35:55.145764] [DEBUG] [VALIDATOR ] ECG: All expected trading days are present. [2025-11-04T18:35:55.145842] [DEBUG] [VALIDATOR ] ECG: Primary data validation passed. [2025-11-04T18:35:55.146498] [INFO ] [MAIN ] ECG: Fetched primary data is valid. [2025-11-04T18:35:55.146540] [DEBUG] [LOCAL ] ECG: Saving primary data. [2025-11-04T18:35:55.147593] [DEBUG] [LOCAL ] ECG: Trimmed 50725 records to 32064 (keeping 200 days of data) [2025-11-04T18:35:55.328059] [DEBUG] [LOCAL ] ECG: Primary data saved (32064 records). [2025-11-04T18:35:55.328193] [INFO ] [MAIN ] ECG: Full history saved. [2025-11-04T18:35:55.328272] [INFO ] [MAIN ] Update Progress: 691/2367 symbols completed. ECG took 38.6s. Estimated remaining: 3h 52m 31s. Rate: 7.2 symbols/min [2025-11-04T18:35:55.372125] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-10. [2025-11-04T18:35:55.496985] [INFO ] [MAIN ] ED: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:35:55.549972] [DEBUG] [LOCAL ] ED: Loaded 52470 existing records from primary storage. [2025-11-04T18:35:55.550360] [INFO ] [MAIN ] ED: Loaded 52470 existing records. [2025-11-04T18:35:55.736517] [WARN ] [MAIN ] ED: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:35:55.736964] [INFO ] [MAIN ] ED: Wiping existing primary data before full refresh. [2025-11-04T18:35:55.737077] [INFO ] [LOCAL ] ED: Starting deletion of primary data. [2025-11-04T18:35:55.737492] [DEBUG] [LOCAL ] ED: Deleted primary data file. [2025-11-04T18:35:55.737574] [INFO ] [MAIN ] ED: Starting full primary refresh. [2025-11-04T18:35:56.080688] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-11-04T18:35:56.157183] [INFO ] [REMOTE ] ED: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:35:56.169778] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-04. [2025-11-04T18:35:56.173568] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T18:35:56.967045] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-11. [2025-11-04T18:35:57.242181] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T18:35:57.767060] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-01. [2025-11-04T18:35:57.874755] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T18:35:57.876049] [INFO ] [REMOTE ] ECO: Full fetch complete (28430 records). [2025-11-04T18:35:57.876117] [DEBUG] [VALIDATOR ] ECO: Validating primary data. [2025-11-04T18:35:58.253388] [DEBUG] [VALIDATOR ] ECO: All expected trading days are present. [2025-11-04T18:35:58.253468] [DEBUG] [VALIDATOR ] ECO: Primary data validation passed. [2025-11-04T18:35:58.253958] [INFO ] [MAIN ] ECO: Fetched primary data is valid. [2025-11-04T18:35:58.254019] [DEBUG] [LOCAL ] ECO: Saving primary data. [2025-11-04T18:35:58.254845] [DEBUG] [LOCAL ] ECO: Trimmed 28430 records to 18477 (keeping 200 days of data) [2025-11-04T18:35:58.370766] [DEBUG] [LOCAL ] ECO: Primary data saved (18477 records). [2025-11-04T18:35:58.370901] [INFO ] [MAIN ] ECO: Full history saved. [2025-11-04T18:35:58.370994] [INFO ] [MAIN ] Update Progress: 692/2367 symbols completed. ECO took 24.2s. Estimated remaining: 3h 52m 10s. Rate: 7.2 symbols/min [2025-11-04T18:35:58.546977] [INFO ] [MAIN ] EDD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:35:58.572174] [DEBUG] [LOCAL ] EDD: Loaded 26811 existing records from primary storage. [2025-11-04T18:35:58.571603] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-05. [2025-11-04T18:35:58.573813] [INFO ] [MAIN ] EDD: Loaded 26811 existing records. [2025-11-04T18:35:58.757903] [WARN ] [MAIN ] EDD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:35:58.758140] [INFO ] [MAIN ] EDD: Wiping existing primary data before full refresh. [2025-11-04T18:35:58.758187] [INFO ] [LOCAL ] EDD: Starting deletion of primary data. [2025-11-04T18:35:58.758352] [DEBUG] [LOCAL ] EDD: Deleted primary data file. [2025-11-04T18:35:58.758390] [INFO ] [MAIN ] EDD: Starting full primary refresh. [2025-11-04T18:35:59.129738] [INFO ] [REMOTE ] EDD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:35:59.367013] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-01. [2025-11-04T18:35:59.428537] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-11-04T18:35:59.887505] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:36:00.167056] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-06. [2025-11-04T18:36:00.416959] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-11-04T18:36:00.967052] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-02. [2025-11-04T18:36:01.767088] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-02. [2025-11-04T18:36:02.567080] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-07. [2025-11-04T18:36:02.951440] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T18:36:03.367129] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-03. [2025-11-04T18:36:04.033064] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-06 (3.87 seconds). [2025-11-04T18:36:04.167018] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-07. [2025-11-04T18:36:04.213096] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-11-04T18:36:04.632796] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T18:36:04.670075] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-02 (3.70 seconds). [2025-11-04T18:36:04.967098] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-03. [2025-11-04T18:36:05.278994] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T18:36:05.767037] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-08. [2025-11-04T18:36:05.894091] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T18:36:06.567044] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-04. [2025-11-04T18:36:06.903168] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T18:36:07.367016] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-09. [2025-11-04T18:36:07.476411] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-11-04T18:36:08.167039] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-04. [2025-11-04T18:36:08.439496] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-11-04T18:36:08.967116] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-08. [2025-11-04T18:36:09.457418] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T18:36:09.767104] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-05. [2025-11-04T18:36:10.185217] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T18:36:10.567096] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-09. [2025-11-04T18:36:11.229891] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T18:36:11.367051] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-05. [2025-11-04T18:36:11.916664] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T18:36:12.167121] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-10. [2025-11-04T18:36:12.429928] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T18:36:12.967090] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-10. [2025-11-04T18:36:13.553621] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T18:36:13.767054] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-06. [2025-11-04T18:36:14.066670] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T18:36:14.567023] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-11. [2025-11-04T18:36:14.639031] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T18:36:15.367047] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-11. [2025-11-04T18:36:15.457716] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T18:36:15.459519] [INFO ] [REMOTE ] ECVT: Full fetch complete (60758 records). [2025-11-04T18:36:15.459608] [DEBUG] [VALIDATOR ] ECVT: Validating primary data. [2025-11-04T18:36:15.884355] [DEBUG] [VALIDATOR ] ECVT: All expected trading days are present. [2025-11-04T18:36:15.884475] [DEBUG] [VALIDATOR ] ECVT: Primary data validation passed. [2025-11-04T18:36:15.885314] [INFO ] [MAIN ] ECVT: Fetched primary data is valid. [2025-11-04T18:36:15.885381] [DEBUG] [LOCAL ] ECVT: Saving primary data. [2025-11-04T18:36:15.886674] [DEBUG] [LOCAL ] ECVT: Trimmed 60758 records to 42606 (keeping 200 days of data) [2025-11-04T18:36:16.126209] [DEBUG] [LOCAL ] ECVT: Primary data saved (42606 records). [2025-11-04T18:36:16.126329] [INFO ] [MAIN ] ECVT: Full history saved. [2025-11-04T18:36:16.126443] [INFO ] [MAIN ] Update Progress: 693/2367 symbols completed. ECVT took 33.2s. Estimated remaining: 3h 52m 25s. Rate: 7.2 symbols/min [2025-11-04T18:36:16.172394] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-07. [2025-11-04T18:36:16.311407] [INFO ] [MAIN ] EDF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:36:16.324605] [DEBUG] [LOCAL ] EDF: Loaded 12652 existing records from primary storage. [2025-11-04T18:36:16.324894] [INFO ] [MAIN ] EDF: Loaded 12652 existing records. [2025-11-04T18:36:16.514423] [WARN ] [MAIN ] EDF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:36:16.514688] [INFO ] [MAIN ] EDF: Wiping existing primary data before full refresh. [2025-11-04T18:36:16.514749] [INFO ] [LOCAL ] EDF: Starting deletion of primary data. [2025-11-04T18:36:16.514954] [DEBUG] [LOCAL ] EDF: Deleted primary data file. [2025-11-04T18:36:16.514993] [INFO ] [MAIN ] EDF: Starting full primary refresh. [2025-11-04T18:36:16.824822] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-11 (1.46 seconds). [2025-11-04T18:36:16.855080] [INFO ] [REMOTE ] ECL: Full fetch complete (68523 records). [2025-11-04T18:36:16.872412] [DEBUG] [VALIDATOR ] ECL: Validating primary data. [2025-11-04T18:36:16.957137] [INFO ] [REMOTE ] EDF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:36:16.985616] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-01. [2025-11-04T18:36:17.374965] [DEBUG] [VALIDATOR ] ECL: All expected trading days are present. [2025-11-04T18:36:17.376760] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T18:36:17.376985] [DEBUG] [VALIDATOR ] ECL: Primary data validation passed. [2025-11-04T18:36:17.386167] [INFO ] [MAIN ] ECL: Fetched primary data is valid. [2025-11-04T18:36:17.386762] [DEBUG] [LOCAL ] ECL: Saving primary data. [2025-11-04T18:36:17.388099] [DEBUG] [LOCAL ] ECL: Trimmed 68523 records to 46728 (keeping 200 days of data) [2025-11-04T18:36:17.677100] [DEBUG] [LOCAL ] ECL: Primary data saved (46728 records). [2025-11-04T18:36:17.677242] [INFO ] [MAIN ] ECL: Full history saved. [2025-11-04T18:36:17.677324] [INFO ] [MAIN ] Update Progress: 694/2367 symbols completed. ECL took 52.4s. Estimated remaining: 3h 52m 0s. Rate: 7.2 symbols/min [2025-11-04T18:36:17.776356] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-06. [2025-11-04T18:36:17.882319] [INFO ] [MAIN ] EDN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:36:17.902576] [DEBUG] [LOCAL ] EDN: Loaded 19958 existing records from primary storage. [2025-11-04T18:36:17.902869] [INFO ] [MAIN ] EDN: Loaded 19958 existing records. [2025-11-04T18:36:18.098721] [WARN ] [MAIN ] EDN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:36:18.100133] [INFO ] [MAIN ] EDN: Wiping existing primary data before full refresh. [2025-11-04T18:36:18.102302] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T18:36:18.102402] [INFO ] [LOCAL ] EDN: Starting deletion of primary data. [2025-11-04T18:36:18.107372] [DEBUG] [LOCAL ] EDN: Deleted primary data file. [2025-11-04T18:36:18.107453] [INFO ] [MAIN ] EDN: Starting full primary refresh. [2025-11-04T18:36:18.491343] [INFO ] [REMOTE ] EDN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:36:18.567065] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-01. [2025-11-04T18:36:19.146184] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-11-04T18:36:19.367140] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-02. [2025-11-04T18:36:19.889506] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T18:36:20.003873] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T18:36:20.167127] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-08. [2025-11-04T18:36:20.967038] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-02. [2025-11-04T18:36:21.004785] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-11-04T18:36:21.633907] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-11-04T18:36:21.767023] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-07. [2025-11-04T18:36:22.567087] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-09. [2025-11-04T18:36:23.075833] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T18:36:23.367006] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-08. [2025-11-04T18:36:23.429976] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-11-04T18:36:24.167084] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-10. [2025-11-04T18:36:24.726352] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-11-04T18:36:24.967037] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-09. [2025-11-04T18:36:24.985517] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T18:36:25.767088] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-11. [2025-11-04T18:36:26.245856] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T18:36:26.410924] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T18:36:26.412313] [INFO ] [REMOTE ] EDD: Full fetch complete (38850 records). [2025-11-04T18:36:26.412388] [DEBUG] [VALIDATOR ] EDD: Validating primary data. [2025-11-04T18:36:26.586938] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-03. [2025-11-04T18:36:26.785422] [DEBUG] [VALIDATOR ] EDD: All expected trading days are present. [2025-11-04T18:36:26.785504] [DEBUG] [VALIDATOR ] EDD: Primary data validation passed. [2025-11-04T18:36:26.786074] [INFO ] [MAIN ] EDD: Fetched primary data is valid. [2025-11-04T18:36:26.786169] [DEBUG] [LOCAL ] EDD: Saving primary data. [2025-11-04T18:36:26.787065] [DEBUG] [LOCAL ] EDD: Trimmed 38850 records to 26767 (keeping 200 days of data) [2025-11-04T18:36:26.935888] [DEBUG] [LOCAL ] EDD: Primary data saved (26767 records). [2025-11-04T18:36:26.936024] [INFO ] [MAIN ] EDD: Full history saved. [2025-11-04T18:36:26.936100] [INFO ] [MAIN ] Update Progress: 695/2367 symbols completed. EDD took 28.6s. Estimated remaining: 3h 51m 54s. Rate: 7.2 symbols/min [2025-11-04T18:36:27.108036] [INFO ] [MAIN ] EDU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:36:27.148042] [DEBUG] [LOCAL ] EDU: Loaded 44578 existing records from primary storage. [2025-11-04T18:36:27.148356] [INFO ] [MAIN ] EDU: Loaded 44578 existing records. [2025-11-04T18:36:27.307310] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-11-04T18:36:27.344979] [WARN ] [MAIN ] EDU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:36:27.345156] [INFO ] [MAIN ] EDU: Wiping existing primary data before full refresh. [2025-11-04T18:36:27.345227] [INFO ] [LOCAL ] EDU: Starting deletion of primary data. [2025-11-04T18:36:27.345436] [DEBUG] [LOCAL ] EDU: Deleted primary data file. [2025-11-04T18:36:27.345492] [INFO ] [MAIN ] EDU: Starting full primary refresh. [2025-11-04T18:36:27.374117] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-10. [2025-11-04T18:36:27.692370] [INFO ] [REMOTE ] EDU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:36:28.167120] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-03. [2025-11-04T18:36:28.812818] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T18:36:28.861628] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-11-04T18:36:28.967009] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-04. [2025-11-04T18:36:29.702004] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T18:36:29.767007] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-05. [2025-11-04T18:36:30.567074] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-11. [2025-11-04T18:36:30.590164] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T18:36:30.808707] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-11 (0.24 seconds). [2025-11-04T18:36:30.810617] [INFO ] [REMOTE ] ED: Full fetch complete (76285 records). [2025-11-04T18:36:30.810731] [DEBUG] [VALIDATOR ] ED: Validating primary data. [2025-11-04T18:36:31.192723] [DEBUG] [VALIDATOR ] ED: All expected trading days are present. [2025-11-04T18:36:31.192809] [DEBUG] [VALIDATOR ] ED: Primary data validation passed. [2025-11-04T18:36:31.193631] [INFO ] [MAIN ] ED: Fetched primary data is valid. [2025-11-04T18:36:31.193707] [DEBUG] [LOCAL ] ED: Saving primary data. [2025-11-04T18:36:31.195065] [DEBUG] [LOCAL ] ED: Trimmed 76285 records to 52467 (keeping 200 days of data) [2025-11-04T18:36:31.374074] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-01. [2025-11-04T18:36:31.518909] [DEBUG] [LOCAL ] ED: Primary data saved (52467 records). [2025-11-04T18:36:31.519037] [INFO ] [MAIN ] ED: Full history saved. [2025-11-04T18:36:31.519114] [INFO ] [MAIN ] Update Progress: 696/2367 symbols completed. ED took 36.2s. Estimated remaining: 3h 51m 37s. Rate: 7.2 symbols/min [2025-11-04T18:36:31.685534] [INFO ] [MAIN ] EE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:36:31.715353] [DEBUG] [LOCAL ] EE: Loaded 32493 existing records from primary storage. [2025-11-04T18:36:31.715616] [INFO ] [MAIN ] EE: Loaded 32493 existing records. [2025-11-04T18:36:31.890769] [WARN ] [MAIN ] EE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:36:31.890995] [INFO ] [MAIN ] EE: Wiping existing primary data before full refresh. [2025-11-04T18:36:31.891042] [INFO ] [LOCAL ] EE: Starting deletion of primary data. [2025-11-04T18:36:31.891238] [DEBUG] [LOCAL ] EE: Deleted primary data file. [2025-11-04T18:36:31.891295] [INFO ] [MAIN ] EE: Starting full primary refresh. [2025-11-04T18:36:32.172183] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-06. [2025-11-04T18:36:32.242874] [INFO ] [REMOTE ] EE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:36:32.671793] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T18:36:32.846159] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-11-04T18:36:32.967072] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-02. [2025-11-04T18:36:33.767062] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-01. [2025-11-04T18:36:34.112197] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T18:36:34.567046] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-04. [2025-11-04T18:36:34.694632] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-11-04T18:36:35.221890] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-11-04T18:36:35.367052] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-02. [2025-11-04T18:36:36.073184] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T18:36:36.167063] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-03. [2025-11-04T18:36:36.967028] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-03. [2025-11-04T18:36:37.098889] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T18:36:37.767093] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-04. [2025-11-04T18:36:38.115726] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T18:36:38.567081] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-07. [2025-11-04T18:36:38.694582] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-11-04T18:36:39.305188] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T18:36:39.367034] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-05. [2025-11-04T18:36:39.981763] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-11-04T18:36:40.167101] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-08. [2025-11-04T18:36:40.920096] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T18:36:40.967077] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-04. [2025-11-04T18:36:41.767063] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-05. [2025-11-04T18:36:42.256202] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T18:36:42.574592] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-09. [2025-11-04T18:36:42.580492] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T18:36:43.369051] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-05. [2025-11-04T18:36:43.371745] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-11-04T18:36:44.167065] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-10. [2025-11-04T18:36:44.801095] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T18:36:44.967067] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-06. [2025-11-04T18:36:45.065632] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-11-04T18:36:45.767022] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-06. [2025-11-04T18:36:45.857754] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T18:36:46.489978] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-11-04T18:36:46.567027] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-07. [2025-11-04T18:36:47.162851] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T18:36:47.367033] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-11. [2025-11-04T18:36:48.167064] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-08. [2025-11-04T18:36:48.243471] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-11 (0.88 seconds). [2025-11-04T18:36:48.244837] [INFO ] [REMOTE ] EDN: Full fetch complete (29793 records). [2025-11-04T18:36:48.244941] [DEBUG] [VALIDATOR ] EDN: Validating primary data. [2025-11-04T18:36:48.628278] [DEBUG] [VALIDATOR ] EDN: All expected trading days are present. [2025-11-04T18:36:48.628424] [DEBUG] [VALIDATOR ] EDN: Primary data validation passed. [2025-11-04T18:36:48.628941] [INFO ] [MAIN ] EDN: Fetched primary data is valid. [2025-11-04T18:36:48.629001] [DEBUG] [LOCAL ] EDN: Saving primary data. [2025-11-04T18:36:48.629754] [DEBUG] [LOCAL ] EDN: Trimmed 29793 records to 20104 (keeping 200 days of data) [2025-11-04T18:36:48.745530] [DEBUG] [LOCAL ] EDN: Primary data saved (20104 records). [2025-11-04T18:36:48.745703] [INFO ] [MAIN ] EDN: Full history saved. [2025-11-04T18:36:48.745826] [INFO ] [MAIN ] Update Progress: 697/2367 symbols completed. EDN took 31.1s. Estimated remaining: 3h 51m 50s. Rate: 7.2 symbols/min [2025-11-04T18:36:48.837807] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T18:36:48.931945] [INFO ] [MAIN ] EEA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:36:48.932127] [DEBUG] [LOCAL ] EEA: No existing data file found in primary storage. [2025-11-04T18:36:48.932165] [WARN ] [MAIN ] EEA: No existing data found, performing full refresh instead. [2025-11-04T18:36:48.932299] [INFO ] [MAIN ] EEA: Wiping existing primary data before full refresh. [2025-11-04T18:36:48.932374] [INFO ] [LOCAL ] EEA: Starting deletion of primary data. [2025-11-04T18:36:48.932435] [DEBUG] [LOCAL ] EEA: No primary data file found to delete. [2025-11-04T18:36:48.932469] [INFO ] [MAIN ] EEA: Starting full primary refresh. [2025-11-04T18:36:48.972198] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-09. [2025-11-04T18:36:49.280846] [INFO ] [REMOTE ] EEA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:36:49.654213] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T18:36:49.767107] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-07. [2025-11-04T18:36:50.567092] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-10. [2025-11-04T18:36:50.895933] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T18:36:51.150090] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-11-04T18:36:51.367065] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-11. [2025-11-04T18:36:51.940737] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T18:36:51.941816] [INFO ] [REMOTE ] EDF: Full fetch complete (19914 records). [2025-11-04T18:36:51.941887] [DEBUG] [VALIDATOR ] EDF: Validating primary data. [2025-11-04T18:36:52.172191] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-08. [2025-11-04T18:36:52.327944] [DEBUG] [VALIDATOR ] EDF: All expected trading days are present. [2025-11-04T18:36:52.328025] [DEBUG] [VALIDATOR ] EDF: Primary data validation passed. [2025-11-04T18:36:52.328465] [INFO ] [MAIN ] EDF: Fetched primary data is valid. [2025-11-04T18:36:52.328506] [DEBUG] [LOCAL ] EDF: Saving primary data. [2025-11-04T18:36:52.329412] [DEBUG] [LOCAL ] EDF: Trimmed 19914 records to 12701 (keeping 200 days of data) [2025-11-04T18:36:52.405335] [DEBUG] [LOCAL ] EDF: Primary data saved (12701 records). [2025-11-04T18:36:52.405492] [INFO ] [MAIN ] EDF: Full history saved. [2025-11-04T18:36:52.405574] [INFO ] [MAIN ] Update Progress: 698/2367 symbols completed. EDF took 36.3s. Estimated remaining: 3h 51m 30s. Rate: 7.2 symbols/min [2025-11-04T18:36:52.595022] [INFO ] [MAIN ] EEX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:36:52.595214] [DEBUG] [LOCAL ] EEX: No existing data file found in primary storage. [2025-11-04T18:36:52.595267] [WARN ] [MAIN ] EEX: No existing data found, performing full refresh instead. [2025-11-04T18:36:52.595372] [INFO ] [MAIN ] EEX: Wiping existing primary data before full refresh. [2025-11-04T18:36:52.595413] [INFO ] [LOCAL ] EEX: Starting deletion of primary data. [2025-11-04T18:36:52.595454] [DEBUG] [LOCAL ] EEX: No primary data file found to delete. [2025-11-04T18:36:52.595485] [INFO ] [MAIN ] EEX: Starting full primary refresh. [2025-11-04T18:36:52.951065] [INFO ] [REMOTE ] EEX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:36:52.967060] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-01. [2025-11-04T18:36:53.417043] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T18:36:53.767016] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-09. [2025-11-04T18:36:53.919527] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T18:36:54.567014] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-01. [2025-11-04T18:36:54.594294] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-11-04T18:36:55.305496] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-11-04T18:36:55.367032] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-10. [2025-11-04T18:36:56.167077] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-06. [2025-11-04T18:36:56.490039] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-11-04T18:36:56.967066] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-02. [2025-11-04T18:36:57.257912] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-11-04T18:36:57.404899] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T18:36:57.767050] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-03. [2025-11-04T18:36:58.057662] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-11-04T18:36:58.567051] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-11. [2025-11-04T18:36:59.367044] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-07. [2025-11-04T18:36:59.433411] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T18:36:59.434964] [INFO ] [REMOTE ] EE: Full fetch complete (44981 records). [2025-11-04T18:36:59.435077] [DEBUG] [VALIDATOR ] EE: Validating primary data. [2025-11-04T18:36:59.828272] [DEBUG] [VALIDATOR ] EE: All expected trading days are present. [2025-11-04T18:36:59.828363] [DEBUG] [VALIDATOR ] EE: Primary data validation passed. [2025-11-04T18:36:59.829048] [INFO ] [MAIN ] EE: Fetched primary data is valid. [2025-11-04T18:36:59.829113] [DEBUG] [LOCAL ] EE: Saving primary data. [2025-11-04T18:36:59.830211] [DEBUG] [LOCAL ] EE: Trimmed 44981 records to 32408 (keeping 200 days of data) [2025-11-04T18:37:00.021371] [DEBUG] [LOCAL ] EE: Primary data saved (32408 records). [2025-11-04T18:37:00.021527] [INFO ] [MAIN ] EE: Full history saved. [2025-11-04T18:37:00.021610] [INFO ] [MAIN ] Update Progress: 699/2367 symbols completed. EE took 28.5s. Estimated remaining: 3h 51m 20s. Rate: 7.2 symbols/min [2025-11-04T18:37:00.170714] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-02. [2025-11-04T18:37:00.189729] [INFO ] [MAIN ] EFC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:37:00.231128] [DEBUG] [LOCAL ] EFC: Loaded 46079 existing records from primary storage. [2025-11-04T18:37:00.231424] [INFO ] [MAIN ] EFC: Loaded 46079 existing records. [2025-11-04T18:37:00.408344] [WARN ] [MAIN ] EFC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:37:00.408571] [INFO ] [MAIN ] EFC: Wiping existing primary data before full refresh. [2025-11-04T18:37:00.408618] [INFO ] [LOCAL ] EFC: Starting deletion of primary data. [2025-11-04T18:37:00.408833] [DEBUG] [LOCAL ] EFC: Deleted primary data file. [2025-11-04T18:37:00.408884] [INFO ] [MAIN ] EFC: Starting full primary refresh. [2025-11-04T18:37:00.755587] [INFO ] [REMOTE ] EFC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:37:00.972275] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-01. [2025-11-04T18:37:00.976111] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-11-04T18:37:00.998518] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T18:37:01.767040] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-03. [2025-11-04T18:37:02.504295] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T18:37:02.562377] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-11-04T18:37:02.571196] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-02. [2025-11-04T18:37:03.367071] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-08. [2025-11-04T18:37:03.855598] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T18:37:04.167043] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-03. [2025-11-04T18:37:04.530594] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T18:37:04.967031] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-09. [2025-11-04T18:37:05.342199] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-11-04T18:37:05.767062] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-04. [2025-11-04T18:37:06.090219] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T18:37:06.567034] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-10. [2025-11-04T18:37:06.589151] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T18:37:07.367090] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-04. [2025-11-04T18:37:07.812231] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-11-04T18:37:08.088219] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-11-04T18:37:08.167050] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-05. [2025-11-04T18:37:08.869046] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-11-04T18:37:08.967079] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-04. [2025-11-04T18:37:09.767088] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-11. [2025-11-04T18:37:10.393614] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T18:37:10.567103] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-05. [2025-11-04T18:37:10.817555] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T18:37:10.819407] [INFO ] [REMOTE ] EDU: Full fetch complete (67128 records). [2025-11-04T18:37:10.819524] [DEBUG] [VALIDATOR ] EDU: Validating primary data. [2025-11-04T18:37:11.235603] [DEBUG] [VALIDATOR ] EDU: All expected trading days are present. [2025-11-04T18:37:11.236318] [DEBUG] [VALIDATOR ] EDU: Primary data validation passed. [2025-11-04T18:37:11.238601] [INFO ] [MAIN ] EDU: Fetched primary data is valid. [2025-11-04T18:37:11.238740] [DEBUG] [LOCAL ] EDU: Saving primary data. [2025-11-04T18:37:11.237233] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T18:37:11.244055] [DEBUG] [LOCAL ] EDU: Trimmed 67128 records to 44630 (keeping 200 days of data) [2025-11-04T18:37:11.367703] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-05. [2025-11-04T18:37:11.522268] [DEBUG] [LOCAL ] EDU: Primary data saved (44630 records). [2025-11-04T18:37:11.522456] [INFO ] [MAIN ] EDU: Full history saved. [2025-11-04T18:37:11.522559] [INFO ] [MAIN ] Update Progress: 700/2367 symbols completed. EDU took 44.6s. Estimated remaining: 3h 51m 20s. Rate: 7.2 symbols/min [2025-11-04T18:37:11.702417] [INFO ] [MAIN ] EFR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:37:11.721741] [DEBUG] [LOCAL ] EFR: Loaded 14364 existing records from primary storage. [2025-11-04T18:37:11.722133] [INFO ] [MAIN ] EFR: Loaded 14364 existing records. [2025-11-04T18:37:11.916606] [WARN ] [MAIN ] EFR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:37:11.916983] [INFO ] [MAIN ] EFR: Wiping existing primary data before full refresh. [2025-11-04T18:37:11.917038] [INFO ] [LOCAL ] EFR: Starting deletion of primary data. [2025-11-04T18:37:11.917234] [DEBUG] [LOCAL ] EFR: Deleted primary data file. [2025-11-04T18:37:11.917275] [INFO ] [MAIN ] EFR: Starting full primary refresh. [2025-11-04T18:37:12.172223] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-06. [2025-11-04T18:37:12.284164] [INFO ] [REMOTE ] EFR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:37:12.794987] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-11-04T18:37:12.908868] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T18:37:12.967001] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-06. [2025-11-04T18:37:13.767088] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-01. [2025-11-04T18:37:14.306638] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T18:37:14.567021] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-07. [2025-11-04T18:37:14.632226] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T18:37:15.367071] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-02. [2025-11-04T18:37:15.867968] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T18:37:15.913794] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T18:37:16.167069] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-06. [2025-11-04T18:37:16.509235] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-11-04T18:37:16.967092] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-07. [2025-11-04T18:37:17.358711] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-11-04T18:37:17.767049] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-08. [2025-11-04T18:37:18.188977] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-11-04T18:37:18.567046] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-03. [2025-11-04T18:37:19.098896] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-11-04T18:37:19.367020] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-09. [2025-11-04T18:37:19.661877] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-09 (0.29 seconds). [2025-11-04T18:37:20.167039] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-04. [2025-11-04T18:37:20.844622] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T18:37:20.967015] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-07. [2025-11-04T18:37:21.616927] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-11-04T18:37:21.767122] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-05. [2025-11-04T18:37:22.384866] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-11-04T18:37:22.567086] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-08. [2025-11-04T18:37:23.367074] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-06. [2025-11-04T18:37:23.994206] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-11-04T18:37:23.995707] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-11-04T18:37:24.167072] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-08. [2025-11-04T18:37:24.638350] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-11-04T18:37:24.967072] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-10. [2025-11-04T18:37:25.292548] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-10 (0.33 seconds). [2025-11-04T18:37:25.767086] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-07. [2025-11-04T18:37:26.542692] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-11-04T18:37:26.566997] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-08. [2025-11-04T18:37:27.201557] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-11-04T18:37:27.367045] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-09. [2025-11-04T18:37:28.167082] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-11. [2025-11-04T18:37:28.720260] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T18:37:28.721011] [INFO ] [REMOTE ] EEA: Full fetch complete (3729 records). [2025-11-04T18:37:28.721078] [DEBUG] [VALIDATOR ] EEA: Validating primary data. [2025-11-04T18:37:28.918973] [WARN ] [VALIDATOR ] EEA: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-11-04T18:37:28.931291] [FATAL] [MAIN ] EEA: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:37:28.931462] [WARN ] [MAIN ] EEA: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:37:28.932058] [INFO ] [LOCAL ] EEA: Starting deletion of primary data. [2025-11-04T18:37:28.933198] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-11-04T18:37:28.933334] [DEBUG] [LOCAL ] EEA: No primary data file found to delete. [2025-11-04T18:37:28.943773] [INFO ] [LOCAL ] EEA: Starting deletion of derivative data. [2025-11-04T18:37:28.943917] [DEBUG] [LOCAL ] EEA: No derivative data file found to delete. [2025-11-04T18:37:28.944333] [INFO ] [MAIN ] Update Progress: 701/2367 symbols completed. EEA took 40.2s. Estimated remaining: 3h 51m 33s. Rate: 7.2 symbols/min [2025-11-04T18:37:28.975567] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-10. [2025-11-04T18:37:29.152100] [INFO ] [MAIN ] EFT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:37:29.170815] [DEBUG] [LOCAL ] EFT: Loaded 12737 existing records from primary storage. [2025-11-04T18:37:29.171196] [INFO ] [MAIN ] EFT: Loaded 12737 existing records. [2025-11-04T18:37:29.353572] [WARN ] [MAIN ] EFT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:37:29.353905] [INFO ] [MAIN ] EFT: Wiping existing primary data before full refresh. [2025-11-04T18:37:29.353956] [INFO ] [LOCAL ] EFT: Starting deletion of primary data. [2025-11-04T18:37:29.354201] [DEBUG] [LOCAL ] EFT: Deleted primary data file. [2025-11-04T18:37:29.354259] [INFO ] [MAIN ] EFT: Starting full primary refresh. [2025-11-04T18:37:29.723511] [INFO ] [REMOTE ] EFT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:37:29.767059] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-09. [2025-11-04T18:37:30.406444] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T18:37:30.426402] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-11-04T18:37:30.567038] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-11. [2025-11-04T18:37:31.367124] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-09. [2025-11-04T18:37:31.615263] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T18:37:31.617401] [INFO ] [REMOTE ] EFC: Full fetch complete (66915 records). [2025-11-04T18:37:31.617557] [DEBUG] [VALIDATOR ] EFC: Validating primary data. [2025-11-04T18:37:31.995433] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T18:37:32.088806] [DEBUG] [VALIDATOR ] EFC: All expected trading days are present. [2025-11-04T18:37:32.088935] [DEBUG] [VALIDATOR ] EFC: Primary data validation passed. [2025-11-04T18:37:32.089721] [INFO ] [MAIN ] EFC: Fetched primary data is valid. [2025-11-04T18:37:32.089782] [DEBUG] [LOCAL ] EFC: Saving primary data. [2025-11-04T18:37:32.090998] [DEBUG] [LOCAL ] EFC: Trimmed 66915 records to 46054 (keeping 200 days of data) [2025-11-04T18:37:32.184533] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-10. [2025-11-04T18:37:32.388549] [DEBUG] [LOCAL ] EFC: Primary data saved (46054 records). [2025-11-04T18:37:32.388732] [INFO ] [MAIN ] EFC: Full history saved. [2025-11-04T18:37:32.388842] [INFO ] [MAIN ] Update Progress: 702/2367 symbols completed. EFC took 32.4s. Estimated remaining: 3h 51m 13s. Rate: 7.2 symbols/min [2025-11-04T18:37:32.570954] [INFO ] [MAIN ] EFX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:37:32.609077] [DEBUG] [LOCAL ] EFX: Loaded 44535 existing records from primary storage. [2025-11-04T18:37:32.609914] [INFO ] [MAIN ] EFX: Loaded 44535 existing records. [2025-11-04T18:37:32.799614] [WARN ] [MAIN ] EFX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:37:32.804925] [INFO ] [MAIN ] EFX: Wiping existing primary data before full refresh. [2025-11-04T18:37:32.805375] [INFO ] [LOCAL ] EFX: Starting deletion of primary data. [2025-11-04T18:37:32.805564] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-11-04T18:37:32.806320] [DEBUG] [LOCAL ] EFX: Deleted primary data file. [2025-11-04T18:37:32.807593] [INFO ] [MAIN ] EFX: Starting full primary refresh. [2025-11-04T18:37:32.967524] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-10. [2025-11-04T18:37:33.197079] [INFO ] [REMOTE ] EFX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:37:33.767010] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-01. [2025-11-04T18:37:33.809722] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-11-04T18:37:34.567051] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-11. [2025-11-04T18:37:34.744636] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T18:37:35.220038] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-11 (0.65 seconds). [2025-11-04T18:37:35.221085] [INFO ] [REMOTE ] EFR: Full fetch complete (19897 records). [2025-11-04T18:37:35.221159] [DEBUG] [VALIDATOR ] EFR: Validating primary data. [2025-11-04T18:37:35.372139] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-01. [2025-11-04T18:37:35.593319] [DEBUG] [VALIDATOR ] EFR: All expected trading days are present. [2025-11-04T18:37:35.593417] [DEBUG] [VALIDATOR ] EFR: Primary data validation passed. [2025-11-04T18:37:35.593924] [INFO ] [MAIN ] EFR: Fetched primary data is valid. [2025-11-04T18:37:35.593985] [DEBUG] [LOCAL ] EFR: Saving primary data. [2025-11-04T18:37:35.594536] [DEBUG] [LOCAL ] EFR: Trimmed 19897 records to 14351 (keeping 200 days of data) [2025-11-04T18:37:35.679321] [DEBUG] [LOCAL ] EFR: Primary data saved (14351 records). [2025-11-04T18:37:35.679453] [INFO ] [MAIN ] EFR: Full history saved. [2025-11-04T18:37:35.679528] [INFO ] [MAIN ] Update Progress: 703/2367 symbols completed. EFR took 24.2s. Estimated remaining: 3h 50m 53s. Rate: 7.2 symbols/min [2025-11-04T18:37:35.845979] [INFO ] [MAIN ] EFXT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:37:35.873624] [DEBUG] [LOCAL ] EFXT: Loaded 29633 existing records from primary storage. [2025-11-04T18:37:35.873895] [INFO ] [MAIN ] EFXT: Loaded 29633 existing records. [2025-11-04T18:37:36.051462] [WARN ] [MAIN ] EFXT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:37:36.051714] [INFO ] [MAIN ] EFXT: Wiping existing primary data before full refresh. [2025-11-04T18:37:36.051788] [INFO ] [LOCAL ] EFXT: Starting deletion of primary data. [2025-11-04T18:37:36.052031] [DEBUG] [LOCAL ] EFXT: Deleted primary data file. [2025-11-04T18:37:36.052088] [INFO ] [MAIN ] EFXT: Starting full primary refresh. [2025-11-04T18:37:36.172169] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-11. [2025-11-04T18:37:36.416992] [INFO ] [REMOTE ] EFXT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:37:36.665823] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-11 (0.49 seconds). [2025-11-04T18:37:36.666692] [INFO ] [REMOTE ] EEX: Full fetch complete (17773 records). [2025-11-04T18:37:36.666793] [DEBUG] [VALIDATOR ] EEX: Validating primary data. [2025-11-04T18:37:36.822366] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T18:37:36.972261] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-02. [2025-11-04T18:37:37.080945] [DEBUG] [VALIDATOR ] EEX: All expected trading days are present. [2025-11-04T18:37:37.081024] [DEBUG] [VALIDATOR ] EEX: Primary data validation passed. [2025-11-04T18:37:37.081438] [INFO ] [MAIN ] EEX: Fetched primary data is valid. [2025-11-04T18:37:37.081477] [DEBUG] [LOCAL ] EEX: Saving primary data. [2025-11-04T18:37:37.081993] [DEBUG] [LOCAL ] EEX: Trimmed 17773 records to 9093 (keeping 200 days of data) [2025-11-04T18:37:37.131885] [DEBUG] [LOCAL ] EEX: Primary data saved (9093 records). [2025-11-04T18:37:37.131992] [INFO ] [MAIN ] EEX: Full history saved. [2025-11-04T18:37:37.132060] [INFO ] [MAIN ] Update Progress: 704/2367 symbols completed. EEX took 44.7s. Estimated remaining: 3h 50m 28s. Rate: 7.2 symbols/min [2025-11-04T18:37:37.304925] [INFO ] [MAIN ] EG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:37:37.305109] [DEBUG] [LOCAL ] EG: No existing data file found in primary storage. [2025-11-04T18:37:37.305146] [WARN ] [MAIN ] EG: No existing data found, performing full refresh instead. [2025-11-04T18:37:37.305256] [INFO ] [MAIN ] EG: Wiping existing primary data before full refresh. [2025-11-04T18:37:37.305297] [INFO ] [LOCAL ] EG: Starting deletion of primary data. [2025-11-04T18:37:37.305339] [DEBUG] [LOCAL ] EG: No primary data file found to delete. [2025-11-04T18:37:37.305371] [INFO ] [MAIN ] EG: Starting full primary refresh. [2025-11-04T18:37:37.666122] [INFO ] [REMOTE ] EG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:37:37.767060] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-01. [2025-11-04T18:37:38.084248] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T18:37:38.567054] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-01. [2025-11-04T18:37:39.367086] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-02. [2025-11-04T18:37:39.657579] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-11-04T18:37:39.772242] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T18:37:39.916410] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T18:37:40.167078] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-02. [2025-11-04T18:37:40.863131] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T18:37:40.967096] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-02. [2025-11-04T18:37:41.767043] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-03. [2025-11-04T18:37:42.001625] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T18:37:42.295798] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-11-04T18:37:42.567086] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-03. [2025-11-04T18:37:43.367042] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-03. [2025-11-04T18:37:43.500627] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T18:37:44.167015] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-04. [2025-11-04T18:37:44.358083] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T18:37:44.920826] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T18:37:44.967015] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-04. [2025-11-04T18:37:45.767022] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-04. [2025-11-04T18:37:45.912825] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-11-04T18:37:46.567017] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-03. [2025-11-04T18:37:46.654410] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-11-04T18:37:47.367029] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-05. [2025-11-04T18:37:47.710680] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T18:37:47.846124] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-11-04T18:37:48.167051] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-05. [2025-11-04T18:37:48.967036] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-04. [2025-11-04T18:37:49.094492] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-11-04T18:37:49.767050] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-06. [2025-11-04T18:37:49.825013] [DEBUG] [REMOTE ] EFT: Error fetching primary data for 2025-06: 'API call returned an empty response.' [2025-11-04T18:37:49.825398] [WARN ] [REMOTE ] EFT: No data returned for month 2025-06 [2025-11-04T18:37:49.825493] [FATAL] [REMOTE ] EFT: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:37:49.825598] [FATAL] [MAIN ] EFT: Failed to fetch primary data. [2025-11-04T18:37:49.825697] [WARN ] [MAIN ] EFT: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:37:49.825776] [INFO ] [LOCAL ] EFT: Starting deletion of primary data. [2025-11-04T18:37:49.825842] [DEBUG] [LOCAL ] EFT: No primary data file found to delete. [2025-11-04T18:37:49.825879] [INFO ] [LOCAL ] EFT: Starting deletion of derivative data. [2025-11-04T18:37:49.826068] [DEBUG] [LOCAL ] EFT: Deleted derivative data file. [2025-11-04T18:37:49.826170] [INFO ] [MAIN ] Update Progress: 705/2367 symbols completed. EFT took 20.9s. Estimated remaining: 3h 50m 30s. Rate: 7.2 symbols/min [2025-11-04T18:37:50.009675] [INFO ] [MAIN ] EGO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:37:50.058114] [DEBUG] [LOCAL ] EGO: Loaded 51693 existing records from primary storage. [2025-11-04T18:37:50.058426] [INFO ] [MAIN ] EGO: Loaded 51693 existing records. [2025-11-04T18:37:50.238148] [WARN ] [MAIN ] EGO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:37:50.238396] [INFO ] [MAIN ] EGO: Wiping existing primary data before full refresh. [2025-11-04T18:37:50.238507] [INFO ] [LOCAL ] EGO: Starting deletion of primary data. [2025-11-04T18:37:50.238775] [DEBUG] [LOCAL ] EGO: Deleted primary data file. [2025-11-04T18:37:50.238856] [INFO ] [MAIN ] EGO: Starting full primary refresh. [2025-11-04T18:37:50.585461] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-05. [2025-11-04T18:37:50.669043] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-11-04T18:37:50.707794] [INFO ] [REMOTE ] EGO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:37:51.367047] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-01. [2025-11-04T18:37:51.718081] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T18:37:52.167111] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-06. [2025-11-04T18:37:52.875335] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T18:37:52.938973] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-11-04T18:37:52.967009] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-02. [2025-11-04T18:37:53.767047] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-06. [2025-11-04T18:37:54.160702] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T18:37:54.567008] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-03. [2025-11-04T18:37:54.592001] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-11-04T18:37:55.367044] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-07. [2025-11-04T18:37:55.677009] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T18:37:56.167014] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-04. [2025-11-04T18:37:56.302397] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-11-04T18:37:56.967074] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-05. [2025-11-04T18:37:57.441590] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T18:37:57.767081] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-07. [2025-11-04T18:37:58.158639] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T18:37:58.572271] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-08. [2025-11-04T18:37:58.584729] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T18:37:59.372308] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-05. [2025-11-04T18:37:59.376978] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-11-04T18:38:00.167057] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-08. [2025-11-04T18:38:00.967049] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-06. [2025-11-04T18:38:01.096347] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-11-04T18:38:01.673926] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T18:38:01.767134] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-06. [2025-11-04T18:38:02.016807] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T18:38:02.567021] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-09. [2025-11-04T18:38:03.274429] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-11-04T18:38:03.367010] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-07. [2025-11-04T18:38:03.475253] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-11-04T18:38:04.167052] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-07. [2025-11-04T18:38:04.647292] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T18:38:04.967063] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-10. [2025-11-04T18:38:05.767031] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-09. [2025-11-04T18:38:05.843354] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-11-04T18:38:05.976231] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-11-04T18:38:06.567079] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-08. [2025-11-04T18:38:06.851565] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T18:38:07.367089] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-11. [2025-11-04T18:38:07.741340] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T18:38:08.167192] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-08. [2025-11-04T18:38:08.535198] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-11 (1.17 seconds). [2025-11-04T18:38:08.537567] [INFO ] [REMOTE ] EG: Full fetch complete (42987 records). [2025-11-04T18:38:08.537694] [DEBUG] [VALIDATOR ] EG: Validating primary data. [2025-11-04T18:38:08.919473] [DEBUG] [VALIDATOR ] EG: All expected trading days are present. [2025-11-04T18:38:08.919584] [DEBUG] [VALIDATOR ] EG: Primary data validation passed. [2025-11-04T18:38:08.920218] [INFO ] [MAIN ] EG: Fetched primary data is valid. [2025-11-04T18:38:08.920282] [DEBUG] [LOCAL ] EG: Saving primary data. [2025-11-04T18:38:08.921287] [DEBUG] [LOCAL ] EG: Trimmed 42987 records to 27869 (keeping 200 days of data) [2025-11-04T18:38:08.974746] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-09. [2025-11-04T18:38:09.100279] [DEBUG] [LOCAL ] EG: Primary data saved (27869 records). [2025-11-04T18:38:09.100431] [INFO ] [MAIN ] EG: Full history saved. [2025-11-04T18:38:09.100505] [INFO ] [MAIN ] Update Progress: 706/2367 symbols completed. EG took 32.0s. Estimated remaining: 3h 50m 47s. Rate: 7.2 symbols/min [2025-11-04T18:38:09.322824] [INFO ] [MAIN ] EGP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:38:09.353228] [DEBUG] [LOCAL ] EGP: Loaded 29691 existing records from primary storage. [2025-11-04T18:38:09.353534] [INFO ] [MAIN ] EGP: Loaded 29691 existing records. [2025-11-04T18:38:09.536186] [WARN ] [MAIN ] EGP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:38:09.536581] [INFO ] [MAIN ] EGP: Wiping existing primary data before full refresh. [2025-11-04T18:38:09.536748] [INFO ] [LOCAL ] EGP: Starting deletion of primary data. [2025-11-04T18:38:09.537086] [DEBUG] [LOCAL ] EGP: Deleted primary data file. [2025-11-04T18:38:09.537161] [INFO ] [MAIN ] EGP: Starting full primary refresh. [2025-11-04T18:38:09.770679] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-10. [2025-11-04T18:38:09.770487] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-11-04T18:38:09.961963] [INFO ] [REMOTE ] EGP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:38:10.195918] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T18:38:10.567042] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-01. [2025-11-04T18:38:11.171212] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T18:38:11.367105] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-10. [2025-11-04T18:38:11.682097] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T18:38:12.167034] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-02. [2025-11-04T18:38:12.694779] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-11-04T18:38:12.967020] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-11. [2025-11-04T18:38:13.207540] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-11-04T18:38:13.767061] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-03. [2025-11-04T18:38:14.214383] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T18:38:14.215814] [INFO ] [REMOTE ] EFX: Full fetch complete (64901 records). [2025-11-04T18:38:14.215902] [DEBUG] [VALIDATOR ] EFX: Validating primary data. [2025-11-04T18:38:14.216551] [FATAL] [VALIDATOR ] EFX: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:38:14.216677] [FATAL] [MAIN ] EFX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:38:14.216804] [WARN ] [MAIN ] EFX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:38:14.216901] [INFO ] [LOCAL ] EFX: Starting deletion of primary data. [2025-11-04T18:38:14.216981] [DEBUG] [LOCAL ] EFX: No primary data file found to delete. [2025-11-04T18:38:14.217019] [INFO ] [LOCAL ] EFX: Starting deletion of derivative data. [2025-11-04T18:38:14.217170] [DEBUG] [LOCAL ] EFX: Deleted derivative data file. [2025-11-04T18:38:14.217262] [INFO ] [MAIN ] Update Progress: 707/2367 symbols completed. EFX took 41.8s. Estimated remaining: 3h 50m 32s. Rate: 7.2 symbols/min [2025-11-04T18:38:14.388022] [INFO ] [MAIN ] EGY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:38:14.424282] [DEBUG] [LOCAL ] EGY: Loaded 40542 existing records from primary storage. [2025-11-04T18:38:14.424575] [INFO ] [MAIN ] EGY: Loaded 40542 existing records. [2025-11-04T18:38:14.572205] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-11. [2025-11-04T18:38:14.615380] [WARN ] [MAIN ] EGY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:38:14.615609] [INFO ] [MAIN ] EGY: Wiping existing primary data before full refresh. [2025-11-04T18:38:14.615691] [INFO ] [LOCAL ] EGY: Starting deletion of primary data. [2025-11-04T18:38:14.615882] [DEBUG] [LOCAL ] EGY: Deleted primary data file. [2025-11-04T18:38:14.615920] [INFO ] [MAIN ] EGY: Starting full primary refresh. [2025-11-04T18:38:14.990358] [INFO ] [REMOTE ] EGY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:38:15.012024] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T18:38:15.367054] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-09. [2025-11-04T18:38:15.412164] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T18:38:15.414255] [INFO ] [REMOTE ] EFXT: Full fetch complete (44350 records). [2025-11-04T18:38:15.414372] [DEBUG] [VALIDATOR ] EFXT: Validating primary data. [2025-11-04T18:38:15.795548] [DEBUG] [VALIDATOR ] EFXT: All expected trading days are present. [2025-11-04T18:38:15.795664] [DEBUG] [VALIDATOR ] EFXT: Primary data validation passed. [2025-11-04T18:38:15.796351] [INFO ] [MAIN ] EFXT: Fetched primary data is valid. [2025-11-04T18:38:15.796395] [DEBUG] [LOCAL ] EFXT: Saving primary data. [2025-11-04T18:38:15.797549] [DEBUG] [LOCAL ] EFXT: Trimmed 44350 records to 29675 (keeping 200 days of data) [2025-11-04T18:38:15.966278] [DEBUG] [LOCAL ] EFXT: Primary data saved (29675 records). [2025-11-04T18:38:15.966419] [INFO ] [MAIN ] EFXT: Full history saved. [2025-11-04T18:38:15.966499] [INFO ] [MAIN ] Update Progress: 708/2367 symbols completed. EFXT took 40.3s. Estimated remaining: 3h 50m 8s. Rate: 7.2 symbols/min [2025-11-04T18:38:16.159237] [INFO ] [MAIN ] EHAB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:38:16.167059] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-01. [2025-11-04T18:38:16.193302] [DEBUG] [LOCAL ] EHAB: Loaded 28900 existing records from primary storage. [2025-11-04T18:38:16.193737] [INFO ] [MAIN ] EHAB: Loaded 28900 existing records. [2025-11-04T18:38:16.385160] [WARN ] [MAIN ] EHAB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:38:16.385491] [INFO ] [MAIN ] EHAB: Wiping existing primary data before full refresh. [2025-11-04T18:38:16.385557] [INFO ] [LOCAL ] EHAB: Starting deletion of primary data. [2025-11-04T18:38:16.385806] [DEBUG] [LOCAL ] EHAB: Deleted primary data file. [2025-11-04T18:38:16.385867] [INFO ] [MAIN ] EHAB: Starting full primary refresh. [2025-11-04T18:38:16.781209] [INFO ] [REMOTE ] EHAB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:38:16.835254] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-11-04T18:38:16.967042] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-10. [2025-11-04T18:38:17.767044] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-04. [2025-11-04T18:38:17.809553] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-11-04T18:38:18.336251] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T18:38:18.567101] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-11. [2025-11-04T18:38:18.860502] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T18:38:19.367051] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-01. [2025-11-04T18:38:19.660355] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T18:38:19.663184] [INFO ] [REMOTE ] EGO: Full fetch complete (74842 records). [2025-11-04T18:38:19.663403] [DEBUG] [VALIDATOR ] EGO: Validating primary data. [2025-11-04T18:38:20.093486] [DEBUG] [VALIDATOR ] EGO: All expected trading days are present. [2025-11-04T18:38:20.093585] [DEBUG] [VALIDATOR ] EGO: Primary data validation passed. [2025-11-04T18:38:20.094479] [INFO ] [MAIN ] EGO: Fetched primary data is valid. [2025-11-04T18:38:20.094546] [DEBUG] [LOCAL ] EGO: Saving primary data. [2025-11-04T18:38:20.096122] [DEBUG] [LOCAL ] EGO: Trimmed 74842 records to 51700 (keeping 200 days of data) [2025-11-04T18:38:20.178177] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-05. [2025-11-04T18:38:20.318002] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T18:38:20.424517] [DEBUG] [LOCAL ] EGO: Primary data saved (51700 records). [2025-11-04T18:38:20.424713] [INFO ] [MAIN ] EGO: Full history saved. [2025-11-04T18:38:20.424812] [INFO ] [MAIN ] Update Progress: 709/2367 symbols completed. EGO took 30.6s. Estimated remaining: 3h 49m 50s. Rate: 7.2 symbols/min [2025-11-04T18:38:20.605247] [INFO ] [MAIN ] EHC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:38:20.650276] [DEBUG] [LOCAL ] EHC: Loaded 40593 existing records from primary storage. [2025-11-04T18:38:20.650569] [INFO ] [MAIN ] EHC: Loaded 40593 existing records. [2025-11-04T18:38:20.831832] [WARN ] [MAIN ] EHC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:38:20.832101] [INFO ] [MAIN ] EHC: Wiping existing primary data before full refresh. [2025-11-04T18:38:20.832157] [INFO ] [LOCAL ] EHC: Starting deletion of primary data. [2025-11-04T18:38:20.832329] [DEBUG] [LOCAL ] EHC: Deleted primary data file. [2025-11-04T18:38:20.832368] [INFO ] [MAIN ] EHC: Starting full primary refresh. [2025-11-04T18:38:20.980762] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-02. [2025-11-04T18:38:21.234624] [INFO ] [REMOTE ] EHC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:38:21.253375] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-11-04T18:38:21.773116] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-02. [2025-11-04T18:38:21.781340] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-11-04T18:38:22.567088] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-03. [2025-11-04T18:38:22.853302] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T18:38:23.282373] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-11-04T18:38:23.367080] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-04. [2025-11-04T18:38:24.167082] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-03. [2025-11-04T18:38:24.222638] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T18:38:24.967081] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-06. [2025-11-04T18:38:25.291714] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T18:38:25.767072] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-05. [2025-11-04T18:38:25.806899] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T18:38:26.575018] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-01. [2025-11-04T18:38:26.577408] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T18:38:27.367083] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-04. [2025-11-04T18:38:28.004590] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T18:38:28.167027] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-02. [2025-11-04T18:38:28.633802] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T18:38:28.967048] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-05. [2025-11-04T18:38:29.217804] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T18:38:29.767039] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-06. [2025-11-04T18:38:30.113036] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T18:38:30.521324] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-11-04T18:38:30.567016] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-07. [2025-11-04T18:38:31.367070] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-03. [2025-11-04T18:38:31.628504] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-11-04T18:38:32.167022] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-06. [2025-11-04T18:38:32.277221] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T18:38:32.967049] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-08. [2025-11-04T18:38:33.234097] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T18:38:33.714336] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T18:38:33.767045] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-09. [2025-11-04T18:38:34.489430] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-11-04T18:38:34.567047] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-10. [2025-11-04T18:38:35.367043] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-07. [2025-11-04T18:38:35.391838] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T18:38:36.167054] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-04. [2025-11-04T18:38:36.576751] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-11-04T18:38:36.967120] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-08. [2025-11-04T18:38:37.425329] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T18:38:37.767073] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-07. [2025-11-04T18:38:37.797713] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-11-04T18:38:38.567098] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-11. [2025-11-04T18:38:38.959334] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-11-04T18:38:39.265343] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-11 (0.70 seconds). [2025-11-04T18:38:39.267960] [INFO ] [REMOTE ] EHAB: Full fetch complete (41147 records). [2025-11-04T18:38:39.268124] [DEBUG] [VALIDATOR ] EHAB: Validating primary data. [2025-11-04T18:38:39.372516] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-08. [2025-11-04T18:38:39.664919] [DEBUG] [VALIDATOR ] EHAB: All expected trading days are present. [2025-11-04T18:38:39.665042] [DEBUG] [VALIDATOR ] EHAB: Primary data validation passed. [2025-11-04T18:38:39.666267] [INFO ] [MAIN ] EHAB: Fetched primary data is valid. [2025-11-04T18:38:39.666352] [DEBUG] [LOCAL ] EHAB: Saving primary data. [2025-11-04T18:38:39.667971] [DEBUG] [LOCAL ] EHAB: Trimmed 41147 records to 28844 (keeping 200 days of data) [2025-11-04T18:38:39.850637] [DEBUG] [LOCAL ] EHAB: Primary data saved (28844 records). [2025-11-04T18:38:39.850792] [INFO ] [MAIN ] EHAB: Full history saved. [2025-11-04T18:38:39.850871] [INFO ] [MAIN ] Update Progress: 710/2367 symbols completed. EHAB took 23.9s. Estimated remaining: 3h 50m 8s. Rate: 7.2 symbols/min [2025-11-04T18:38:40.023084] [INFO ] [MAIN ] EHI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:38:40.033662] [DEBUG] [LOCAL ] EHI: Loaded 10847 existing records from primary storage. [2025-11-04T18:38:40.033876] [INFO ] [MAIN ] EHI: Loaded 10847 existing records. [2025-11-04T18:38:40.172187] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-09. [2025-11-04T18:38:40.230321] [WARN ] [MAIN ] EHI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:38:40.230514] [INFO ] [MAIN ] EHI: Wiping existing primary data before full refresh. [2025-11-04T18:38:40.230561] [INFO ] [LOCAL ] EHI: Starting deletion of primary data. [2025-11-04T18:38:40.230794] [DEBUG] [LOCAL ] EHI: Deleted primary data file. [2025-11-04T18:38:40.230850] [INFO ] [MAIN ] EHI: Starting full primary refresh. [2025-11-04T18:38:40.615742] [INFO ] [REMOTE ] EHI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:38:40.639062] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T18:38:40.967012] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-01. [2025-11-04T18:38:41.134508] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T18:38:41.767026] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-10. [2025-11-04T18:38:41.980428] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T18:38:42.567031] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-05. [2025-11-04T18:38:42.711936] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T18:38:43.367046] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-09. [2025-11-04T18:38:43.738742] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T18:38:44.167072] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-02. [2025-11-04T18:38:44.447505] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T18:38:44.867955] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T18:38:44.967025] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-03. [2025-11-04T18:38:45.513963] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-11-04T18:38:45.767045] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-10. [2025-11-04T18:38:46.567057] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-06. [2025-11-04T18:38:46.784750] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T18:38:47.357238] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T18:38:47.366992] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-07. [2025-11-04T18:38:48.167052] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-11. [2025-11-04T18:38:48.713295] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T18:38:48.967038] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-11. [2025-11-04T18:38:49.097668] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T18:38:49.099017] [INFO ] [REMOTE ] EGP: Full fetch complete (45455 records). [2025-11-04T18:38:49.099093] [DEBUG] [VALIDATOR ] EGP: Validating primary data. [2025-11-04T18:38:49.478511] [DEBUG] [VALIDATOR ] EGP: All expected trading days are present. [2025-11-04T18:38:49.478601] [DEBUG] [VALIDATOR ] EGP: Primary data validation passed. [2025-11-04T18:38:49.479222] [INFO ] [MAIN ] EGP: Fetched primary data is valid. [2025-11-04T18:38:49.479283] [DEBUG] [LOCAL ] EGP: Saving primary data. [2025-11-04T18:38:49.480264] [DEBUG] [LOCAL ] EGP: Trimmed 45455 records to 29594 (keeping 200 days of data) [2025-11-04T18:38:49.661109] [DEBUG] [LOCAL ] EGP: Primary data saved (29594 records). [2025-11-04T18:38:49.661237] [INFO ] [MAIN ] EGP: Full history saved. [2025-11-04T18:38:49.661314] [INFO ] [MAIN ] Update Progress: 711/2367 symbols completed. EGP took 40.6s. Estimated remaining: 3h 50m 3s. Rate: 7.2 symbols/min [2025-11-04T18:38:49.772145] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-08. [2025-11-04T18:38:49.830547] [INFO ] [MAIN ] EIC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:38:49.850380] [DEBUG] [LOCAL ] EIC: Loaded 21764 existing records from primary storage. [2025-11-04T18:38:49.850616] [INFO ] [MAIN ] EIC: Loaded 21764 existing records. [2025-11-04T18:38:50.008571] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T18:38:50.036983] [WARN ] [MAIN ] EIC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:38:50.037449] [INFO ] [MAIN ] EIC: Wiping existing primary data before full refresh. [2025-11-04T18:38:50.037680] [INFO ] [REMOTE ] EGY: Full fetch complete (60252 records). [2025-11-04T18:38:50.037906] [DEBUG] [VALIDATOR ] EGY: Validating primary data. [2025-11-04T18:38:50.037504] [INFO ] [LOCAL ] EIC: Starting deletion of primary data. [2025-11-04T18:38:50.038261] [DEBUG] [LOCAL ] EIC: Deleted primary data file. [2025-11-04T18:38:50.038392] [INFO ] [MAIN ] EIC: Starting full primary refresh. [2025-11-04T18:38:50.577345] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-04. [2025-11-04T18:38:50.737500] [INFO ] [REMOTE ] EIC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:38:50.810329] [DEBUG] [VALIDATOR ] EGY: All expected trading days are present. [2025-11-04T18:38:50.810439] [DEBUG] [VALIDATOR ] EGY: Primary data validation passed. [2025-11-04T18:38:50.811202] [INFO ] [MAIN ] EGY: Fetched primary data is valid. [2025-11-04T18:38:50.811277] [DEBUG] [LOCAL ] EGY: Saving primary data. [2025-11-04T18:38:50.812930] [DEBUG] [LOCAL ] EGY: Trimmed 60252 records to 40486 (keeping 200 days of data) [2025-11-04T18:38:51.027684] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T18:38:51.072544] [DEBUG] [LOCAL ] EGY: Primary data saved (40486 records). [2025-11-04T18:38:51.072689] [INFO ] [MAIN ] EGY: Full history saved. [2025-11-04T18:38:51.072782] [INFO ] [MAIN ] Update Progress: 712/2367 symbols completed. EGY took 36.9s. Estimated remaining: 3h 49m 39s. Rate: 7.2 symbols/min [2025-11-04T18:38:51.224309] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-11-04T18:38:51.258872] [INFO ] [MAIN ] EICA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:38:51.259408] [DEBUG] [LOCAL ] EICA: No existing data file found in primary storage. [2025-11-04T18:38:51.259453] [WARN ] [MAIN ] EICA: No existing data found, performing full refresh instead. [2025-11-04T18:38:51.259562] [INFO ] [MAIN ] EICA: Wiping existing primary data before full refresh. [2025-11-04T18:38:51.259604] [INFO ] [LOCAL ] EICA: Starting deletion of primary data. [2025-11-04T18:38:51.259700] [DEBUG] [LOCAL ] EICA: No primary data file found to delete. [2025-11-04T18:38:51.259750] [INFO ] [MAIN ] EICA: Starting full primary refresh. [2025-11-04T18:38:51.372184] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-05. [2025-11-04T18:38:51.631139] [INFO ] [REMOTE ] EICA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:38:51.956495] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-11-04T18:38:52.167048] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-09. [2025-11-04T18:38:52.967051] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-01. [2025-11-04T18:38:53.168352] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T18:38:53.464805] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-11-04T18:38:53.767040] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-06. [2025-11-04T18:38:54.438696] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-11-04T18:38:54.567041] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-02. [2025-11-04T18:38:54.787381] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-02 (0.22 seconds). [2025-11-04T18:38:55.367034] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-03. [2025-11-04T18:38:55.616896] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-03 (0.25 seconds). [2025-11-04T18:38:56.167041] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-04. [2025-11-04T18:38:56.413546] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-04 (0.25 seconds). [2025-11-04T18:38:56.967032] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-05. [2025-11-04T18:38:57.193766] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-05 (0.23 seconds). [2025-11-04T18:38:57.767044] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-06. [2025-11-04T18:38:58.019150] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-06 (0.25 seconds). [2025-11-04T18:38:58.567046] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-01. [2025-11-04T18:38:59.367041] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-10. [2025-11-04T18:38:59.553620] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T18:39:00.167032] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-02. [2025-11-04T18:39:00.631383] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-11-04T18:39:00.967029] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-07. [2025-11-04T18:39:01.167608] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T18:39:01.568326] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T18:39:01.767086] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-03. [2025-11-04T18:39:02.567049] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-07. [2025-11-04T18:39:02.651437] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-11-04T18:39:02.804872] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-07 (0.24 seconds). [2025-11-04T18:39:03.367040] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-04. [2025-11-04T18:39:04.167110] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-08. [2025-11-04T18:39:04.200270] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-11-04T18:39:04.617686] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-11-04T18:39:04.967042] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-05. [2025-11-04T18:39:05.735182] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-11-04T18:39:05.767006] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-06. [2025-11-04T18:39:06.567012] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-09. [2025-11-04T18:39:06.695252] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T18:39:07.102061] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T18:39:07.367050] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-10. [2025-11-04T18:39:08.020820] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-11-04T18:39:08.167040] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-08. [2025-11-04T18:39:08.512727] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-11-04T18:39:08.967009] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-09. [2025-11-04T18:39:09.200871] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-09 (0.23 seconds). [2025-11-04T18:39:09.767052] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-11. [2025-11-04T18:39:10.325984] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-11 (0.56 seconds). [2025-11-04T18:39:10.327050] [INFO ] [REMOTE ] EHI: Full fetch complete (15933 records). [2025-11-04T18:39:10.327118] [DEBUG] [VALIDATOR ] EHI: Validating primary data. [2025-11-04T18:39:10.572320] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-10. [2025-11-04T18:39:10.683737] [DEBUG] [VALIDATOR ] EHI: All expected trading days are present. [2025-11-04T18:39:10.683812] [DEBUG] [VALIDATOR ] EHI: Primary data validation passed. [2025-11-04T18:39:10.684169] [INFO ] [MAIN ] EHI: Fetched primary data is valid. [2025-11-04T18:39:10.684207] [DEBUG] [LOCAL ] EHI: Saving primary data. [2025-11-04T18:39:10.684700] [DEBUG] [LOCAL ] EHI: Trimmed 15933 records to 10970 (keeping 200 days of data) [2025-11-04T18:39:10.745880] [DEBUG] [LOCAL ] EHI: Primary data saved (10970 records). [2025-11-04T18:39:10.746008] [INFO ] [MAIN ] EHI: Full history saved. [2025-11-04T18:39:10.746085] [INFO ] [MAIN ] Update Progress: 713/2367 symbols completed. EHI took 30.9s. Estimated remaining: 3h 49m 57s. Rate: 7.2 symbols/min [2025-11-04T18:39:10.915849] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-10 (0.34 seconds). [2025-11-04T18:39:10.918793] [INFO ] [MAIN ] EICB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:39:10.920592] [DEBUG] [LOCAL ] EICB: No existing data file found in primary storage. [2025-11-04T18:39:10.920667] [WARN ] [MAIN ] EICB: No existing data found, performing full refresh instead. [2025-11-04T18:39:10.920783] [INFO ] [MAIN ] EICB: Wiping existing primary data before full refresh. [2025-11-04T18:39:10.920824] [INFO ] [LOCAL ] EICB: Starting deletion of primary data. [2025-11-04T18:39:10.920863] [DEBUG] [LOCAL ] EICB: No primary data file found to delete. [2025-11-04T18:39:10.920894] [INFO ] [MAIN ] EICB: Starting full primary refresh. [2025-11-04T18:39:11.271111] [INFO ] [REMOTE ] EICB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:39:11.367005] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-01. [2025-11-04T18:39:11.931796] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T18:39:12.167044] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-02. [2025-11-04T18:39:12.483506] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T18:39:12.967036] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-11. [2025-11-04T18:39:13.767127] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-11. [2025-11-04T18:39:14.045048] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-11 (0.28 seconds). [2025-11-04T18:39:14.045690] [INFO ] [REMOTE ] EICA: Full fetch complete (1172 records). [2025-11-04T18:39:14.045757] [DEBUG] [VALIDATOR ] EICA: Validating primary data. [2025-11-04T18:39:14.046084] [FATAL] [VALIDATOR ] EICA: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:39:14.046201] [FATAL] [MAIN ] EICA: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:39:14.046296] [WARN ] [MAIN ] EICA: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:39:14.046387] [INFO ] [LOCAL ] EICA: Starting deletion of primary data. [2025-11-04T18:39:14.046467] [DEBUG] [LOCAL ] EICA: No primary data file found to delete. [2025-11-04T18:39:14.046504] [INFO ] [LOCAL ] EICA: Starting deletion of derivative data. [2025-11-04T18:39:14.046545] [DEBUG] [LOCAL ] EICA: No derivative data file found to delete. [2025-11-04T18:39:14.046598] [INFO ] [MAIN ] Update Progress: 714/2367 symbols completed. EICA took 23.0s. Estimated remaining: 3h 49m 37s. Rate: 7.2 symbols/min [2025-11-04T18:39:14.223062] [INFO ] [MAIN ] EICC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:39:14.223352] [DEBUG] [LOCAL ] EICC: No existing data file found in primary storage. [2025-11-04T18:39:14.223440] [WARN ] [MAIN ] EICC: No existing data found, performing full refresh instead. [2025-11-04T18:39:14.227637] [INFO ] [MAIN ] EICC: Wiping existing primary data before full refresh. [2025-11-04T18:39:14.228305] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-11 (1.26 seconds). [2025-11-04T18:39:14.228143] [INFO ] [LOCAL ] EICC: Starting deletion of primary data. [2025-11-04T18:39:14.230057] [DEBUG] [LOCAL ] EICC: No primary data file found to delete. [2025-11-04T18:39:14.230524] [INFO ] [MAIN ] EICC: Starting full primary refresh. [2025-11-04T18:39:14.262158] [INFO ] [REMOTE ] EHC: Full fetch complete (59265 records). [2025-11-04T18:39:14.286138] [DEBUG] [VALIDATOR ] EHC: Validating primary data. [2025-11-04T18:39:14.591071] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-03. [2025-11-04T18:39:14.963840] [INFO ] [REMOTE ] EICC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:39:14.972626] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-11-04T18:39:15.043174] [DEBUG] [VALIDATOR ] EHC: All expected trading days are present. [2025-11-04T18:39:15.043247] [DEBUG] [VALIDATOR ] EHC: Primary data validation passed. [2025-11-04T18:39:15.043984] [INFO ] [MAIN ] EHC: Fetched primary data is valid. [2025-11-04T18:39:15.044046] [DEBUG] [LOCAL ] EHC: Saving primary data. [2025-11-04T18:39:15.045185] [DEBUG] [LOCAL ] EHC: Trimmed 59265 records to 40607 (keeping 200 days of data) [2025-11-04T18:39:15.296797] [DEBUG] [LOCAL ] EHC: Primary data saved (40607 records). [2025-11-04T18:39:15.296918] [INFO ] [MAIN ] EHC: Full history saved. [2025-11-04T18:39:15.296994] [INFO ] [MAIN ] Update Progress: 715/2367 symbols completed. EHC took 54.9s. Estimated remaining: 3h 49m 12s. Rate: 7.2 symbols/min [2025-11-04T18:39:15.372127] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-07. [2025-11-04T18:39:15.470798] [INFO ] [MAIN ] EIG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:39:15.490421] [DEBUG] [LOCAL ] EIG: Loaded 19931 existing records from primary storage. [2025-11-04T18:39:15.490684] [INFO ] [MAIN ] EIG: Loaded 19931 existing records. [2025-11-04T18:39:15.664142] [WARN ] [MAIN ] EIG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:39:15.664396] [INFO ] [MAIN ] EIG: Wiping existing primary data before full refresh. [2025-11-04T18:39:15.664458] [INFO ] [LOCAL ] EIG: Starting deletion of primary data. [2025-11-04T18:39:15.664639] [DEBUG] [LOCAL ] EIG: Deleted primary data file. [2025-11-04T18:39:15.664738] [INFO ] [MAIN ] EIG: Starting full primary refresh. [2025-11-04T18:39:16.020112] [INFO ] [REMOTE ] EIG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:39:16.167027] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-04. [2025-11-04T18:39:16.459811] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-11-04T18:39:16.477395] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-11-04T18:39:16.967042] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-08. [2025-11-04T18:39:17.743123] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-11-04T18:39:17.766996] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-09. [2025-11-04T18:39:18.431190] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-11-04T18:39:18.567067] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-01. [2025-11-04T18:39:19.025552] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-11-04T18:39:19.367023] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-05. [2025-11-04T18:39:19.655515] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-05 (0.29 seconds). [2025-11-04T18:39:20.167050] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-06. [2025-11-04T18:39:20.441571] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-11-04T18:39:20.967048] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-10. [2025-11-04T18:39:21.652740] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-11-04T18:39:21.767004] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-11. [2025-11-04T18:39:22.393303] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T18:39:22.394461] [INFO ] [REMOTE ] EIC: Full fetch complete (33625 records). [2025-11-04T18:39:22.394529] [DEBUG] [VALIDATOR ] EIC: Validating primary data. [2025-11-04T18:39:22.568549] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-01. [2025-11-04T18:39:22.766895] [DEBUG] [VALIDATOR ] EIC: All expected trading days are present. [2025-11-04T18:39:22.766975] [DEBUG] [VALIDATOR ] EIC: Primary data validation passed. [2025-11-04T18:39:22.767540] [INFO ] [MAIN ] EIC: Fetched primary data is valid. [2025-11-04T18:39:22.767583] [DEBUG] [LOCAL ] EIC: Saving primary data. [2025-11-04T18:39:22.768336] [DEBUG] [LOCAL ] EIC: Trimmed 33625 records to 21816 (keeping 200 days of data) [2025-11-04T18:39:22.899675] [DEBUG] [LOCAL ] EIC: Primary data saved (21816 records). [2025-11-04T18:39:22.899810] [INFO ] [MAIN ] EIC: Full history saved. [2025-11-04T18:39:22.899924] [INFO ] [MAIN ] Update Progress: 716/2367 symbols completed. EIC took 33.2s. Estimated remaining: 3h 49m 2s. Rate: 7.2 symbols/min [2025-11-04T18:39:23.104871] [INFO ] [MAIN ] EIX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:39:23.157143] [DEBUG] [LOCAL ] EIX: Loaded 53346 existing records from primary storage. [2025-11-04T18:39:23.157449] [INFO ] [MAIN ] EIX: Loaded 53346 existing records. [2025-11-04T18:39:23.338603] [WARN ] [MAIN ] EIX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:39:23.339032] [INFO ] [MAIN ] EIX: Wiping existing primary data before full refresh. [2025-11-04T18:39:23.339130] [INFO ] [LOCAL ] EIX: Starting deletion of primary data. [2025-11-04T18:39:23.339472] [DEBUG] [LOCAL ] EIX: Deleted primary data file. [2025-11-04T18:39:23.339548] [INFO ] [MAIN ] EIX: Starting full primary refresh. [2025-11-04T18:39:23.371192] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-07. [2025-11-04T18:39:23.424770] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T18:39:23.708909] [INFO ] [REMOTE ] EIX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:39:23.949848] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-11-04T18:39:24.167075] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-01. [2025-11-04T18:39:24.967043] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-08. [2025-11-04T18:39:25.411828] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T18:39:25.552693] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-11-04T18:39:25.767075] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-09. [2025-11-04T18:39:26.066021] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-09 (0.30 seconds). [2025-11-04T18:39:26.567124] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-02. [2025-11-04T18:39:27.220508] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-11-04T18:39:27.367058] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-02. [2025-11-04T18:39:27.713894] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-11-04T18:39:28.167052] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-10. [2025-11-04T18:39:28.448459] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-10 (0.28 seconds). [2025-11-04T18:39:28.967047] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-03. [2025-11-04T18:39:29.622868] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-11-04T18:39:29.767098] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-03. [2025-11-04T18:39:30.167550] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T18:39:30.567059] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-04. [2025-11-04T18:39:31.259406] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-11-04T18:39:31.367052] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-02. [2025-11-04T18:39:32.167049] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-04. [2025-11-04T18:39:32.619000] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-11-04T18:39:32.701212] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T18:39:32.967029] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-05. [2025-11-04T18:39:33.340072] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-11-04T18:39:33.767073] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-06. [2025-11-04T18:39:34.174790] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-11-04T18:39:34.567099] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-03. [2025-11-04T18:39:35.367047] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-07. [2025-11-04T18:39:35.810963] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-11-04T18:39:36.077171] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-11-04T18:39:36.167008] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-08. [2025-11-04T18:39:36.642887] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-11-04T18:39:36.967075] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-09. [2025-11-04T18:39:37.314201] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-11-04T18:39:37.767073] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-10. [2025-11-04T18:39:38.174340] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-11-04T18:39:38.567078] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-11. [2025-11-04T18:39:38.866628] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-11 (0.30 seconds). [2025-11-04T18:39:38.867622] [INFO ] [REMOTE ] EICC: Full fetch complete (8474 records). [2025-11-04T18:39:38.867743] [DEBUG] [VALIDATOR ] EICC: Validating primary data. [2025-11-04T18:39:38.868290] [FATAL] [VALIDATOR ] EICC: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:39:38.868382] [FATAL] [MAIN ] EICC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:39:38.868427] [WARN ] [MAIN ] EICC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:39:38.868479] [INFO ] [LOCAL ] EICC: Starting deletion of primary data. [2025-11-04T18:39:38.868551] [DEBUG] [LOCAL ] EICC: No primary data file found to delete. [2025-11-04T18:39:38.868591] [INFO ] [LOCAL ] EICC: Starting deletion of derivative data. [2025-11-04T18:39:38.868632] [DEBUG] [LOCAL ] EICC: No derivative data file found to delete. [2025-11-04T18:39:38.868730] [INFO ] [MAIN ] Update Progress: 717/2367 symbols completed. EICC took 24.8s. Estimated remaining: 3h 49m 11s. Rate: 7.2 symbols/min [2025-11-04T18:39:39.367053] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-11. [2025-11-04T18:39:39.735067] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-11 (0.37 seconds). [2025-11-04T18:39:39.735734] [INFO ] [REMOTE ] EICB: Full fetch complete (2611 records). [2025-11-04T18:39:39.735802] [DEBUG] [VALIDATOR ] EICB: Validating primary data. [2025-11-04T18:39:39.920797] [WARN ] [VALIDATOR ] EICB: Validation failed - insufficient historical data. Have 197 unique days, need 200 days [2025-11-04T18:39:39.921273] [FATAL] [MAIN ] EICB: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:39:39.921368] [WARN ] [MAIN ] EICB: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:39:39.921459] [INFO ] [LOCAL ] EICB: Starting deletion of primary data. [2025-11-04T18:39:39.921549] [DEBUG] [LOCAL ] EICB: No primary data file found to delete. [2025-11-04T18:39:39.921604] [INFO ] [LOCAL ] EICB: Starting deletion of derivative data. [2025-11-04T18:39:39.921673] [DEBUG] [LOCAL ] EICB: No derivative data file found to delete. [2025-11-04T18:39:39.921748] [INFO ] [MAIN ] Update Progress: 718/2367 symbols completed. EICB took 29.2s. Estimated remaining: 3h 48m 46s. Rate: 7.2 symbols/min [2025-11-04T18:39:40.167054] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-05. [2025-11-04T18:39:40.967047] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-04. [2025-11-04T18:39:41.072852] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-11-04T18:39:41.767044] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-06. [2025-11-04T18:39:42.211929] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T18:39:42.469087] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T18:39:42.567005] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-07. [2025-11-04T18:39:43.340830] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T18:39:43.367012] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-08. [2025-11-04T18:39:44.167104] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-05. [2025-11-04T18:39:44.261700] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T18:39:44.967105] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-09. [2025-11-04T18:39:45.621662] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T18:39:45.645693] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T18:39:45.767057] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-06. [2025-11-04T18:39:46.567085] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-10. [2025-11-04T18:39:47.212169] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T18:39:47.274950] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-11-04T18:39:47.367010] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-11. [2025-11-04T18:39:48.067935] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-11 (0.70 seconds). [2025-11-04T18:39:48.069326] [INFO ] [REMOTE ] EIG: Full fetch complete (27801 records). [2025-11-04T18:39:48.069397] [DEBUG] [VALIDATOR ] EIG: Validating primary data. [2025-11-04T18:39:48.168611] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-07. [2025-11-04T18:39:48.433706] [DEBUG] [VALIDATOR ] EIG: All expected trading days are present. [2025-11-04T18:39:48.433807] [DEBUG] [VALIDATOR ] EIG: Primary data validation passed. [2025-11-04T18:39:48.434262] [INFO ] [MAIN ] EIG: Fetched primary data is valid. [2025-11-04T18:39:48.434320] [DEBUG] [LOCAL ] EIG: Saving primary data. [2025-11-04T18:39:48.435029] [DEBUG] [LOCAL ] EIG: Trimmed 27801 records to 20107 (keeping 200 days of data) [2025-11-04T18:39:48.549085] [DEBUG] [LOCAL ] EIG: Primary data saved (20107 records). [2025-11-04T18:39:48.549200] [INFO ] [MAIN ] EIG: Full history saved. [2025-11-04T18:39:48.549272] [INFO ] [MAIN ] Update Progress: 719/2367 symbols completed. EIG took 33.3s. Estimated remaining: 3h 48m 39s. Rate: 7.2 symbols/min [2025-11-04T18:39:49.830342] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-11-04T18:39:49.841886] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-08. [2025-11-04T18:39:51.437493] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-11-04T18:39:51.449144] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-09. [2025-11-04T18:39:52.987075] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-11-04T18:39:52.998993] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-10. [2025-11-04T18:39:54.574900] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-11-04T18:39:54.587607] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-11. [2025-11-04T18:39:54.883801] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-11 (0.30 seconds). [2025-11-04T18:39:54.885830] [INFO ] [REMOTE ] EIX: Full fetch complete (77442 records). [2025-11-04T18:39:54.885972] [DEBUG] [VALIDATOR ] EIX: Validating primary data. [2025-11-04T18:39:55.276265] [DEBUG] [VALIDATOR ] EIX: All expected trading days are present. [2025-11-04T18:39:55.276353] [DEBUG] [VALIDATOR ] EIX: Primary data validation passed. [2025-11-04T18:39:55.277267] [INFO ] [MAIN ] EIX: Fetched primary data is valid. [2025-11-04T18:39:55.277334] [DEBUG] [LOCAL ] EIX: Saving primary data. [2025-11-04T18:39:55.278908] [DEBUG] [LOCAL ] EIX: Trimmed 77442 records to 53355 (keeping 200 days of data) [2025-11-04T18:39:55.601024] [DEBUG] [LOCAL ] EIX: Primary data saved (53355 records). [2025-11-04T18:39:55.601194] [INFO ] [MAIN ] EIX: Full history saved. [2025-11-04T18:39:55.601293] [INFO ] [MAIN ] Update Progress: 720/2367 symbols completed. EIX took 32.7s. Estimated remaining: 3h 48m 27s. Rate: 7.2 symbols/min [2025-11-04T18:39:56.251329] [INFO ] [MAIN ] ELC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:39:56.230685] [INFO ] [MAIN ] ELF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:39:56.262921] [DEBUG] [LOCAL ] ELC: No existing data file found in primary storage. [2025-11-04T18:39:56.294916] [WARN ] [MAIN ] ELC: No existing data found, performing full refresh instead. [2025-11-04T18:39:56.297148] [INFO ] [MAIN ] EL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:39:56.297378] [INFO ] [MAIN ] ELC: Wiping existing primary data before full refresh. [2025-11-04T18:39:56.342236] [INFO ] [MAIN ] ELAN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:39:56.342497] [INFO ] [LOCAL ] ELC: Starting deletion of primary data. [2025-11-04T18:39:56.343326] [DEBUG] [LOCAL ] ELC: No primary data file found to delete. [2025-11-04T18:39:56.343719] [INFO ] [MAIN ] ELC: Starting full primary refresh. [2025-11-04T18:39:56.730353] [DEBUG] [LOCAL ] ELF: Loaded 51553 existing records from primary storage. [2025-11-04T18:39:56.731338] [DEBUG] [LOCAL ] ELAN: Loaded 53302 existing records from primary storage. [2025-11-04T18:39:56.736433] [INFO ] [MAIN ] ELF: Loaded 51553 existing records. [2025-11-04T18:39:56.736638] [INFO ] [MAIN ] ELAN: Loaded 53302 existing records. [2025-11-04T18:39:56.878562] [DEBUG] [LOCAL ] EL: Loaded 53369 existing records from primary storage. [2025-11-04T18:39:56.934512] [INFO ] [REMOTE ] ELC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:39:56.994167] [INFO ] [MAIN ] EL: Loaded 53369 existing records. [2025-11-04T18:39:56.988818] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-01. [2025-11-04T18:39:57.362216] [WARN ] [MAIN ] ELF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:39:57.376168] [WARN ] [MAIN ] ELAN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:39:57.392057] [INFO ] [MAIN ] ELF: Wiping existing primary data before full refresh. [2025-11-04T18:39:57.410744] [INFO ] [LOCAL ] ELF: Starting deletion of primary data. [2025-11-04T18:39:57.416092] [INFO ] [MAIN ] ELAN: Wiping existing primary data before full refresh. [2025-11-04T18:39:57.453164] [WARN ] [MAIN ] EL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:39:57.453593] [INFO ] [LOCAL ] ELAN: Starting deletion of primary data. [2025-11-04T18:39:57.453686] [INFO ] [MAIN ] EL: Wiping existing primary data before full refresh. [2025-11-04T18:39:57.453806] [INFO ] [LOCAL ] EL: Starting deletion of primary data. [2025-11-04T18:39:57.454056] [DEBUG] [LOCAL ] ELAN: Deleted primary data file. [2025-11-04T18:39:57.453909] [DEBUG] [LOCAL ] ELF: Deleted primary data file. [2025-11-04T18:39:57.454190] [INFO ] [MAIN ] ELAN: Starting full primary refresh. [2025-11-04T18:39:57.454484] [DEBUG] [LOCAL ] EL: Deleted primary data file. [2025-11-04T18:39:57.454439] [INFO ] [MAIN ] ELF: Starting full primary refresh. [2025-11-04T18:39:57.459807] [INFO ] [MAIN ] EL: Starting full primary refresh. [2025-11-04T18:39:57.892916] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T18:39:58.048232] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-02. [2025-11-04T18:39:58.585455] [INFO ] [REMOTE ] ELAN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:39:58.599461] [INFO ] [REMOTE ] ELF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:39:58.599941] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-01. [2025-11-04T18:39:58.618224] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-11-04T18:39:58.629393] [INFO ] [REMOTE ] EL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:39:58.618041] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-01. [2025-11-04T18:39:58.629744] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-01. [2025-11-04T18:39:58.631793] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-03. [2025-11-04T18:39:58.975046] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-11-04T18:39:58.976443] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-04. [2025-11-04T18:39:59.344158] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-11-04T18:39:59.345672] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-05. [2025-11-04T18:39:59.729612] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-11-04T18:39:59.731070] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-06. [2025-11-04T18:39:59.847216] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T18:39:59.857835] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-02. [2025-11-04T18:40:00.026768] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T18:40:00.101672] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-11-04T18:40:00.145040] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-11-04T18:40:00.167013] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-07. [2025-11-04T18:40:00.644007] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-11-04T18:40:00.967069] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-08. [2025-11-04T18:40:01.041473] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T18:40:01.369358] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-11-04T18:40:01.767063] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-09. [2025-11-04T18:40:02.129727] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-09 (0.36 seconds). [2025-11-04T18:40:02.567046] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-02. [2025-11-04T18:40:03.367018] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-10. [2025-11-04T18:40:03.760903] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-10 (0.39 seconds). [2025-11-04T18:40:04.140951] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-11-04T18:40:04.167005] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-03. [2025-11-04T18:40:04.967075] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-03. [2025-11-04T18:40:05.767106] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-02. [2025-11-04T18:40:05.940767] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-11-04T18:40:06.540290] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-11-04T18:40:06.567029] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-04. [2025-11-04T18:40:07.367070] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-11. [2025-11-04T18:40:07.598207] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-11-04T18:40:07.753373] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T18:40:07.809726] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-11 (0.44 seconds). [2025-11-04T18:40:07.810428] [INFO ] [REMOTE ] ELC: Full fetch complete (5043 records). [2025-11-04T18:40:07.810495] [DEBUG] [VALIDATOR ] ELC: Validating primary data. [2025-11-04T18:40:08.171123] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-03. [2025-11-04T18:40:08.177930] [DEBUG] [VALIDATOR ] ELC: All expected trading days are present. [2025-11-04T18:40:08.178012] [DEBUG] [VALIDATOR ] ELC: Primary data validation passed. [2025-11-04T18:40:08.178180] [INFO ] [MAIN ] ELC: Fetched primary data is valid. [2025-11-04T18:40:08.178232] [DEBUG] [LOCAL ] ELC: Saving primary data. [2025-11-04T18:40:08.178861] [DEBUG] [LOCAL ] ELC: Trimmed 5043 records to 3579 (keeping 200 days of data) [2025-11-04T18:40:08.201003] [DEBUG] [LOCAL ] ELC: Primary data saved (3579 records). [2025-11-04T18:40:08.201099] [INFO ] [MAIN ] ELC: Full history saved. [2025-11-04T18:40:08.201164] [INFO ] [MAIN ] Update Progress: 721/2367 symbols completed. ELC took 12.6s. Estimated remaining: 3h 48m 29s. Rate: 7.2 symbols/min [2025-11-04T18:40:08.374607] [INFO ] [MAIN ] ELME: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:40:08.411917] [DEBUG] [LOCAL ] ELME: Loaded 39541 existing records from primary storage. [2025-11-04T18:40:08.412237] [INFO ] [MAIN ] ELME: Loaded 39541 existing records. [2025-11-04T18:40:08.602985] [WARN ] [MAIN ] ELME: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:40:08.603219] [INFO ] [MAIN ] ELME: Wiping existing primary data before full refresh. [2025-11-04T18:40:08.603267] [INFO ] [LOCAL ] ELME: Starting deletion of primary data. [2025-11-04T18:40:08.603451] [DEBUG] [LOCAL ] ELME: Deleted primary data file. [2025-11-04T18:40:08.603496] [INFO ] [MAIN ] ELME: Starting full primary refresh. [2025-11-04T18:40:08.967333] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-04. [2025-11-04T18:40:08.987764] [INFO ] [REMOTE ] ELME: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:40:09.767100] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-05. [2025-11-04T18:40:10.071146] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-11-04T18:40:10.567047] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-04. [2025-11-04T18:40:10.946970] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-11-04T18:40:11.367069] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-05. [2025-11-04T18:40:11.751450] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-11-04T18:40:12.167050] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-06. [2025-11-04T18:40:12.240475] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-11-04T18:40:12.967410] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-01. [2025-11-04T18:40:12.973234] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-11-04T18:40:13.211770] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T18:40:13.767074] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-05. [2025-11-04T18:40:14.231154] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T18:40:14.567103] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-07. [2025-11-04T18:40:15.367052] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-06. [2025-11-04T18:40:15.616083] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-11-04T18:40:15.815752] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:40:16.167101] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-06. [2025-11-04T18:40:16.967064] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-02. [2025-11-04T18:40:17.080533] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-11-04T18:40:17.432567] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T18:40:17.767074] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-07. [2025-11-04T18:40:18.488800] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-11-04T18:40:18.567089] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-08. [2025-11-04T18:40:19.188626] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-11-04T18:40:19.367035] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-08. [2025-11-04T18:40:20.079138] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T18:40:20.167022] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-09. [2025-11-04T18:40:20.967016] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-03. [2025-11-04T18:40:21.079342] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-11-04T18:40:21.535625] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-11-04T18:40:21.767083] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-09. [2025-11-04T18:40:21.947044] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T18:40:22.567045] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-04. [2025-11-04T18:40:23.282442] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-11-04T18:40:23.367059] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-10. [2025-11-04T18:40:23.566256] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-11-04T18:40:24.167119] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-07. [2025-11-04T18:40:24.967022] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-10. [2025-11-04T18:40:25.091526] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-11-04T18:40:25.767017] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-11. [2025-11-04T18:40:25.918485] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-11-04T18:40:26.567080] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-08. [2025-11-04T18:40:26.771711] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T18:40:26.773770] [INFO ] [REMOTE ] ELAN: Full fetch complete (76727 records). [2025-11-04T18:40:26.773952] [DEBUG] [VALIDATOR ] ELAN: Validating primary data. [2025-11-04T18:40:26.775070] [FATAL] [VALIDATOR ] ELAN: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:40:26.775159] [FATAL] [MAIN ] ELAN: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:40:26.775203] [WARN ] [MAIN ] ELAN: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:40:26.775244] [INFO ] [LOCAL ] ELAN: Starting deletion of primary data. [2025-11-04T18:40:26.775308] [DEBUG] [LOCAL ] ELAN: No primary data file found to delete. [2025-11-04T18:40:26.775344] [INFO ] [LOCAL ] ELAN: Starting deletion of derivative data. [2025-11-04T18:40:26.775760] [DEBUG] [LOCAL ] ELAN: Deleted derivative data file. [2025-11-04T18:40:26.776095] [INFO ] [MAIN ] Update Progress: 722/2367 symbols completed. ELAN took 31.2s. Estimated remaining: 3h 48m 44s. Rate: 7.2 symbols/min [2025-11-04T18:40:26.969421] [INFO ] [MAIN ] ELP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:40:27.012977] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-11-04T18:40:27.046508] [DEBUG] [LOCAL ] ELP: Loaded 25135 existing records from primary storage. [2025-11-04T18:40:27.046869] [INFO ] [MAIN ] ELP: Loaded 25135 existing records. [2025-11-04T18:40:27.219999] [WARN ] [MAIN ] ELP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:40:27.220211] [INFO ] [MAIN ] ELP: Wiping existing primary data before full refresh. [2025-11-04T18:40:27.220257] [INFO ] [LOCAL ] ELP: Starting deletion of primary data. [2025-11-04T18:40:27.220444] [DEBUG] [LOCAL ] ELP: Deleted primary data file. [2025-11-04T18:40:27.220498] [INFO ] [MAIN ] ELP: Starting full primary refresh. [2025-11-04T18:40:27.372218] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-11. [2025-11-04T18:40:27.590214] [INFO ] [REMOTE ] ELP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:40:27.790142] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T18:40:27.792785] [INFO ] [REMOTE ] EL: Full fetch complete (77448 records). [2025-11-04T18:40:27.792932] [DEBUG] [VALIDATOR ] EL: Validating primary data. [2025-11-04T18:40:28.169536] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-01. [2025-11-04T18:40:28.267200] [DEBUG] [VALIDATOR ] EL: All expected trading days are present. [2025-11-04T18:40:28.267458] [DEBUG] [VALIDATOR ] EL: Primary data validation passed. [2025-11-04T18:40:28.268826] [INFO ] [MAIN ] EL: Fetched primary data is valid. [2025-11-04T18:40:28.268920] [DEBUG] [LOCAL ] EL: Saving primary data. [2025-11-04T18:40:28.299445] [DEBUG] [LOCAL ] EL: Trimmed 77448 records to 53364 (keeping 200 days of data) [2025-11-04T18:40:28.300664] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-11-04T18:40:28.667450] [DEBUG] [LOCAL ] EL: Primary data saved (53364 records). [2025-11-04T18:40:28.667598] [INFO ] [MAIN ] EL: Full history saved. [2025-11-04T18:40:28.667703] [INFO ] [MAIN ] Update Progress: 723/2367 symbols completed. EL took 33.1s. Estimated remaining: 3h 48m 21s. Rate: 7.2 symbols/min [2025-11-04T18:40:28.838938] [INFO ] [MAIN ] ELPC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:40:28.839103] [DEBUG] [LOCAL ] ELPC: No existing data file found in primary storage. [2025-11-04T18:40:28.839141] [WARN ] [MAIN ] ELPC: No existing data found, performing full refresh instead. [2025-11-04T18:40:28.839253] [INFO ] [MAIN ] ELPC: Wiping existing primary data before full refresh. [2025-11-04T18:40:28.839295] [INFO ] [LOCAL ] ELPC: Starting deletion of primary data. [2025-11-04T18:40:28.839337] [DEBUG] [LOCAL ] ELPC: No primary data file found to delete. [2025-11-04T18:40:28.839370] [INFO ] [MAIN ] ELPC: Starting full primary refresh. [2025-11-04T18:40:28.970523] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-09. [2025-11-04T18:40:29.165953] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T18:40:29.234085] [INFO ] [REMOTE ] ELPC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:40:29.767070] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-05. [2025-11-04T18:40:30.470048] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T18:40:30.567012] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-10. [2025-11-04T18:40:31.031272] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T18:40:31.367042] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-06. [2025-11-04T18:40:32.141107] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-11-04T18:40:32.167034] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-02. [2025-11-04T18:40:32.256232] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T18:40:32.960470] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T18:40:32.966994] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-03. [2025-11-04T18:40:33.742528] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-11-04T18:40:33.767021] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-04. [2025-11-04T18:40:34.567049] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-11. [2025-11-04T18:40:34.732001] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-11-04T18:40:35.367052] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-01. [2025-11-04T18:40:35.930470] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T18:40:36.167055] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-07. [2025-11-04T18:40:36.189083] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-11 (1.62 seconds). [2025-11-04T18:40:36.190932] [INFO ] [REMOTE ] ELF: Full fetch complete (75390 records). [2025-11-04T18:40:36.191016] [DEBUG] [VALIDATOR ] ELF: Validating primary data. [2025-11-04T18:40:36.578089] [DEBUG] [VALIDATOR ] ELF: All expected trading days are present. [2025-11-04T18:40:36.578171] [DEBUG] [VALIDATOR ] ELF: Primary data validation passed. [2025-11-04T18:40:36.579123] [INFO ] [MAIN ] ELF: Fetched primary data is valid. [2025-11-04T18:40:36.579214] [DEBUG] [LOCAL ] ELF: Saving primary data. [2025-11-04T18:40:36.580612] [DEBUG] [LOCAL ] ELF: Trimmed 75390 records to 51527 (keeping 200 days of data) [2025-11-04T18:40:36.890612] [DEBUG] [LOCAL ] ELF: Primary data saved (51527 records). [2025-11-04T18:40:36.890788] [INFO ] [MAIN ] ELF: Full history saved. [2025-11-04T18:40:36.890902] [INFO ] [MAIN ] Update Progress: 724/2367 symbols completed. ELF took 41.3s. Estimated remaining: 3h 48m 12s. Rate: 7.2 symbols/min [2025-11-04T18:40:36.972242] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-02. [2025-11-04T18:40:37.083942] [INFO ] [MAIN ] ELS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:40:37.084172] [DEBUG] [LOCAL ] ELS: No existing data file found in primary storage. [2025-11-04T18:40:37.084254] [WARN ] [MAIN ] ELS: No existing data found, performing full refresh instead. [2025-11-04T18:40:37.084473] [INFO ] [MAIN ] ELS: Wiping existing primary data before full refresh. [2025-11-04T18:40:37.084529] [INFO ] [LOCAL ] ELS: Starting deletion of primary data. [2025-11-04T18:40:37.084573] [DEBUG] [LOCAL ] ELS: No primary data file found to delete. [2025-11-04T18:40:37.084606] [INFO ] [MAIN ] ELS: Starting full primary refresh. [2025-11-04T18:40:37.295674] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T18:40:37.356235] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-11-04T18:40:37.489942] [INFO ] [REMOTE ] ELS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:40:37.767055] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-05. [2025-11-04T18:40:38.567032] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-08. [2025-11-04T18:40:38.650186] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T18:40:39.367090] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-03. [2025-11-04T18:40:39.633568] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-03 (0.27 seconds). [2025-11-04T18:40:39.662914] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-11-04T18:40:40.167081] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-01. [2025-11-04T18:40:40.967030] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-06. [2025-11-04T18:40:41.608877] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T18:40:41.767036] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-02. [2025-11-04T18:40:41.869271] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-11-04T18:40:42.567034] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-04. [2025-11-04T18:40:42.818454] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-04 (0.25 seconds). [2025-11-04T18:40:42.819727] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T18:40:43.367090] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-09. [2025-11-04T18:40:44.167078] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-05. [2025-11-04T18:40:44.406538] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-11-04T18:40:44.618899] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T18:40:44.967059] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-10. [2025-11-04T18:40:45.767051] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-03. [2025-11-04T18:40:46.185876] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T18:40:46.567055] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-06. [2025-11-04T18:40:46.931818] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-06 (0.36 seconds). [2025-11-04T18:40:47.143871] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T18:40:47.367049] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-07. [2025-11-04T18:40:48.167040] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-11. [2025-11-04T18:40:48.287239] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T18:40:48.967031] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-07. [2025-11-04T18:40:49.137170] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T18:40:49.139733] [INFO ] [REMOTE ] ELME: Full fetch complete (56691 records). [2025-11-04T18:40:49.139857] [DEBUG] [VALIDATOR ] ELME: Validating primary data. [2025-11-04T18:40:49.337071] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-11-04T18:40:49.532866] [DEBUG] [VALIDATOR ] ELME: All expected trading days are present. [2025-11-04T18:40:49.532955] [DEBUG] [VALIDATOR ] ELME: Primary data validation passed. [2025-11-04T18:40:49.533724] [INFO ] [MAIN ] ELME: Fetched primary data is valid. [2025-11-04T18:40:49.533789] [DEBUG] [LOCAL ] ELME: Saving primary data. [2025-11-04T18:40:49.535038] [DEBUG] [LOCAL ] ELME: Trimmed 56691 records to 39573 (keeping 200 days of data) [2025-11-04T18:40:49.768248] [DEBUG] [LOCAL ] ELME: Primary data saved (39573 records). [2025-11-04T18:40:49.768424] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-04. [2025-11-04T18:40:49.768696] [INFO ] [MAIN ] ELME: Full history saved. [2025-11-04T18:40:49.769963] [INFO ] [MAIN ] Update Progress: 725/2367 symbols completed. ELME took 41.6s. Estimated remaining: 3h 48m 14s. Rate: 7.2 symbols/min [2025-11-04T18:40:49.936866] [INFO ] [MAIN ] ELV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:40:49.985829] [DEBUG] [LOCAL ] ELV: Loaded 50726 existing records from primary storage. [2025-11-04T18:40:49.986102] [INFO ] [MAIN ] ELV: Loaded 50726 existing records. [2025-11-04T18:40:50.166344] [WARN ] [MAIN ] ELV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:40:50.166587] [INFO ] [MAIN ] ELV: Wiping existing primary data before full refresh. [2025-11-04T18:40:50.166636] [INFO ] [LOCAL ] ELV: Starting deletion of primary data. [2025-11-04T18:40:50.166923] [DEBUG] [LOCAL ] ELV: Deleted primary data file. [2025-11-04T18:40:50.166980] [INFO ] [MAIN ] ELV: Starting full primary refresh. [2025-11-04T18:40:50.522133] [INFO ] [REMOTE ] ELV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:40:50.567044] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-01. [2025-11-04T18:40:51.196970] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T18:40:51.367073] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-08. [2025-11-04T18:40:52.126225] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-11-04T18:40:52.166998] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-02. [2025-11-04T18:40:52.225617] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-11-04T18:40:52.967021] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-05. [2025-11-04T18:40:53.632731] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-11-04T18:40:53.767036] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-03. [2025-11-04T18:40:54.284789] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:40:54.567044] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-06. [2025-11-04T18:40:55.081096] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T18:40:55.367031] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-04. [2025-11-04T18:40:55.648187] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T18:40:56.166981] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-07. [2025-11-04T18:40:56.704316] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T18:40:56.967067] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-08. [2025-11-04T18:40:57.349727] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-11-04T18:40:57.439042] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T18:40:57.767013] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-08. [2025-11-04T18:40:58.567082] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-05. [2025-11-04T18:40:59.114785] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-11-04T18:40:59.367043] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-09. [2025-11-04T18:41:00.128812] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-11-04T18:41:00.167006] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-06. [2025-11-04T18:41:00.967085] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-09. [2025-11-04T18:41:01.251241] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-11-04T18:41:01.497272] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T18:41:01.767065] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-09. [2025-11-04T18:41:01.835922] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-11-04T18:41:01.972582] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-09 (0.21 seconds). [2025-11-04T18:41:02.567087] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-07. [2025-11-04T18:41:03.367053] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-10. [2025-11-04T18:41:04.167030] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-10. [2025-11-04T18:41:04.214752] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-11-04T18:41:04.407028] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-10 (0.24 seconds). [2025-11-04T18:41:04.410208] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-11-04T18:41:04.967046] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-11. [2025-11-04T18:41:05.767045] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-10. [2025-11-04T18:41:05.825550] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T18:41:05.826961] [INFO ] [REMOTE ] ELP: Full fetch complete (36710 records). [2025-11-04T18:41:05.827068] [DEBUG] [VALIDATOR ] ELP: Validating primary data. [2025-11-04T18:41:06.363421] [DEBUG] [VALIDATOR ] ELP: All expected trading days are present. [2025-11-04T18:41:06.363543] [DEBUG] [VALIDATOR ] ELP: Primary data validation passed. [2025-11-04T18:41:06.364828] [INFO ] [MAIN ] ELP: Fetched primary data is valid. [2025-11-04T18:41:06.364919] [DEBUG] [LOCAL ] ELP: Saving primary data. [2025-11-04T18:41:06.366289] [DEBUG] [LOCAL ] ELP: Trimmed 36710 records to 25143 (keeping 200 days of data) [2025-11-04T18:41:06.548966] [DEBUG] [LOCAL ] ELP: Primary data saved (25143 records). [2025-11-04T18:41:06.549110] [INFO ] [MAIN ] ELP: Full history saved. [2025-11-04T18:41:06.549200] [INFO ] [MAIN ] Update Progress: 726/2367 symbols completed. ELP took 39.8s. Estimated remaining: 3h 48m 25s. Rate: 7.2 symbols/min [2025-11-04T18:41:06.573628] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-08. [2025-11-04T18:41:06.760372] [INFO ] [MAIN ] EMA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:06.760615] [DEBUG] [LOCAL ] EMA: No existing data file found in primary storage. [2025-11-04T18:41:06.760705] [WARN ] [MAIN ] EMA: No existing data found, performing full refresh instead. [2025-11-04T18:41:06.760857] [INFO ] [MAIN ] EMA: Wiping existing primary data before full refresh. [2025-11-04T18:41:06.760941] [INFO ] [LOCAL ] EMA: Starting deletion of primary data. [2025-11-04T18:41:06.761007] [DEBUG] [LOCAL ] EMA: No primary data file found to delete. [2025-11-04T18:41:06.761056] [INFO ] [MAIN ] EMA: Starting full primary refresh. [2025-11-04T18:41:07.158584] [INFO ] [REMOTE ] EMA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:07.272829] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T18:41:07.367055] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-11. [2025-11-04T18:41:07.786760] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T18:41:07.787418] [INFO ] [REMOTE ] ELPC: Full fetch complete (1242 records). [2025-11-04T18:41:07.787485] [DEBUG] [VALIDATOR ] ELPC: Validating primary data. [2025-11-04T18:41:07.978124] [WARN ] [VALIDATOR ] ELPC: Validation failed - insufficient historical data. Have 190 unique days, need 200 days [2025-11-04T18:41:07.978353] [FATAL] [MAIN ] ELPC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:41:07.978468] [WARN ] [MAIN ] ELPC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:41:07.978550] [INFO ] [LOCAL ] ELPC: Starting deletion of primary data. [2025-11-04T18:41:07.978750] [DEBUG] [LOCAL ] ELPC: No primary data file found to delete. [2025-11-04T18:41:07.978817] [INFO ] [LOCAL ] ELPC: Starting deletion of derivative data. [2025-11-04T18:41:07.978934] [DEBUG] [LOCAL ] ELPC: No derivative data file found to delete. [2025-11-04T18:41:07.979056] [INFO ] [MAIN ] Update Progress: 727/2367 symbols completed. ELPC took 39.3s. Estimated remaining: 3h 48m 1s. Rate: 7.2 symbols/min [2025-11-04T18:41:08.155191] [INFO ] [MAIN ] EMBJ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:08.155376] [DEBUG] [LOCAL ] EMBJ: No existing data file found in primary storage. [2025-11-04T18:41:08.155492] [WARN ] [MAIN ] EMBJ: No existing data found, performing full refresh instead. [2025-11-04T18:41:08.155703] [INFO ] [MAIN ] EMBJ: Wiping existing primary data before full refresh. [2025-11-04T18:41:08.155881] [INFO ] [LOCAL ] EMBJ: Starting deletion of primary data. [2025-11-04T18:41:08.155966] [DEBUG] [LOCAL ] EMBJ: No primary data file found to delete. [2025-11-04T18:41:08.156115] [INFO ] [MAIN ] EMBJ: Starting full primary refresh. [2025-11-04T18:41:08.177213] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-11. [2025-11-04T18:41:08.478132] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-11-04T18:41:08.565484] [INFO ] [REMOTE ] EMBJ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:08.967064] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-01. [2025-11-04T18:41:09.112233] [DEBUG] [REMOTE ] EMA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:41:09.112605] [WARN ] [REMOTE ] EMA: No data returned for month 2025-01 [2025-11-04T18:41:09.112715] [FATAL] [REMOTE ] EMA: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:41:09.112761] [FATAL] [MAIN ] EMA: Failed to fetch primary data. [2025-11-04T18:41:09.112793] [WARN ] [MAIN ] EMA: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:41:09.112853] [INFO ] [LOCAL ] EMA: Starting deletion of primary data. [2025-11-04T18:41:09.112913] [DEBUG] [LOCAL ] EMA: No primary data file found to delete. [2025-11-04T18:41:09.112953] [INFO ] [LOCAL ] EMA: Starting deletion of derivative data. [2025-11-04T18:41:09.113022] [DEBUG] [LOCAL ] EMA: No derivative data file found to delete. [2025-11-04T18:41:09.113124] [INFO ] [MAIN ] Update Progress: 728/2367 symbols completed. EMA took 2.6s. Estimated remaining: 3h 47m 36s. Rate: 7.2 symbols/min [2025-11-04T18:41:09.264186] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T18:41:09.288628] [INFO ] [MAIN ] EMD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:09.290394] [DEBUG] [LOCAL ] EMD: No existing data file found in primary storage. [2025-11-04T18:41:09.290559] [WARN ] [MAIN ] EMD: No existing data found, performing full refresh instead. [2025-11-04T18:41:09.290708] [INFO ] [MAIN ] EMD: Wiping existing primary data before full refresh. [2025-11-04T18:41:09.290761] [INFO ] [LOCAL ] EMD: Starting deletion of primary data. [2025-11-04T18:41:09.290803] [DEBUG] [LOCAL ] EMD: No primary data file found to delete. [2025-11-04T18:41:09.290834] [INFO ] [MAIN ] EMD: Starting full primary refresh. [2025-11-04T18:41:09.290931] [INFO ] [REMOTE ] ELS: Full fetch complete (71674 records). [2025-11-04T18:41:09.335732] [DEBUG] [VALIDATOR ] ELS: Validating primary data. [2025-11-04T18:41:09.772168] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-09. [2025-11-04T18:41:09.997348] [INFO ] [REMOTE ] EMD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:10.075458] [DEBUG] [VALIDATOR ] ELS: All expected trading days are present. [2025-11-04T18:41:10.075574] [DEBUG] [VALIDATOR ] ELS: Primary data validation passed. [2025-11-04T18:41:10.076375] [INFO ] [MAIN ] ELS: Fetched primary data is valid. [2025-11-04T18:41:10.076435] [DEBUG] [LOCAL ] ELS: Saving primary data. [2025-11-04T18:41:10.077765] [DEBUG] [LOCAL ] ELS: Trimmed 71674 records to 49930 (keeping 200 days of data) [2025-11-04T18:41:10.370394] [DEBUG] [LOCAL ] ELS: Primary data saved (49930 records). [2025-11-04T18:41:10.370531] [INFO ] [MAIN ] ELS: Full history saved. [2025-11-04T18:41:10.370602] [INFO ] [MAIN ] Update Progress: 729/2367 symbols completed. ELS took 33.5s. Estimated remaining: 3h 47m 12s. Rate: 7.2 symbols/min [2025-11-04T18:41:10.534983] [INFO ] [MAIN ] EME: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:10.564577] [DEBUG] [LOCAL ] EME: Loaded 30672 existing records from primary storage. [2025-11-04T18:41:10.564863] [INFO ] [MAIN ] EME: Loaded 30672 existing records. [2025-11-04T18:41:10.575955] [INFO ] [REMOTE ] EMBJ: Fetching primary data for 2025-01. [2025-11-04T18:41:10.745524] [WARN ] [MAIN ] EME: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:41:10.745838] [INFO ] [MAIN ] EME: Wiping existing primary data before full refresh. [2025-11-04T18:41:10.745894] [INFO ] [LOCAL ] EME: Starting deletion of primary data. [2025-11-04T18:41:10.746079] [DEBUG] [LOCAL ] EME: Deleted primary data file. [2025-11-04T18:41:10.746137] [INFO ] [MAIN ] EME: Starting full primary refresh. [2025-11-04T18:41:10.843004] [DEBUG] [REMOTE ] EMBJ: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:41:10.845789] [WARN ] [REMOTE ] EMBJ: No data returned for month 2025-01 [2025-11-04T18:41:10.861443] [FATAL] [REMOTE ] EMBJ: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:41:10.866623] [FATAL] [MAIN ] EMBJ: Failed to fetch primary data. [2025-11-04T18:41:10.884079] [WARN ] [MAIN ] EMBJ: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:41:10.902360] [INFO ] [LOCAL ] EMBJ: Starting deletion of primary data. [2025-11-04T18:41:10.953236] [DEBUG] [LOCAL ] EMBJ: No primary data file found to delete. [2025-11-04T18:41:10.990094] [INFO ] [LOCAL ] EMBJ: Starting deletion of derivative data. [2025-11-04T18:41:11.011263] [DEBUG] [LOCAL ] EMBJ: No derivative data file found to delete. [2025-11-04T18:41:11.021404] [INFO ] [MAIN ] Update Progress: 730/2367 symbols completed. EMBJ took 3.0s. Estimated remaining: 3h 46m 47s. Rate: 7.2 symbols/min [2025-11-04T18:41:11.211484] [INFO ] [REMOTE ] EME: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:11.281432] [INFO ] [MAIN ] EMF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:11.287745] [DEBUG] [LOCAL ] EMF: Loaded 3058 existing records from primary storage. [2025-11-04T18:41:11.287973] [INFO ] [MAIN ] EMF: Loaded 3058 existing records. [2025-11-04T18:41:11.370328] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-01. [2025-11-04T18:41:11.472001] [WARN ] [MAIN ] EMF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:41:11.472333] [INFO ] [MAIN ] EMF: Wiping existing primary data before full refresh. [2025-11-04T18:41:11.481850] [INFO ] [LOCAL ] EMF: Starting deletion of primary data. [2025-11-04T18:41:11.495473] [DEBUG] [LOCAL ] EMF: Deleted primary data file. [2025-11-04T18:41:11.498360] [INFO ] [MAIN ] EMF: Starting full primary refresh. [2025-11-04T18:41:11.575093] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-11-04T18:41:11.859115] [INFO ] [REMOTE ] EMF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:12.167055] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-01. [2025-11-04T18:41:12.458486] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-11-04T18:41:12.719401] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-11-04T18:41:12.967031] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-02. [2025-11-04T18:41:13.767041] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-10. [2025-11-04T18:41:13.963556] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T18:41:14.567041] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-02. [2025-11-04T18:41:15.010998] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-11-04T18:41:15.367063] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-01. [2025-11-04T18:41:15.473345] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-11-04T18:41:16.167010] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-03. [2025-11-04T18:41:16.324695] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T18:41:16.519584] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-11-04T18:41:16.967040] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-02. [2025-11-04T18:41:17.767511] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-11-04T18:41:17.767014] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-03. [2025-11-04T18:41:18.567049] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-04. [2025-11-04T18:41:18.881910] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T18:41:18.962390] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-11-04T18:41:19.367035] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-04. [2025-11-04T18:41:20.167079] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-11. [2025-11-04T18:41:20.536873] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-11-04T18:41:20.967074] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-03. [2025-11-04T18:41:21.421317] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T18:41:21.423498] [INFO ] [REMOTE ] ELV: Full fetch complete (73306 records). [2025-11-04T18:41:21.423736] [DEBUG] [VALIDATOR ] ELV: Validating primary data. [2025-11-04T18:41:21.767115] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-05. [2025-11-04T18:41:21.918260] [DEBUG] [VALIDATOR ] ELV: All expected trading days are present. [2025-11-04T18:41:21.934556] [DEBUG] [VALIDATOR ] ELV: Primary data validation passed. [2025-11-04T18:41:21.935607] [INFO ] [MAIN ] ELV: Fetched primary data is valid. [2025-11-04T18:41:21.937707] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-11-04T18:41:21.937792] [DEBUG] [LOCAL ] ELV: Saving primary data. [2025-11-04T18:41:21.945737] [DEBUG] [LOCAL ] ELV: Trimmed 73306 records to 50726 (keeping 200 days of data) [2025-11-04T18:41:22.176104] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-11-04T18:41:22.275101] [DEBUG] [LOCAL ] ELV: Primary data saved (50726 records). [2025-11-04T18:41:22.275244] [INFO ] [MAIN ] ELV: Full history saved. [2025-11-04T18:41:22.275328] [INFO ] [MAIN ] Update Progress: 731/2367 symbols completed. ELV took 32.5s. Estimated remaining: 3h 46m 45s. Rate: 7.2 symbols/min [2025-11-04T18:41:22.439741] [INFO ] [MAIN ] EMN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:22.485975] [DEBUG] [LOCAL ] EMN: Loaded 50866 existing records from primary storage. [2025-11-04T18:41:22.486294] [INFO ] [MAIN ] EMN: Loaded 50866 existing records. [2025-11-04T18:41:22.572345] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-04. [2025-11-04T18:41:22.661540] [WARN ] [MAIN ] EMN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:41:22.661757] [INFO ] [MAIN ] EMN: Wiping existing primary data before full refresh. [2025-11-04T18:41:22.661847] [INFO ] [LOCAL ] EMN: Starting deletion of primary data. [2025-11-04T18:41:22.662157] [DEBUG] [LOCAL ] EMN: Deleted primary data file. [2025-11-04T18:41:22.662195] [INFO ] [MAIN ] EMN: Starting full primary refresh. [2025-11-04T18:41:23.011622] [INFO ] [REMOTE ] EMN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:23.367032] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-05. [2025-11-04T18:41:23.456267] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-11-04T18:41:24.167065] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-06. [2025-11-04T18:41:24.589585] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T18:41:24.616050] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-11-04T18:41:24.967014] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-05. [2025-11-04T18:41:25.767055] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-01. [2025-11-04T18:41:25.853115] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T18:41:26.567047] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-06. [2025-11-04T18:41:27.051808] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T18:41:27.360832] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T18:41:27.367011] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-02. [2025-11-04T18:41:28.167034] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-07. [2025-11-04T18:41:28.522316] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-11-04T18:41:28.574294] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T18:41:28.967074] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-03. [2025-11-04T18:41:29.767059] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-07. [2025-11-04T18:41:30.114622] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T18:41:30.567019] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-06. [2025-11-04T18:41:30.623879] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-11-04T18:41:31.367047] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-08. [2025-11-04T18:41:31.659422] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T18:41:32.066614] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T18:41:32.167078] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-09. [2025-11-04T18:41:32.906864] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-11-04T18:41:32.967055] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-04. [2025-11-04T18:41:33.767073] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-07. [2025-11-04T18:41:34.252970] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T18:41:34.567041] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-05. [2025-11-04T18:41:34.714069] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-11-04T18:41:35.367040] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-08. [2025-11-04T18:41:35.794386] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T18:41:36.168114] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-08. [2025-11-04T18:41:36.171847] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-11-04T18:41:36.638412] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-11-04T18:41:36.967053] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-06. [2025-11-04T18:41:37.767054] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-09. [2025-11-04T18:41:38.302308] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T18:41:38.567064] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-09. [2025-11-04T18:41:38.733095] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-11-04T18:41:38.937417] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-11-04T18:41:39.367063] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-10. [2025-11-04T18:41:39.722812] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-11-04T18:41:40.167059] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-11. [2025-11-04T18:41:40.578914] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T18:41:40.579704] [INFO ] [REMOTE ] EMF: Full fetch complete (5833 records). [2025-11-04T18:41:40.579774] [DEBUG] [VALIDATOR ] EMF: Validating primary data. [2025-11-04T18:41:40.936423] [DEBUG] [VALIDATOR ] EMF: All expected trading days are present. [2025-11-04T18:41:40.936498] [DEBUG] [VALIDATOR ] EMF: Primary data validation passed. [2025-11-04T18:41:40.936679] [INFO ] [MAIN ] EMF: Fetched primary data is valid. [2025-11-04T18:41:40.936766] [DEBUG] [LOCAL ] EMF: Saving primary data. [2025-11-04T18:41:40.937128] [DEBUG] [LOCAL ] EMF: Trimmed 5833 records to 3088 (keeping 200 days of data) [2025-11-04T18:41:40.955219] [DEBUG] [LOCAL ] EMF: Primary data saved (3088 records). [2025-11-04T18:41:40.955284] [INFO ] [MAIN ] EMF: Full history saved. [2025-11-04T18:41:40.955350] [INFO ] [MAIN ] Update Progress: 732/2367 symbols completed. EMF took 29.9s. Estimated remaining: 3h 46m 60s. Rate: 7.2 symbols/min [2025-11-04T18:41:40.975822] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-10. [2025-11-04T18:41:41.129429] [INFO ] [MAIN ] EMO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:41.144076] [DEBUG] [LOCAL ] EMO: Loaded 13436 existing records from primary storage. [2025-11-04T18:41:41.144320] [INFO ] [MAIN ] EMO: Loaded 13436 existing records. [2025-11-04T18:41:41.319048] [WARN ] [MAIN ] EMO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:41:41.319281] [INFO ] [MAIN ] EMO: Wiping existing primary data before full refresh. [2025-11-04T18:41:41.319330] [INFO ] [LOCAL ] EMO: Starting deletion of primary data. [2025-11-04T18:41:41.319475] [DEBUG] [LOCAL ] EMO: Deleted primary data file. [2025-11-04T18:41:41.319512] [INFO ] [MAIN ] EMO: Starting full primary refresh. [2025-11-04T18:41:41.682325] [INFO ] [REMOTE ] EMO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:41.767103] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-07. [2025-11-04T18:41:42.166028] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-11-04T18:41:42.567076] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-10. [2025-11-04T18:41:42.942095] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T18:41:43.353721] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T18:41:43.366997] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-11. [2025-11-04T18:41:44.027396] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T18:41:44.028795] [INFO ] [REMOTE ] EMD: Full fetch complete (31911 records). [2025-11-04T18:41:44.028868] [DEBUG] [VALIDATOR ] EMD: Validating primary data. [2025-11-04T18:41:44.176063] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-08. [2025-11-04T18:41:44.398196] [DEBUG] [VALIDATOR ] EMD: All expected trading days are present. [2025-11-04T18:41:44.398268] [DEBUG] [VALIDATOR ] EMD: Primary data validation passed. [2025-11-04T18:41:44.398762] [INFO ] [MAIN ] EMD: Fetched primary data is valid. [2025-11-04T18:41:44.398821] [DEBUG] [LOCAL ] EMD: Saving primary data. [2025-11-04T18:41:44.399531] [DEBUG] [LOCAL ] EMD: Trimmed 31911 records to 20623 (keeping 200 days of data) [2025-11-04T18:41:44.519106] [DEBUG] [LOCAL ] EMD: Primary data saved (20623 records). [2025-11-04T18:41:44.519228] [INFO ] [MAIN ] EMD: Full history saved. [2025-11-04T18:41:44.519294] [INFO ] [MAIN ] Update Progress: 733/2367 symbols completed. EMD took 35.4s. Estimated remaining: 3h 46m 41s. Rate: 7.2 symbols/min [2025-11-04T18:41:44.684162] [INFO ] [MAIN ] EMP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:44.684340] [DEBUG] [LOCAL ] EMP: No existing data file found in primary storage. [2025-11-04T18:41:44.684376] [WARN ] [MAIN ] EMP: No existing data found, performing full refresh instead. [2025-11-04T18:41:44.684481] [INFO ] [MAIN ] EMP: Wiping existing primary data before full refresh. [2025-11-04T18:41:44.684521] [INFO ] [LOCAL ] EMP: Starting deletion of primary data. [2025-11-04T18:41:44.684561] [DEBUG] [LOCAL ] EMP: No primary data file found to delete. [2025-11-04T18:41:44.684591] [INFO ] [MAIN ] EMP: Starting full primary refresh. [2025-11-04T18:41:44.970484] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-11. [2025-11-04T18:41:45.056966] [INFO ] [REMOTE ] EMP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:45.641208] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-11-04T18:41:45.767045] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-01. [2025-11-04T18:41:46.284461] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-11 (1.31 seconds). [2025-11-04T18:41:46.286908] [INFO ] [REMOTE ] EME: Full fetch complete (47415 records). [2025-11-04T18:41:46.287019] [DEBUG] [VALIDATOR ] EME: Validating primary data. [2025-11-04T18:41:46.568825] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-09. [2025-11-04T18:41:46.610469] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-11-04T18:41:46.707713] [DEBUG] [VALIDATOR ] EME: All expected trading days are present. [2025-11-04T18:41:46.707799] [DEBUG] [VALIDATOR ] EME: Primary data validation passed. [2025-11-04T18:41:46.708476] [INFO ] [MAIN ] EME: Fetched primary data is valid. [2025-11-04T18:41:46.708540] [DEBUG] [LOCAL ] EME: Saving primary data. [2025-11-04T18:41:46.709551] [DEBUG] [LOCAL ] EME: Trimmed 47415 records to 30746 (keeping 200 days of data) [2025-11-04T18:41:46.900286] [DEBUG] [LOCAL ] EME: Primary data saved (30746 records). [2025-11-04T18:41:46.900412] [INFO ] [MAIN ] EME: Full history saved. [2025-11-04T18:41:46.900486] [INFO ] [MAIN ] Update Progress: 734/2367 symbols completed. EME took 36.5s. Estimated remaining: 3h 46m 19s. Rate: 7.2 symbols/min [2025-11-04T18:41:47.082963] [INFO ] [MAIN ] EMR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:47.135005] [DEBUG] [LOCAL ] EMR: Loaded 53117 existing records from primary storage. [2025-11-04T18:41:47.135390] [INFO ] [MAIN ] EMR: Loaded 53117 existing records. [2025-11-04T18:41:47.315804] [WARN ] [MAIN ] EMR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:41:47.316056] [INFO ] [MAIN ] EMR: Wiping existing primary data before full refresh. [2025-11-04T18:41:47.316103] [INFO ] [LOCAL ] EMR: Starting deletion of primary data. [2025-11-04T18:41:47.316307] [DEBUG] [LOCAL ] EMR: Deleted primary data file. [2025-11-04T18:41:47.316361] [INFO ] [MAIN ] EMR: Starting full primary refresh. [2025-11-04T18:41:47.377581] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-01. [2025-11-04T18:41:47.687582] [INFO ] [REMOTE ] EMR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:47.896391] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T18:41:47.968766] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-11-04T18:41:48.167042] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-02. [2025-11-04T18:41:48.459806] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-11-04T18:41:48.967021] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-03. [2025-11-04T18:41:49.244239] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-03 (0.28 seconds). [2025-11-04T18:41:49.767038] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-01. [2025-11-04T18:41:50.567015] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-04. [2025-11-04T18:41:50.942343] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T18:41:51.114227] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T18:41:51.367050] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-02. [2025-11-04T18:41:51.906394] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-11-04T18:41:52.167022] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-03. [2025-11-04T18:41:52.649287] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-11-04T18:41:52.967050] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-04. [2025-11-04T18:41:53.468612] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-11-04T18:41:53.767093] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-10. [2025-11-04T18:41:54.567063] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-05. [2025-11-04T18:41:54.924815] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-11-04T18:41:55.258538] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-11-04T18:41:55.367011] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-11. [2025-11-04T18:41:56.167036] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-06. [2025-11-04T18:41:56.445806] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-06 (0.28 seconds). [2025-11-04T18:41:56.499899] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T18:41:56.502756] [INFO ] [REMOTE ] EMN: Full fetch complete (72716 records). [2025-11-04T18:41:56.502901] [DEBUG] [VALIDATOR ] EMN: Validating primary data. [2025-11-04T18:41:56.930705] [DEBUG] [VALIDATOR ] EMN: All expected trading days are present. [2025-11-04T18:41:56.930817] [DEBUG] [VALIDATOR ] EMN: Primary data validation passed. [2025-11-04T18:41:56.931732] [INFO ] [MAIN ] EMN: Fetched primary data is valid. [2025-11-04T18:41:56.931794] [DEBUG] [LOCAL ] EMN: Saving primary data. [2025-11-04T18:41:56.933058] [DEBUG] [LOCAL ] EMN: Trimmed 72716 records to 50890 (keeping 200 days of data) [2025-11-04T18:41:56.985369] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-02. [2025-11-04T18:41:57.230421] [DEBUG] [LOCAL ] EMN: Primary data saved (50890 records). [2025-11-04T18:41:57.230580] [INFO ] [MAIN ] EMN: Full history saved. [2025-11-04T18:41:57.230683] [INFO ] [MAIN ] Update Progress: 735/2367 symbols completed. EMN took 35.0s. Estimated remaining: 3h 46m 15s. Rate: 7.2 symbols/min [2025-11-04T18:41:57.396333] [INFO ] [MAIN ] ENB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:41:57.396493] [DEBUG] [LOCAL ] ENB: No existing data file found in primary storage. [2025-11-04T18:41:57.396529] [WARN ] [MAIN ] ENB: No existing data found, performing full refresh instead. [2025-11-04T18:41:57.396633] [INFO ] [MAIN ] ENB: Wiping existing primary data before full refresh. [2025-11-04T18:41:57.396726] [INFO ] [LOCAL ] ENB: Starting deletion of primary data. [2025-11-04T18:41:57.396793] [DEBUG] [LOCAL ] ENB: No primary data file found to delete. [2025-11-04T18:41:57.396826] [INFO ] [MAIN ] ENB: Starting full primary refresh. [2025-11-04T18:41:57.744530] [INFO ] [REMOTE ] ENB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:41:57.766998] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-01. [2025-11-04T18:41:58.389569] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-11-04T18:41:58.567020] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-07. [2025-11-04T18:41:59.010853] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-11-04T18:41:59.180033] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T18:41:59.367081] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-02. [2025-11-04T18:42:00.167081] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-03. [2025-11-04T18:42:00.696145] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-11-04T18:42:00.967065] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-05. [2025-11-04T18:42:01.558423] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T18:42:01.690874] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T18:42:01.767097] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-03. [2025-11-04T18:42:02.567048] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-06. [2025-11-04T18:42:03.179545] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T18:42:03.181523] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T18:42:03.367056] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-04. [2025-11-04T18:42:04.167033] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-08. [2025-11-04T18:42:04.483756] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-11-04T18:42:04.762805] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T18:42:04.967066] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-05. [2025-11-04T18:42:05.767043] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-09. [2025-11-04T18:42:06.078736] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-09 (0.31 seconds). [2025-11-04T18:42:06.374243] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T18:42:06.567050] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-04. [2025-11-04T18:42:07.367113] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-06. [2025-11-04T18:42:07.895483] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T18:42:08.167110] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-07. [2025-11-04T18:42:08.718619] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-11-04T18:42:08.843457] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-11-04T18:42:08.967107] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-10. [2025-11-04T18:42:09.504672] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-11-04T18:42:09.767069] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-08. [2025-11-04T18:42:10.308304] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-11-04T18:42:10.567058] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-09. [2025-11-04T18:42:11.374566] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-05. [2025-11-04T18:42:11.376446] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-11-04T18:42:12.167084] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-07. [2025-11-04T18:42:12.717091] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T18:42:12.967030] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-06. [2025-11-04T18:42:13.767005] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-11. [2025-11-04T18:42:13.832021] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-11-04T18:42:14.116511] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-11 (0.35 seconds). [2025-11-04T18:42:14.117207] [INFO ] [REMOTE ] EMP: Full fetch complete (4058 records). [2025-11-04T18:42:14.117275] [DEBUG] [VALIDATOR ] EMP: Validating primary data. [2025-11-04T18:42:14.481046] [DEBUG] [VALIDATOR ] EMP: All expected trading days are present. [2025-11-04T18:42:14.481260] [DEBUG] [VALIDATOR ] EMP: Primary data validation passed. [2025-11-04T18:42:14.481568] [INFO ] [MAIN ] EMP: Fetched primary data is valid. [2025-11-04T18:42:14.481766] [DEBUG] [LOCAL ] EMP: Saving primary data. [2025-11-04T18:42:14.482549] [DEBUG] [LOCAL ] EMP: Trimmed 4058 records to 2819 (keeping 200 days of data) [2025-11-04T18:42:14.501267] [DEBUG] [LOCAL ] EMP: Primary data saved (2819 records). [2025-11-04T18:42:14.501462] [INFO ] [MAIN ] EMP: Full history saved. [2025-11-04T18:42:14.501568] [INFO ] [MAIN ] Update Progress: 736/2367 symbols completed. EMP took 30.0s. Estimated remaining: 3h 46m 27s. Rate: 7.2 symbols/min [2025-11-04T18:42:14.571503] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-08. [2025-11-04T18:42:14.691410] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-11-04T18:42:14.727346] [INFO ] [MAIN ] ENIC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:42:14.759387] [DEBUG] [LOCAL ] ENIC: Loaded 35929 existing records from primary storage. [2025-11-04T18:42:14.759695] [INFO ] [MAIN ] ENIC: Loaded 35929 existing records. [2025-11-04T18:42:14.931396] [WARN ] [MAIN ] ENIC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:42:14.931618] [INFO ] [MAIN ] ENIC: Wiping existing primary data before full refresh. [2025-11-04T18:42:14.931714] [INFO ] [LOCAL ] ENIC: Starting deletion of primary data. [2025-11-04T18:42:14.931958] [DEBUG] [LOCAL ] ENIC: Deleted primary data file. [2025-11-04T18:42:14.932013] [INFO ] [MAIN ] ENIC: Starting full primary refresh. [2025-11-04T18:42:15.276476] [INFO ] [REMOTE ] ENIC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:42:15.367047] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-01. [2025-11-04T18:42:15.969554] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T18:42:16.167084] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-10. [2025-11-04T18:42:16.567634] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T18:42:16.800117] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-11-04T18:42:16.967081] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-11. [2025-11-04T18:42:17.767049] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-09. [2025-11-04T18:42:17.847005] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-11 (0.88 seconds). [2025-11-04T18:42:17.848410] [INFO ] [REMOTE ] EMO: Full fetch complete (17952 records). [2025-11-04T18:42:17.848505] [DEBUG] [VALIDATOR ] EMO: Validating primary data. [2025-11-04T18:42:18.223372] [DEBUG] [VALIDATOR ] EMO: All expected trading days are present. [2025-11-04T18:42:18.223447] [DEBUG] [VALIDATOR ] EMO: Primary data validation passed. [2025-11-04T18:42:18.223867] [INFO ] [MAIN ] EMO: Fetched primary data is valid. [2025-11-04T18:42:18.223927] [DEBUG] [LOCAL ] EMO: Saving primary data. [2025-11-04T18:42:18.224431] [DEBUG] [LOCAL ] EMO: Trimmed 17952 records to 13497 (keeping 200 days of data) [2025-11-04T18:42:18.302577] [DEBUG] [LOCAL ] EMO: Primary data saved (13497 records). [2025-11-04T18:42:18.302727] [INFO ] [MAIN ] EMO: Full history saved. [2025-11-04T18:42:18.302815] [INFO ] [MAIN ] Update Progress: 737/2367 symbols completed. EMO took 37.3s. Estimated remaining: 3h 46m 9s. Rate: 7.2 symbols/min [2025-11-04T18:42:18.466025] [INFO ] [MAIN ] ENO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:42:18.466145] [DEBUG] [LOCAL ] ENO: No existing data file found in primary storage. [2025-11-04T18:42:18.466181] [WARN ] [MAIN ] ENO: No existing data found, performing full refresh instead. [2025-11-04T18:42:18.466282] [INFO ] [MAIN ] ENO: Wiping existing primary data before full refresh. [2025-11-04T18:42:18.466322] [INFO ] [LOCAL ] ENO: Starting deletion of primary data. [2025-11-04T18:42:18.466361] [DEBUG] [LOCAL ] ENO: No primary data file found to delete. [2025-11-04T18:42:18.466392] [INFO ] [MAIN ] ENO: Starting full primary refresh. [2025-11-04T18:42:18.572238] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-02. [2025-11-04T18:42:18.836059] [INFO ] [REMOTE ] ENO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:42:19.234122] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-11-04T18:42:19.367093] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-10. [2025-11-04T18:42:19.445583] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T18:42:20.167051] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-03. [2025-11-04T18:42:20.879926] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T18:42:20.967006] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-07. [2025-11-04T18:42:21.075039] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T18:42:21.767046] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-04. [2025-11-04T18:42:22.267816] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T18:42:22.567038] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-08. [2025-11-04T18:42:22.886388] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T18:42:23.367046] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-01. [2025-11-04T18:42:23.865813] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-11-04T18:42:23.997152] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-11-04T18:42:24.167073] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-11. [2025-11-04T18:42:24.967056] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-02. [2025-11-04T18:42:25.308606] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-11-04T18:42:25.447348] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-11 (1.28 seconds). [2025-11-04T18:42:25.448968] [INFO ] [REMOTE ] ENB: Full fetch complete (77275 records). [2025-11-04T18:42:25.449063] [DEBUG] [VALIDATOR ] ENB: Validating primary data. [2025-11-04T18:42:25.449734] [FATAL] [VALIDATOR ] ENB: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:42:25.449823] [FATAL] [MAIN ] ENB: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:42:25.449866] [WARN ] [MAIN ] ENB: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:42:25.449905] [INFO ] [LOCAL ] ENB: Starting deletion of primary data. [2025-11-04T18:42:25.449959] [DEBUG] [LOCAL ] ENB: No primary data file found to delete. [2025-11-04T18:42:25.449992] [INFO ] [LOCAL ] ENB: Starting deletion of derivative data. [2025-11-04T18:42:25.450029] [DEBUG] [LOCAL ] ENB: No derivative data file found to delete. [2025-11-04T18:42:25.450083] [INFO ] [MAIN ] Update Progress: 738/2367 symbols completed. ENB took 28.2s. Estimated remaining: 3h 45m 58s. Rate: 7.2 symbols/min [2025-11-04T18:42:25.625109] [INFO ] [MAIN ] ENOV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:42:25.665107] [DEBUG] [LOCAL ] ENOV: Loaded 44489 existing records from primary storage. [2025-11-04T18:42:25.665402] [INFO ] [MAIN ] ENOV: Loaded 44489 existing records. [2025-11-04T18:42:25.772292] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-05. [2025-11-04T18:42:25.854328] [WARN ] [MAIN ] ENOV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:42:25.854516] [INFO ] [MAIN ] ENOV: Wiping existing primary data before full refresh. [2025-11-04T18:42:25.854563] [INFO ] [LOCAL ] ENOV: Starting deletion of primary data. [2025-11-04T18:42:25.854773] [DEBUG] [LOCAL ] ENOV: Deleted primary data file. [2025-11-04T18:42:25.854835] [INFO ] [MAIN ] ENOV: Starting full primary refresh. [2025-11-04T18:42:26.201580] [INFO ] [REMOTE ] ENOV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:42:26.567052] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-09. [2025-11-04T18:42:26.970588] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T18:42:27.367114] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-06. [2025-11-04T18:42:27.963124] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T18:42:28.167032] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-10. [2025-11-04T18:42:28.365326] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T18:42:28.967048] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-01. [2025-11-04T18:42:29.767491] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-07. [2025-11-04T18:42:29.767410] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-11-04T18:42:30.057208] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-11-04T18:42:30.567046] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-02. [2025-11-04T18:42:30.620470] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T18:42:31.367045] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-11. [2025-11-04T18:42:31.660722] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T18:42:31.672786] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-11 (0.31 seconds). [2025-11-04T18:42:31.674742] [INFO ] [REMOTE ] EMR: Full fetch complete (77180 records). [2025-11-04T18:42:31.674845] [DEBUG] [VALIDATOR ] EMR: Validating primary data. [2025-11-04T18:42:32.055620] [DEBUG] [VALIDATOR ] EMR: All expected trading days are present. [2025-11-04T18:42:32.055733] [DEBUG] [VALIDATOR ] EMR: Primary data validation passed. [2025-11-04T18:42:32.056937] [INFO ] [MAIN ] EMR: Fetched primary data is valid. [2025-11-04T18:42:32.057019] [DEBUG] [LOCAL ] EMR: Saving primary data. [2025-11-04T18:42:32.058842] [DEBUG] [LOCAL ] EMR: Trimmed 77180 records to 53117 (keeping 200 days of data) [2025-11-04T18:42:32.169685] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-03. [2025-11-04T18:42:32.381323] [DEBUG] [LOCAL ] EMR: Primary data saved (53117 records). [2025-11-04T18:42:32.381457] [INFO ] [MAIN ] EMR: Full history saved. [2025-11-04T18:42:32.381533] [INFO ] [MAIN ] Update Progress: 739/2367 symbols completed. EMR took 45.5s. Estimated remaining: 3h 45m 46s. Rate: 7.2 symbols/min [2025-11-04T18:42:32.552160] [INFO ] [MAIN ] ENR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:42:32.590900] [DEBUG] [LOCAL ] ENR: Loaded 43704 existing records from primary storage. [2025-11-04T18:42:32.591192] [INFO ] [MAIN ] ENR: Loaded 43704 existing records. [2025-11-04T18:42:32.766730] [WARN ] [MAIN ] ENR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:42:32.766972] [INFO ] [MAIN ] ENR: Wiping existing primary data before full refresh. [2025-11-04T18:42:32.767018] [INFO ] [LOCAL ] ENR: Starting deletion of primary data. [2025-11-04T18:42:32.767235] [DEBUG] [LOCAL ] ENR: Deleted primary data file. [2025-11-04T18:42:32.767290] [INFO ] [MAIN ] ENR: Starting full primary refresh. [2025-11-04T18:42:32.972258] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-08. [2025-11-04T18:42:33.122759] [INFO ] [REMOTE ] ENR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:42:33.281045] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T18:42:33.767103] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-01. [2025-11-04T18:42:34.009082] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T18:42:34.567062] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-09. [2025-11-04T18:42:34.729371] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T18:42:35.367040] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-04. [2025-11-04T18:42:35.586318] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T18:42:36.167044] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-02. [2025-11-04T18:42:36.447799] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T18:42:36.967014] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-03. [2025-11-04T18:42:37.016260] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T18:42:37.401276] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-11-04T18:42:37.767038] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-04. [2025-11-04T18:42:38.089345] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-04 (0.32 seconds). [2025-11-04T18:42:38.567039] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-10. [2025-11-04T18:42:39.367052] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-05. [2025-11-04T18:42:39.691600] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-11-04T18:42:40.167071] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-11. [2025-11-04T18:42:40.668731] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T18:42:40.967049] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-06. [2025-11-04T18:42:41.122849] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-11 (0.96 seconds). [2025-11-04T18:42:41.124520] [INFO ] [REMOTE ] ENIC: Full fetch complete (51109 records). [2025-11-04T18:42:41.124666] [DEBUG] [VALIDATOR ] ENIC: Validating primary data. [2025-11-04T18:42:41.505714] [DEBUG] [VALIDATOR ] ENIC: All expected trading days are present. [2025-11-04T18:42:41.505804] [DEBUG] [VALIDATOR ] ENIC: Primary data validation passed. [2025-11-04T18:42:41.506501] [INFO ] [MAIN ] ENIC: Fetched primary data is valid. [2025-11-04T18:42:41.506565] [DEBUG] [LOCAL ] ENIC: Saving primary data. [2025-11-04T18:42:41.507724] [DEBUG] [LOCAL ] ENIC: Trimmed 51109 records to 36063 (keeping 200 days of data) [2025-11-04T18:42:41.699197] [DEBUG] [LOCAL ] ENIC: Primary data saved (36063 records). [2025-11-04T18:42:41.699336] [INFO ] [MAIN ] ENIC: Full history saved. [2025-11-04T18:42:41.699418] [INFO ] [MAIN ] Update Progress: 740/2367 symbols completed. ENIC took 27.2s. Estimated remaining: 3h 45m 40s. Rate: 7.2 symbols/min [2025-11-04T18:42:41.772232] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-05. [2025-11-04T18:42:41.872881] [INFO ] [MAIN ] ENS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:42:41.899361] [DEBUG] [LOCAL ] ENS: Loaded 29584 existing records from primary storage. [2025-11-04T18:42:41.899615] [INFO ] [MAIN ] ENS: Loaded 29584 existing records. [2025-11-04T18:42:42.078424] [WARN ] [MAIN ] ENS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:42:42.078864] [INFO ] [MAIN ] ENS: Wiping existing primary data before full refresh. [2025-11-04T18:42:42.078972] [INFO ] [LOCAL ] ENS: Starting deletion of primary data. [2025-11-04T18:42:42.079250] [DEBUG] [LOCAL ] ENS: Deleted primary data file. [2025-11-04T18:42:42.079309] [INFO ] [MAIN ] ENS: Starting full primary refresh. [2025-11-04T18:42:42.162858] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-11-04T18:42:42.446137] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-11-04T18:42:42.482662] [INFO ] [REMOTE ] ENS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:42:42.567061] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-01. [2025-11-04T18:42:43.367021] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-03. [2025-11-04T18:42:43.445590] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T18:42:44.167055] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-06. [2025-11-04T18:42:44.278820] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T18:42:44.394358] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-06 (0.23 seconds). [2025-11-04T18:42:44.967047] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-04. [2025-11-04T18:42:45.767039] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-07. [2025-11-04T18:42:46.263171] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T18:42:46.567029] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-05. [2025-11-04T18:42:46.954375] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-11-04T18:42:47.367041] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-02. [2025-11-04T18:42:47.726060] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T18:42:48.167055] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-07. [2025-11-04T18:42:48.213459] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T18:42:48.556537] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-11-04T18:42:48.967058] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-08. [2025-11-04T18:42:49.767049] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-03. [2025-11-04T18:42:50.212479] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T18:42:50.567048] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-06. [2025-11-04T18:42:50.746231] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T18:42:51.367034] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-04. [2025-11-04T18:42:51.694061] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T18:42:52.167053] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-07. [2025-11-04T18:42:52.397017] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-11-04T18:42:52.967038] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-08. [2025-11-04T18:42:53.022220] [DEBUG] [REMOTE ] ENO: Error fetching primary data for 2025-08: 'API call returned an empty response.' [2025-11-04T18:42:53.022437] [WARN ] [REMOTE ] ENO: No data returned for month 2025-08 [2025-11-04T18:42:53.022520] [FATAL] [REMOTE ] ENO: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:42:53.022593] [FATAL] [MAIN ] ENO: Failed to fetch primary data. [2025-11-04T18:42:53.022683] [WARN ] [MAIN ] ENO: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:42:53.022755] [INFO ] [LOCAL ] ENO: Starting deletion of primary data. [2025-11-04T18:42:53.022815] [DEBUG] [LOCAL ] ENO: No primary data file found to delete. [2025-11-04T18:42:53.022849] [INFO ] [LOCAL ] ENO: Starting deletion of derivative data. [2025-11-04T18:42:53.022887] [DEBUG] [LOCAL ] ENO: No derivative data file found to delete. [2025-11-04T18:42:53.022931] [INFO ] [MAIN ] Update Progress: 741/2367 symbols completed. ENO took 34.7s. Estimated remaining: 3h 45m 38s. Rate: 7.2 symbols/min [2025-11-04T18:42:53.190446] [INFO ] [MAIN ] ENVA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:42:53.213867] [DEBUG] [LOCAL ] ENVA: Loaded 24880 existing records from primary storage. [2025-11-04T18:42:53.214114] [INFO ] [MAIN ] ENVA: Loaded 24880 existing records. [2025-11-04T18:42:53.395107] [WARN ] [MAIN ] ENVA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:42:53.395362] [INFO ] [MAIN ] ENVA: Wiping existing primary data before full refresh. [2025-11-04T18:42:53.395456] [INFO ] [LOCAL ] ENVA: Starting deletion of primary data. [2025-11-04T18:42:53.395870] [DEBUG] [LOCAL ] ENVA: Deleted primary data file. [2025-11-04T18:42:53.396059] [INFO ] [MAIN ] ENVA: Starting full primary refresh. [2025-11-04T18:42:53.684722] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-11-04T18:42:53.772363] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-09. [2025-11-04T18:42:53.792805] [INFO ] [REMOTE ] ENVA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:42:54.567046] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-01. [2025-11-04T18:42:55.036995] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T18:42:55.367049] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-08. [2025-11-04T18:42:55.641914] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-11-04T18:42:56.167048] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-02. [2025-11-04T18:42:56.664165] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T18:42:56.967052] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-10. [2025-11-04T18:42:57.059256] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T18:42:57.767072] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-09. [2025-11-04T18:42:58.127952] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T18:42:58.567010] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-03. [2025-11-04T18:42:58.778222] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T18:42:59.367011] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-10. [2025-11-04T18:42:59.406198] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-11-04T18:43:00.167034] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-04. [2025-11-04T18:43:00.681253] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T18:43:00.967066] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-11. [2025-11-04T18:43:01.179710] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-11-04T18:43:01.767086] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-05. [2025-11-04T18:43:01.907173] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T18:43:01.908487] [INFO ] [REMOTE ] ENR: Full fetch complete (59932 records). [2025-11-04T18:43:01.908568] [DEBUG] [VALIDATOR ] ENR: Validating primary data. [2025-11-04T18:43:01.909271] [FATAL] [VALIDATOR ] ENR: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:43:01.909353] [FATAL] [MAIN ] ENR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:43:01.909395] [WARN ] [MAIN ] ENR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:43:01.909431] [INFO ] [LOCAL ] ENR: Starting deletion of primary data. [2025-11-04T18:43:01.909483] [DEBUG] [LOCAL ] ENR: No primary data file found to delete. [2025-11-04T18:43:01.909516] [INFO ] [LOCAL ] ENR: Starting deletion of derivative data. [2025-11-04T18:43:01.909723] [DEBUG] [LOCAL ] ENR: Deleted derivative data file. [2025-11-04T18:43:01.909809] [INFO ] [MAIN ] Update Progress: 742/2367 symbols completed. ENR took 29.5s. Estimated remaining: 3h 45m 31s. Rate: 7.2 symbols/min [2025-11-04T18:43:02.074358] [INFO ] [MAIN ] EOD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:43:02.088288] [DEBUG] [LOCAL ] EOD: Loaded 14876 existing records from primary storage. [2025-11-04T18:43:02.088524] [INFO ] [MAIN ] EOD: Loaded 14876 existing records. [2025-11-04T18:43:02.270300] [WARN ] [MAIN ] EOD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:43:02.270576] [INFO ] [MAIN ] EOD: Wiping existing primary data before full refresh. [2025-11-04T18:43:02.270627] [INFO ] [LOCAL ] EOD: Starting deletion of primary data. [2025-11-04T18:43:02.270865] [DEBUG] [LOCAL ] EOD: Deleted primary data file. [2025-11-04T18:43:02.270920] [INFO ] [MAIN ] EOD: Starting full primary refresh. [2025-11-04T18:43:02.569757] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-11. [2025-11-04T18:43:02.704720] [INFO ] [REMOTE ] EOD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:43:02.729834] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T18:43:03.367049] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-06. [2025-11-04T18:43:03.485917] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T18:43:03.487803] [INFO ] [REMOTE ] ENOV: Full fetch complete (63285 records). [2025-11-04T18:43:03.487906] [DEBUG] [VALIDATOR ] ENOV: Validating primary data. [2025-11-04T18:43:03.875938] [DEBUG] [VALIDATOR ] ENOV: All expected trading days are present. [2025-11-04T18:43:03.876033] [DEBUG] [VALIDATOR ] ENOV: Primary data validation passed. [2025-11-04T18:43:03.876862] [INFO ] [MAIN ] ENOV: Fetched primary data is valid. [2025-11-04T18:43:03.876925] [DEBUG] [LOCAL ] ENOV: Saving primary data. [2025-11-04T18:43:03.878206] [DEBUG] [LOCAL ] ENOV: Trimmed 63285 records to 44510 (keeping 200 days of data) [2025-11-04T18:43:04.143802] [DEBUG] [LOCAL ] ENOV: Primary data saved (44510 records). [2025-11-04T18:43:04.143933] [INFO ] [MAIN ] ENOV: Full history saved. [2025-11-04T18:43:04.144011] [INFO ] [MAIN ] Update Progress: 743/2367 symbols completed. ENOV took 38.7s. Estimated remaining: 3h 45m 10s. Rate: 7.2 symbols/min [2025-11-04T18:43:04.178719] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-01. [2025-11-04T18:43:04.354716] [INFO ] [MAIN ] EOG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:43:04.406380] [DEBUG] [LOCAL ] EOG: Loaded 53388 existing records from primary storage. [2025-11-04T18:43:04.407165] [INFO ] [MAIN ] EOG: Loaded 53388 existing records. [2025-11-04T18:43:04.604176] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T18:43:04.614733] [WARN ] [MAIN ] EOG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:43:04.621925] [INFO ] [MAIN ] EOG: Wiping existing primary data before full refresh. [2025-11-04T18:43:04.622312] [INFO ] [LOCAL ] EOG: Starting deletion of primary data. [2025-11-04T18:43:04.622733] [DEBUG] [LOCAL ] EOG: Deleted primary data file. [2025-11-04T18:43:04.622790] [INFO ] [MAIN ] EOG: Starting full primary refresh. [2025-11-04T18:43:04.971694] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-05. [2025-11-04T18:43:04.976224] [INFO ] [REMOTE ] EOG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:43:05.133114] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T18:43:05.767107] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-01. [2025-11-04T18:43:05.975885] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-11-04T18:43:06.567061] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-06. [2025-11-04T18:43:07.146290] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T18:43:07.278073] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T18:43:07.367122] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-07. [2025-11-04T18:43:08.167056] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-02. [2025-11-04T18:43:08.188717] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T18:43:08.722743] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-11-04T18:43:08.967051] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-03. [2025-11-04T18:43:09.562451] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-11-04T18:43:09.767074] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-07. [2025-11-04T18:43:10.567050] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-04. [2025-11-04T18:43:10.829016] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-11-04T18:43:11.130716] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-11-04T18:43:11.367072] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-02. [2025-11-04T18:43:12.167084] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-08. [2025-11-04T18:43:12.802223] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-11-04T18:43:12.876094] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T18:43:12.967009] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-09. [2025-11-04T18:43:13.767020] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-03. [2025-11-04T18:43:13.793410] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-11-04T18:43:14.567041] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-08. [2025-11-04T18:43:15.154358] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T18:43:15.367033] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-10. [2025-11-04T18:43:15.480003] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-11-04T18:43:16.167065] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-09. [2025-11-04T18:43:16.247827] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-11-04T18:43:16.967091] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-05. [2025-11-04T18:43:17.123711] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T18:43:17.574355] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-11-04T18:43:17.767043] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-06. [2025-11-04T18:43:18.426925] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-11-04T18:43:18.567046] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-11. [2025-11-04T18:43:19.258155] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-11 (0.69 seconds). [2025-11-04T18:43:19.259740] [INFO ] [REMOTE ] ENVA: Full fetch complete (36828 records). [2025-11-04T18:43:19.259840] [DEBUG] [VALIDATOR ] ENVA: Validating primary data. [2025-11-04T18:43:19.372300] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-10. [2025-11-04T18:43:19.693060] [DEBUG] [VALIDATOR ] ENVA: All expected trading days are present. [2025-11-04T18:43:19.693191] [DEBUG] [VALIDATOR ] ENVA: Primary data validation passed. [2025-11-04T18:43:19.694106] [INFO ] [MAIN ] ENVA: Fetched primary data is valid. [2025-11-04T18:43:19.694189] [DEBUG] [LOCAL ] ENVA: Saving primary data. [2025-11-04T18:43:19.695817] [DEBUG] [LOCAL ] ENVA: Trimmed 36828 records to 24893 (keeping 200 days of data) [2025-11-04T18:43:19.849480] [DEBUG] [LOCAL ] ENVA: Primary data saved (24893 records). [2025-11-04T18:43:19.849610] [INFO ] [MAIN ] ENVA: Full history saved. [2025-11-04T18:43:19.849718] [INFO ] [MAIN ] Update Progress: 744/2367 symbols completed. ENVA took 26.8s. Estimated remaining: 3h 45m 17s. Rate: 7.2 symbols/min [2025-11-04T18:43:20.018484] [INFO ] [MAIN ] EOI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:43:20.029235] [DEBUG] [LOCAL ] EOI: Loaded 9889 existing records from primary storage. [2025-11-04T18:43:20.029405] [INFO ] [MAIN ] EOI: Loaded 9889 existing records. [2025-11-04T18:43:20.173794] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-04. [2025-11-04T18:43:20.202301] [WARN ] [MAIN ] EOI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:43:20.202739] [INFO ] [MAIN ] EOI: Wiping existing primary data before full refresh. [2025-11-04T18:43:20.202819] [INFO ] [LOCAL ] EOI: Starting deletion of primary data. [2025-11-04T18:43:20.203024] [DEBUG] [LOCAL ] EOI: Deleted primary data file. [2025-11-04T18:43:20.203081] [INFO ] [MAIN ] EOI: Starting full primary refresh. [2025-11-04T18:43:20.553831] [INFO ] [REMOTE ] EOI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:43:20.685433] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T18:43:20.967060] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-11. [2025-11-04T18:43:21.547445] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-11-04T18:43:21.767092] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-01. [2025-11-04T18:43:22.098634] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T18:43:22.100031] [INFO ] [REMOTE ] ENS: Full fetch complete (43047 records). [2025-11-04T18:43:22.100113] [DEBUG] [VALIDATOR ] ENS: Validating primary data. [2025-11-04T18:43:22.470458] [DEBUG] [VALIDATOR ] ENS: All expected trading days are present. [2025-11-04T18:43:22.470588] [DEBUG] [VALIDATOR ] ENS: Primary data validation passed. [2025-11-04T18:43:22.471191] [INFO ] [MAIN ] ENS: Fetched primary data is valid. [2025-11-04T18:43:22.471266] [DEBUG] [LOCAL ] ENS: Saving primary data. [2025-11-04T18:43:22.479618] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T18:43:22.483150] [DEBUG] [LOCAL ] ENS: Trimmed 43047 records to 29629 (keeping 200 days of data) [2025-11-04T18:43:22.574399] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-02. [2025-11-04T18:43:22.667332] [DEBUG] [LOCAL ] ENS: Primary data saved (29629 records). [2025-11-04T18:43:22.667488] [INFO ] [MAIN ] ENS: Full history saved. [2025-11-04T18:43:22.667578] [INFO ] [MAIN ] Update Progress: 745/2367 symbols completed. ENS took 41.0s. Estimated remaining: 3h 44m 57s. Rate: 7.2 symbols/min [2025-11-04T18:43:22.831402] [INFO ] [MAIN ] EOS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:43:22.846611] [DEBUG] [LOCAL ] EOS: Loaded 14388 existing records from primary storage. [2025-11-04T18:43:22.846898] [INFO ] [MAIN ] EOS: Loaded 14388 existing records. [2025-11-04T18:43:23.025845] [WARN ] [MAIN ] EOS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:43:23.033219] [INFO ] [MAIN ] EOS: Wiping existing primary data before full refresh. [2025-11-04T18:43:23.033311] [INFO ] [LOCAL ] EOS: Starting deletion of primary data. [2025-11-04T18:43:23.033931] [DEBUG] [LOCAL ] EOS: Deleted primary data file. [2025-11-04T18:43:23.034326] [INFO ] [MAIN ] EOS: Starting full primary refresh. [2025-11-04T18:43:23.034266] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-11-04T18:43:23.368291] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-03. [2025-11-04T18:43:23.391178] [INFO ] [REMOTE ] EOS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:43:23.933325] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T18:43:24.167048] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-07. [2025-11-04T18:43:24.967104] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-01. [2025-11-04T18:43:24.982603] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T18:43:25.626863] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-11-04T18:43:25.767026] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-02. [2025-11-04T18:43:26.432882] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-11-04T18:43:26.567077] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-05. [2025-11-04T18:43:27.367055] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-08. [2025-11-04T18:43:28.040484] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T18:43:28.106086] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T18:43:28.167045] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-04. [2025-11-04T18:43:28.708782] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-11-04T18:43:28.967074] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-03. [2025-11-04T18:43:29.679407] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T18:43:29.767062] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-04. [2025-11-04T18:43:30.501171] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T18:43:30.567062] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-05. [2025-11-04T18:43:31.152102] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T18:43:31.367064] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-06. [2025-11-04T18:43:32.167056] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-09. [2025-11-04T18:43:32.688491] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T18:43:32.711726] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T18:43:32.967073] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-07. [2025-11-04T18:43:33.767053] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-05. [2025-11-04T18:43:34.252570] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-11-04T18:43:34.374565] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-11-04T18:43:34.567086] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-10. [2025-11-04T18:43:35.208687] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-11-04T18:43:35.367077] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-06. [2025-11-04T18:43:36.137851] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T18:43:36.167087] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-08. [2025-11-04T18:43:36.967128] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-07. [2025-11-04T18:43:37.487706] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-11-04T18:43:37.607429] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-11-04T18:43:37.767065] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-09. [2025-11-04T18:43:38.567074] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-06. [2025-11-04T18:43:39.018670] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-11-04T18:43:39.224928] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-11-04T18:43:39.367048] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-10. [2025-11-04T18:43:40.167035] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-07. [2025-11-04T18:43:40.658085] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-11-04T18:43:40.873753] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T18:43:40.967034] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-08. [2025-11-04T18:43:41.463751] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-11-04T18:43:41.767053] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-08. [2025-11-04T18:43:42.300667] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-11-04T18:43:42.567076] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-09. [2025-11-04T18:43:43.118501] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-11-04T18:43:43.367050] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-11. [2025-11-04T18:43:44.045239] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T18:43:44.046409] [INFO ] [REMOTE ] EOD: Full fetch complete (21002 records). [2025-11-04T18:43:44.046492] [DEBUG] [VALIDATOR ] EOD: Validating primary data. [2025-11-04T18:43:44.172156] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-11. [2025-11-04T18:43:44.422809] [DEBUG] [VALIDATOR ] EOD: All expected trading days are present. [2025-11-04T18:43:44.422896] [DEBUG] [VALIDATOR ] EOD: Primary data validation passed. [2025-11-04T18:43:44.423310] [INFO ] [MAIN ] EOD: Fetched primary data is valid. [2025-11-04T18:43:44.423409] [DEBUG] [LOCAL ] EOD: Saving primary data. [2025-11-04T18:43:44.424172] [DEBUG] [LOCAL ] EOD: Trimmed 21002 records to 14909 (keeping 200 days of data) [2025-11-04T18:43:44.505602] [DEBUG] [LOCAL ] EOD: Primary data saved (14909 records). [2025-11-04T18:43:44.505766] [INFO ] [MAIN ] EOD: Full history saved. [2025-11-04T18:43:44.505850] [INFO ] [MAIN ] Update Progress: 746/2367 symbols completed. EOD took 42.6s. Estimated remaining: 3h 45m 18s. Rate: 7.2 symbols/min [2025-11-04T18:43:44.640559] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T18:43:44.669551] [INFO ] [REMOTE ] EOG: Full fetch complete (77470 records). [2025-11-04T18:43:44.676171] [DEBUG] [VALIDATOR ] EOG: Validating primary data. [2025-11-04T18:43:44.695765] [INFO ] [MAIN ] EOT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:43:44.869629] [DEBUG] [LOCAL ] EOT: Loaded 6936 existing records from primary storage. [2025-11-04T18:43:44.890778] [INFO ] [MAIN ] EOT: Loaded 6936 existing records. [2025-11-04T18:43:44.973815] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-09. [2025-11-04T18:43:45.301834] [DEBUG] [VALIDATOR ] EOG: All expected trading days are present. [2025-11-04T18:43:45.306113] [DEBUG] [VALIDATOR ] EOG: Primary data validation passed. [2025-11-04T18:43:45.326262] [INFO ] [MAIN ] EOG: Fetched primary data is valid. [2025-11-04T18:43:45.326346] [DEBUG] [LOCAL ] EOG: Saving primary data. [2025-11-04T18:43:45.331764] [WARN ] [MAIN ] EOT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:43:45.332712] [INFO ] [MAIN ] EOT: Wiping existing primary data before full refresh. [2025-11-04T18:43:45.332921] [INFO ] [LOCAL ] EOT: Starting deletion of primary data. [2025-11-04T18:43:45.333320] [DEBUG] [LOCAL ] EOT: Deleted primary data file. [2025-11-04T18:43:45.333407] [INFO ] [MAIN ] EOT: Starting full primary refresh. [2025-11-04T18:43:45.338737] [DEBUG] [LOCAL ] EOG: Trimmed 77470 records to 53388 (keeping 200 days of data) [2025-11-04T18:43:45.792631] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-10. [2025-11-04T18:43:45.947474] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-11-04T18:43:46.091235] [DEBUG] [LOCAL ] EOG: Primary data saved (53388 records). [2025-11-04T18:43:46.096003] [INFO ] [MAIN ] EOG: Full history saved. [2025-11-04T18:43:46.097011] [INFO ] [MAIN ] Update Progress: 747/2367 symbols completed. EOG took 42.0s. Estimated remaining: 3h 44m 55s. Rate: 7.2 symbols/min [2025-11-04T18:43:46.095773] [INFO ] [REMOTE ] EOT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:43:46.292889] [INFO ] [MAIN ] EPAC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:43:46.319705] [DEBUG] [LOCAL ] EPAC: Loaded 26093 existing records from primary storage. [2025-11-04T18:43:46.319988] [INFO ] [MAIN ] EPAC: Loaded 26093 existing records. [2025-11-04T18:43:46.495620] [WARN ] [MAIN ] EPAC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:43:46.495853] [INFO ] [MAIN ] EPAC: Wiping existing primary data before full refresh. [2025-11-04T18:43:46.495901] [INFO ] [LOCAL ] EPAC: Starting deletion of primary data. [2025-11-04T18:43:46.496093] [DEBUG] [LOCAL ] EPAC: Deleted primary data file. [2025-11-04T18:43:46.496131] [INFO ] [MAIN ] EPAC: Starting full primary refresh. [2025-11-04T18:43:46.572224] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-01. [2025-11-04T18:43:46.861869] [INFO ] [REMOTE ] EPAC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:43:46.875333] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-11-04T18:43:47.046804] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-01 (0.47 seconds). [2025-11-04T18:43:47.367086] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-02. [2025-11-04T18:43:47.937441] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-11-04T18:43:48.167073] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-03. [2025-11-04T18:43:48.601096] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-11-04T18:43:48.967073] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-04. [2025-11-04T18:43:49.529241] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-11-04T18:43:49.767053] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-05. [2025-11-04T18:43:50.203681] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-11-04T18:43:50.567043] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-10. [2025-11-04T18:43:51.125194] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-11-04T18:43:51.367038] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-01. [2025-11-04T18:43:52.167045] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-11. [2025-11-04T18:43:52.424768] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T18:43:52.796915] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T18:43:52.798329] [INFO ] [REMOTE ] EOS: Full fetch complete (22982 records). [2025-11-04T18:43:52.798420] [DEBUG] [VALIDATOR ] EOS: Validating primary data. [2025-11-04T18:43:52.975771] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-11. [2025-11-04T18:43:53.167795] [DEBUG] [VALIDATOR ] EOS: All expected trading days are present. [2025-11-04T18:43:53.167877] [DEBUG] [VALIDATOR ] EOS: Primary data validation passed. [2025-11-04T18:43:53.168418] [INFO ] [MAIN ] EOS: Fetched primary data is valid. [2025-11-04T18:43:53.168459] [DEBUG] [LOCAL ] EOS: Saving primary data. [2025-11-04T18:43:53.169089] [DEBUG] [LOCAL ] EOS: Trimmed 22982 records to 14371 (keeping 200 days of data) [2025-11-04T18:43:53.255382] [DEBUG] [LOCAL ] EOS: Primary data saved (14371 records). [2025-11-04T18:43:53.255514] [INFO ] [MAIN ] EOS: Full history saved. [2025-11-04T18:43:53.255594] [INFO ] [MAIN ] Update Progress: 748/2367 symbols completed. EOS took 30.6s. Estimated remaining: 3h 44m 44s. Rate: 7.2 symbols/min [2025-11-04T18:43:53.432590] [INFO ] [MAIN ] EPAM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:43:53.468836] [DEBUG] [LOCAL ] EPAM: Loaded 38876 existing records from primary storage. [2025-11-04T18:43:53.469138] [INFO ] [MAIN ] EPAM: Loaded 38876 existing records. [2025-11-04T18:43:53.603794] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T18:43:53.635146] [INFO ] [REMOTE ] EOI: Full fetch complete (15829 records). [2025-11-04T18:43:53.644845] [DEBUG] [VALIDATOR ] EOI: Validating primary data. [2025-11-04T18:43:53.655342] [WARN ] [MAIN ] EPAM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:43:53.723302] [INFO ] [MAIN ] EPAM: Wiping existing primary data before full refresh. [2025-11-04T18:43:53.740698] [INFO ] [LOCAL ] EPAM: Starting deletion of primary data. [2025-11-04T18:43:53.763116] [DEBUG] [LOCAL ] EPAM: Deleted primary data file. [2025-11-04T18:43:53.763269] [INFO ] [MAIN ] EPAM: Starting full primary refresh. [2025-11-04T18:43:53.783887] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-02. [2025-11-04T18:43:54.315143] [DEBUG] [VALIDATOR ] EOI: All expected trading days are present. [2025-11-04T18:43:54.320381] [DEBUG] [VALIDATOR ] EOI: Primary data validation passed. [2025-11-04T18:43:54.339503] [INFO ] [MAIN ] EOI: Fetched primary data is valid. [2025-11-04T18:43:54.339581] [DEBUG] [LOCAL ] EOI: Saving primary data. [2025-11-04T18:43:54.364281] [DEBUG] [LOCAL ] EOI: Trimmed 15829 records to 9864 (keeping 200 days of data) [2025-11-04T18:43:54.449755] [INFO ] [REMOTE ] EPAM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:43:54.488807] [DEBUG] [LOCAL ] EOI: Primary data saved (9864 records). [2025-11-04T18:43:54.488942] [INFO ] [MAIN ] EOI: Full history saved. [2025-11-04T18:43:54.489026] [INFO ] [MAIN ] Update Progress: 749/2367 symbols completed. EOI took 34.6s. Estimated remaining: 3h 44m 21s. Rate: 7.2 symbols/min [2025-11-04T18:43:54.575614] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-01. [2025-11-04T18:43:54.665867] [INFO ] [MAIN ] EPC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:43:54.682478] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T18:43:54.721421] [DEBUG] [LOCAL ] EPC: Loaded 39708 existing records from primary storage. [2025-11-04T18:43:54.721763] [INFO ] [MAIN ] EPC: Loaded 39708 existing records. [2025-11-04T18:43:54.902530] [WARN ] [MAIN ] EPC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:43:54.902775] [INFO ] [MAIN ] EPC: Wiping existing primary data before full refresh. [2025-11-04T18:43:54.902833] [INFO ] [LOCAL ] EPC: Starting deletion of primary data. [2025-11-04T18:43:54.903036] [DEBUG] [LOCAL ] EPC: Deleted primary data file. [2025-11-04T18:43:54.903091] [INFO ] [MAIN ] EPC: Starting full primary refresh. [2025-11-04T18:43:55.253163] [INFO ] [REMOTE ] EPC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:43:55.367011] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-01. [2025-11-04T18:43:55.709618] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T18:43:56.167067] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-03. [2025-11-04T18:43:56.389007] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T18:43:56.967020] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-06. [2025-11-04T18:43:57.038372] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-11-04T18:43:57.513923] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-11-04T18:43:57.767052] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-04. [2025-11-04T18:43:58.567048] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-02. [2025-11-04T18:43:58.599447] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-11-04T18:43:59.367048] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-07. [2025-11-04T18:43:59.553687] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T18:43:59.844847] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-11-04T18:44:00.167055] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-02. [2025-11-04T18:44:00.967064] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-08. [2025-11-04T18:44:01.639330] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-11-04T18:44:01.701573] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-11-04T18:44:01.767088] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-09. [2025-11-04T18:44:02.328804] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T18:44:02.567108] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-03. [2025-11-04T18:44:03.367039] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-05. [2025-11-04T18:44:03.586474] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-11-04T18:44:04.167002] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-03. [2025-11-04T18:44:04.248186] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T18:44:04.967078] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-04. [2025-11-04T18:44:05.249113] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T18:44:05.767019] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-10. [2025-11-04T18:44:05.840161] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-11-04T18:44:06.225562] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-11-04T18:44:06.567052] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-05. [2025-11-04T18:44:07.367038] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-06. [2025-11-04T18:44:07.733935] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T18:44:08.135847] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T18:44:08.167054] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-11. [2025-11-04T18:44:08.828430] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T18:44:08.829240] [INFO ] [REMOTE ] EOT: Full fetch complete (10676 records). [2025-11-04T18:44:08.829307] [DEBUG] [VALIDATOR ] EOT: Validating primary data. [2025-11-04T18:44:08.976008] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-07. [2025-11-04T18:44:09.194834] [DEBUG] [VALIDATOR ] EOT: All expected trading days are present. [2025-11-04T18:44:09.194912] [DEBUG] [VALIDATOR ] EOT: Primary data validation passed. [2025-11-04T18:44:09.195199] [INFO ] [MAIN ] EOT: Fetched primary data is valid. [2025-11-04T18:44:09.195239] [DEBUG] [LOCAL ] EOT: Saving primary data. [2025-11-04T18:44:09.195693] [DEBUG] [LOCAL ] EOT: Trimmed 10676 records to 6955 (keeping 200 days of data) [2025-11-04T18:44:09.237344] [DEBUG] [LOCAL ] EOT: Primary data saved (6955 records). [2025-11-04T18:44:09.237483] [INFO ] [MAIN ] EOT: Full history saved. [2025-11-04T18:44:09.237576] [INFO ] [MAIN ] Update Progress: 750/2367 symbols completed. EOT took 24.7s. Estimated remaining: 3h 44m 26s. Rate: 7.2 symbols/min [2025-11-04T18:44:09.409039] [INFO ] [MAIN ] EPD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:44:09.456793] [DEBUG] [LOCAL ] EPD: Loaded 53606 existing records from primary storage. [2025-11-04T18:44:09.457124] [INFO ] [MAIN ] EPD: Loaded 53606 existing records. [2025-11-04T18:44:09.653419] [WARN ] [MAIN ] EPD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:44:09.653687] [INFO ] [MAIN ] EPD: Wiping existing primary data before full refresh. [2025-11-04T18:44:09.653746] [INFO ] [LOCAL ] EPD: Starting deletion of primary data. [2025-11-04T18:44:09.653932] [DEBUG] [LOCAL ] EPD: Deleted primary data file. [2025-11-04T18:44:09.653971] [INFO ] [MAIN ] EPD: Starting full primary refresh. [2025-11-04T18:44:09.768260] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-06. [2025-11-04T18:44:10.081451] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T18:44:10.087745] [INFO ] [REMOTE ] EPD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:44:10.567064] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-01. [2025-11-04T18:44:10.935912] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T18:44:11.367043] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-07. [2025-11-04T18:44:12.032830] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T18:44:12.167077] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-02. [2025-11-04T18:44:12.470345] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T18:44:12.967084] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-08. [2025-11-04T18:44:13.379524] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T18:44:13.767037] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-03. [2025-11-04T18:44:14.110522] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T18:44:14.567055] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-04. [2025-11-04T18:44:15.317945] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T18:44:15.367057] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-04. [2025-11-04T18:44:15.945186] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T18:44:16.167060] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-09. [2025-11-04T18:44:16.881245] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T18:44:16.967063] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-08. [2025-11-04T18:44:17.272824] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T18:44:17.767066] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-10. [2025-11-04T18:44:17.802285] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-11-04T18:44:18.567023] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-09. [2025-11-04T18:44:19.082066] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T18:44:19.366958] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-11-04T18:44:19.367084] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-05. [2025-11-04T18:44:20.167015] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-10. [2025-11-04T18:44:20.546243] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-11-04T18:44:20.967017] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-05. [2025-11-04T18:44:21.029549] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-11-04T18:44:21.767053] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-06. [2025-11-04T18:44:22.431836] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T18:44:22.567010] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-06. [2025-11-04T18:44:22.774923] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-11-04T18:44:23.367100] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-11. [2025-11-04T18:44:23.937540] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-11-04T18:44:24.152275] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-11 (0.79 seconds). [2025-11-04T18:44:24.153631] [INFO ] [REMOTE ] EPAC: Full fetch complete (38839 records). [2025-11-04T18:44:24.153762] [DEBUG] [VALIDATOR ] EPAC: Validating primary data. [2025-11-04T18:44:24.174749] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-07. [2025-11-04T18:44:24.522345] [DEBUG] [VALIDATOR ] EPAC: All expected trading days are present. [2025-11-04T18:44:24.522419] [DEBUG] [VALIDATOR ] EPAC: Primary data validation passed. [2025-11-04T18:44:24.523004] [INFO ] [MAIN ] EPAC: Fetched primary data is valid. [2025-11-04T18:44:24.523065] [DEBUG] [LOCAL ] EPAC: Saving primary data. [2025-11-04T18:44:24.523917] [DEBUG] [LOCAL ] EPAC: Trimmed 38839 records to 26092 (keeping 200 days of data) [2025-11-04T18:44:24.680485] [DEBUG] [LOCAL ] EPAC: Primary data saved (26092 records). [2025-11-04T18:44:24.680694] [INFO ] [MAIN ] EPAC: Full history saved. [2025-11-04T18:44:24.680827] [INFO ] [MAIN ] Update Progress: 751/2367 symbols completed. EPAC took 38.6s. Estimated remaining: 3h 44m 33s. Rate: 7.2 symbols/min [2025-11-04T18:44:24.845298] [INFO ] [MAIN ] EPR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:44:24.883767] [DEBUG] [LOCAL ] EPR: Loaded 43642 existing records from primary storage. [2025-11-04T18:44:24.884065] [INFO ] [MAIN ] EPR: Loaded 43642 existing records. [2025-11-04T18:44:24.972978] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-07. [2025-11-04T18:44:25.077258] [WARN ] [MAIN ] EPR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:44:25.077491] [INFO ] [MAIN ] EPR: Wiping existing primary data before full refresh. [2025-11-04T18:44:25.077539] [INFO ] [LOCAL ] EPR: Starting deletion of primary data. [2025-11-04T18:44:25.077758] [DEBUG] [LOCAL ] EPR: Deleted primary data file. [2025-11-04T18:44:25.077814] [INFO ] [MAIN ] EPR: Starting full primary refresh. [2025-11-04T18:44:25.423135] [INFO ] [REMOTE ] EPR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:44:25.764204] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-11-04T18:44:25.774809] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-01. [2025-11-04T18:44:26.076086] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T18:44:26.567074] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-08. [2025-11-04T18:44:26.962558] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T18:44:27.367090] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-08. [2025-11-04T18:44:27.831794] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T18:44:28.167034] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-02. [2025-11-04T18:44:28.348048] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-11-04T18:44:28.967062] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-09. [2025-11-04T18:44:29.343222] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T18:44:29.767037] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-11. [2025-11-04T18:44:30.148180] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:44:30.567045] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-03. [2025-11-04T18:44:30.969128] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T18:44:30.970993] [INFO ] [REMOTE ] EPC: Full fetch complete (57515 records). [2025-11-04T18:44:30.971085] [DEBUG] [VALIDATOR ] EPC: Validating primary data. [2025-11-04T18:44:31.349957] [DEBUG] [VALIDATOR ] EPC: All expected trading days are present. [2025-11-04T18:44:31.350106] [DEBUG] [VALIDATOR ] EPC: Primary data validation passed. [2025-11-04T18:44:31.350845] [INFO ] [MAIN ] EPC: Fetched primary data is valid. [2025-11-04T18:44:31.350910] [DEBUG] [LOCAL ] EPC: Saving primary data. [2025-11-04T18:44:31.352048] [DEBUG] [LOCAL ] EPC: Trimmed 57515 records to 39747 (keeping 200 days of data) [2025-11-04T18:44:31.404967] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-10. [2025-11-04T18:44:31.589363] [DEBUG] [LOCAL ] EPC: Primary data saved (39747 records). [2025-11-04T18:44:31.589497] [INFO ] [MAIN ] EPC: Full history saved. [2025-11-04T18:44:31.589581] [INFO ] [MAIN ] Update Progress: 752/2367 symbols completed. EPC took 37.1s. Estimated remaining: 3h 44m 22s. Rate: 7.2 symbols/min [2025-11-04T18:44:31.756804] [INFO ] [MAIN ] EPRT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:44:31.757004] [DEBUG] [LOCAL ] EPRT: No existing data file found in primary storage. [2025-11-04T18:44:31.757142] [WARN ] [MAIN ] EPRT: No existing data found, performing full refresh instead. [2025-11-04T18:44:31.757276] [INFO ] [MAIN ] EPRT: Wiping existing primary data before full refresh. [2025-11-04T18:44:31.757396] [INFO ] [LOCAL ] EPRT: Starting deletion of primary data. [2025-11-04T18:44:31.757503] [DEBUG] [LOCAL ] EPRT: No primary data file found to delete. [2025-11-04T18:44:31.757608] [INFO ] [MAIN ] EPRT: Starting full primary refresh. [2025-11-04T18:44:31.969780] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T18:44:32.148902] [INFO ] [REMOTE ] EPRT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:44:32.167029] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-01. [2025-11-04T18:44:32.597960] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-11-04T18:44:32.967055] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-04. [2025-11-04T18:44:33.355028] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T18:44:33.767047] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-02. [2025-11-04T18:44:34.145151] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T18:44:34.567023] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-11. [2025-11-04T18:44:34.636429] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T18:44:35.367075] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-03. [2025-11-04T18:44:35.599116] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T18:44:35.600814] [INFO ] [REMOTE ] EPAM: Full fetch complete (57226 records). [2025-11-04T18:44:35.600889] [DEBUG] [VALIDATOR ] EPAM: Validating primary data. [2025-11-04T18:44:35.978677] [DEBUG] [VALIDATOR ] EPAM: All expected trading days are present. [2025-11-04T18:44:35.978762] [DEBUG] [VALIDATOR ] EPAM: Primary data validation passed. [2025-11-04T18:44:35.979659] [INFO ] [MAIN ] EPAM: Fetched primary data is valid. [2025-11-04T18:44:35.979720] [DEBUG] [LOCAL ] EPAM: Saving primary data. [2025-11-04T18:44:35.981165] [DEBUG] [LOCAL ] EPAM: Trimmed 57226 records to 38932 (keeping 200 days of data) [2025-11-04T18:44:36.179046] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-09. [2025-11-04T18:44:36.212553] [DEBUG] [LOCAL ] EPAM: Primary data saved (38932 records). [2025-11-04T18:44:36.212678] [INFO ] [MAIN ] EPAM: Full history saved. [2025-11-04T18:44:36.212774] [INFO ] [MAIN ] Update Progress: 753/2367 symbols completed. EPAM took 43.0s. Estimated remaining: 3h 44m 6s. Rate: 7.2 symbols/min [2025-11-04T18:44:36.381885] [INFO ] [MAIN ] EQBK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:44:36.393813] [DEBUG] [LOCAL ] EQBK: Loaded 11022 existing records from primary storage. [2025-11-04T18:44:36.394142] [INFO ] [MAIN ] EQBK: Loaded 11022 existing records. [2025-11-04T18:44:36.575892] [WARN ] [MAIN ] EQBK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:44:36.576189] [INFO ] [MAIN ] EQBK: Wiping existing primary data before full refresh. [2025-11-04T18:44:36.576353] [INFO ] [LOCAL ] EQBK: Starting deletion of primary data. [2025-11-04T18:44:36.585785] [DEBUG] [LOCAL ] EQBK: Deleted primary data file. [2025-11-04T18:44:36.601171] [INFO ] [MAIN ] EQBK: Starting full primary refresh. [2025-11-04T18:44:36.670002] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T18:44:36.969671] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-04. [2025-11-04T18:44:36.969594] [INFO ] [REMOTE ] EQBK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:44:37.623833] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T18:44:37.767036] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-10. [2025-11-04T18:44:37.990711] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T18:44:38.567089] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-05. [2025-11-04T18:44:39.372133] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-01. [2025-11-04T18:44:39.392797] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-11-04T18:44:39.612358] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T18:44:40.167026] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-06. [2025-11-04T18:44:40.249074] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T18:44:40.967103] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-11. [2025-11-04T18:44:41.212391] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T18:44:41.767074] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-02. [2025-11-04T18:44:42.213829] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-11-04T18:44:42.229096] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-11 (1.26 seconds). [2025-11-04T18:44:42.231358] [INFO ] [REMOTE ] EPD: Full fetch complete (77775 records). [2025-11-04T18:44:42.231539] [DEBUG] [VALIDATOR ] EPD: Validating primary data. [2025-11-04T18:44:42.572311] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-03. [2025-11-04T18:44:42.786138] [DEBUG] [VALIDATOR ] EPD: All expected trading days are present. [2025-11-04T18:44:42.786232] [DEBUG] [VALIDATOR ] EPD: Primary data validation passed. [2025-11-04T18:44:42.787279] [INFO ] [MAIN ] EPD: Fetched primary data is valid. [2025-11-04T18:44:42.787362] [DEBUG] [LOCAL ] EPD: Saving primary data. [2025-11-04T18:44:42.788941] [DEBUG] [LOCAL ] EPD: Trimmed 77775 records to 53606 (keeping 200 days of data) [2025-11-04T18:44:43.114879] [DEBUG] [LOCAL ] EPD: Primary data saved (53606 records). [2025-11-04T18:44:43.115007] [INFO ] [MAIN ] EPD: Full history saved. [2025-11-04T18:44:43.115083] [INFO ] [MAIN ] Update Progress: 754/2367 symbols completed. EPD took 33.9s. Estimated remaining: 3h 43m 54s. Rate: 7.2 symbols/min [2025-11-04T18:44:43.286974] [INFO ] [MAIN ] EQH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:44:43.293878] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-11-04T18:44:43.344331] [DEBUG] [LOCAL ] EQH: Loaded 52648 existing records from primary storage. [2025-11-04T18:44:43.344688] [INFO ] [MAIN ] EQH: Loaded 52648 existing records. [2025-11-04T18:44:43.373452] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-07. [2025-11-04T18:44:43.546544] [WARN ] [MAIN ] EQH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:44:43.546868] [INFO ] [MAIN ] EQH: Wiping existing primary data before full refresh. [2025-11-04T18:44:43.546961] [INFO ] [LOCAL ] EQH: Starting deletion of primary data. [2025-11-04T18:44:43.547190] [DEBUG] [LOCAL ] EQH: Deleted primary data file. [2025-11-04T18:44:43.547247] [INFO ] [MAIN ] EQH: Starting full primary refresh. [2025-11-04T18:44:43.904599] [INFO ] [REMOTE ] EQH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:44:44.167047] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-01. [2025-11-04T18:44:44.633763] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T18:44:44.967052] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-08. [2025-11-04T18:44:45.503120] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T18:44:45.767027] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-02. [2025-11-04T18:44:46.222221] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T18:44:46.567042] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-04. [2025-11-04T18:44:46.894467] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T18:44:47.229216] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-11-04T18:44:47.367017] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-03. [2025-11-04T18:44:48.167026] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-09. [2025-11-04T18:44:48.581268] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-11-04T18:44:48.967066] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-05. [2025-11-04T18:44:49.356621] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T18:44:49.517013] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T18:44:49.767039] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-06. [2025-11-04T18:44:50.331186] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-11-04T18:44:50.567038] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-04. [2025-11-04T18:44:51.367042] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-07. [2025-11-04T18:44:51.908725] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T18:44:52.033129] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T18:44:52.167031] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-08. [2025-11-04T18:44:52.688248] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-11-04T18:44:52.967036] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-09. [2025-11-04T18:44:53.517944] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-11-04T18:44:53.767050] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-05. [2025-11-04T18:44:54.567078] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-05. [2025-11-04T18:44:55.085681] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T18:44:55.367062] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-10. [2025-11-04T18:44:55.908749] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T18:44:56.013883] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-11-04T18:44:56.167042] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-11. [2025-11-04T18:44:56.695050] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-11 (0.53 seconds). [2025-11-04T18:44:56.695943] [INFO ] [REMOTE ] EQBK: Full fetch complete (15663 records). [2025-11-04T18:44:56.696013] [DEBUG] [VALIDATOR ] EQBK: Validating primary data. [2025-11-04T18:44:56.971921] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-10. [2025-11-04T18:44:57.053620] [DEBUG] [VALIDATOR ] EQBK: All expected trading days are present. [2025-11-04T18:44:57.053726] [DEBUG] [VALIDATOR ] EQBK: Primary data validation passed. [2025-11-04T18:44:57.054096] [INFO ] [MAIN ] EQBK: Fetched primary data is valid. [2025-11-04T18:44:57.054134] [DEBUG] [LOCAL ] EQBK: Saving primary data. [2025-11-04T18:44:57.054679] [DEBUG] [LOCAL ] EQBK: Trimmed 15663 records to 11046 (keeping 200 days of data) [2025-11-04T18:44:57.120820] [DEBUG] [LOCAL ] EQBK: Primary data saved (11046 records). [2025-11-04T18:44:57.120970] [INFO ] [MAIN ] EQBK: Full history saved. [2025-11-04T18:44:57.121052] [INFO ] [MAIN ] Update Progress: 755/2367 symbols completed. EQBK took 20.9s. Estimated remaining: 3h 43m 58s. Rate: 7.2 symbols/min [2025-11-04T18:44:57.287268] [INFO ] [MAIN ] EQNR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:44:57.334662] [DEBUG] [LOCAL ] EQNR: Loaded 53013 existing records from primary storage. [2025-11-04T18:44:57.334986] [INFO ] [MAIN ] EQNR: Loaded 53013 existing records. [2025-11-04T18:44:57.511894] [WARN ] [MAIN ] EQNR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:44:57.512118] [INFO ] [MAIN ] EQNR: Wiping existing primary data before full refresh. [2025-11-04T18:44:57.512164] [INFO ] [LOCAL ] EQNR: Starting deletion of primary data. [2025-11-04T18:44:57.512391] [DEBUG] [LOCAL ] EQNR: Deleted primary data file. [2025-11-04T18:44:57.512448] [INFO ] [MAIN ] EQNR: Starting full primary refresh. [2025-11-04T18:44:57.774188] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-06. [2025-11-04T18:44:57.870532] [INFO ] [REMOTE ] EQNR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:44:58.246168] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T18:44:58.567077] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-06. [2025-11-04T18:44:58.953687] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T18:44:59.367073] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-01. [2025-11-04T18:44:59.778839] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T18:45:00.167066] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-07. [2025-11-04T18:45:00.967065] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-07. [2025-11-04T18:45:01.180344] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-11-04T18:45:01.541863] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T18:45:01.767055] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-02. [2025-11-04T18:45:02.567064] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-11. [2025-11-04T18:45:02.663979] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-11-04T18:45:03.206720] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-11-04T18:45:03.367059] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-03. [2025-11-04T18:45:03.626427] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T18:45:03.627867] [INFO ] [REMOTE ] EPR: Full fetch complete (63217 records). [2025-11-04T18:45:03.628047] [DEBUG] [VALIDATOR ] EPR: Validating primary data. [2025-11-04T18:45:03.628734] [FATAL] [VALIDATOR ] EPR: Data is stale - most recent date (2025-11-03) is before last completed trading day (2025-11-04). Symbol may be delisted or halted. [2025-11-04T18:45:03.628820] [FATAL] [MAIN ] EPR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:45:03.628862] [WARN ] [MAIN ] EPR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:45:03.628899] [INFO ] [LOCAL ] EPR: Starting deletion of primary data. [2025-11-04T18:45:03.628965] [DEBUG] [LOCAL ] EPR: No primary data file found to delete. [2025-11-04T18:45:03.629000] [INFO ] [LOCAL ] EPR: Starting deletion of derivative data. [2025-11-04T18:45:03.629159] [DEBUG] [LOCAL ] EPR: Deleted derivative data file. [2025-11-04T18:45:03.629247] [INFO ] [MAIN ] Update Progress: 756/2367 symbols completed. EPR took 38.9s. Estimated remaining: 3h 43m 46s. Rate: 7.2 symbols/min [2025-11-04T18:45:03.792361] [INFO ] [MAIN ] EQR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:45:03.840115] [DEBUG] [LOCAL ] EQR: Loaded 51833 existing records from primary storage. [2025-11-04T18:45:03.840413] [INFO ] [MAIN ] EQR: Loaded 51833 existing records. [2025-11-04T18:45:04.014911] [WARN ] [MAIN ] EQR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:45:04.015137] [INFO ] [MAIN ] EQR: Wiping existing primary data before full refresh. [2025-11-04T18:45:04.015184] [INFO ] [LOCAL ] EQR: Starting deletion of primary data. [2025-11-04T18:45:04.015366] [DEBUG] [LOCAL ] EQR: Deleted primary data file. [2025-11-04T18:45:04.015403] [INFO ] [MAIN ] EQR: Starting full primary refresh. [2025-11-04T18:45:04.171241] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-08. [2025-11-04T18:45:04.367621] [INFO ] [REMOTE ] EQR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:45:04.750786] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T18:45:04.967026] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-04. [2025-11-04T18:45:05.510245] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T18:45:05.767038] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-09. [2025-11-04T18:45:06.243456] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T18:45:06.567067] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-01. [2025-11-04T18:45:07.119210] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T18:45:07.367113] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-05. [2025-11-04T18:45:08.007713] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T18:45:08.167044] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-02. [2025-11-04T18:45:08.967038] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-08. [2025-11-04T18:45:09.143635] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-11-04T18:45:09.353914] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T18:45:09.767047] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-03. [2025-11-04T18:45:10.567087] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-06. [2025-11-04T18:45:11.157892] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T18:45:11.367089] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-10. [2025-11-04T18:45:11.941852] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-08 (2.97 seconds). [2025-11-04T18:45:12.167097] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-09. [2025-11-04T18:45:12.502116] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-11-04T18:45:12.967052] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-07. [2025-11-04T18:45:13.113458] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-11-04T18:45:13.402372] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-11-04T18:45:13.767096] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-04. [2025-11-04T18:45:14.542896] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-11-04T18:45:14.567003] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-08. [2025-11-04T18:45:15.292806] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-11-04T18:45:15.367080] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-11. [2025-11-04T18:45:15.938950] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-11-04T18:45:16.167044] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-10. [2025-11-04T18:45:16.619553] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T18:45:16.621578] [INFO ] [REMOTE ] EQH: Full fetch complete (76589 records). [2025-11-04T18:45:16.621741] [DEBUG] [VALIDATOR ] EQH: Validating primary data. [2025-11-04T18:45:16.972538] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-09. [2025-11-04T18:45:17.013329] [DEBUG] [VALIDATOR ] EQH: All expected trading days are present. [2025-11-04T18:45:17.013412] [DEBUG] [VALIDATOR ] EQH: Primary data validation passed. [2025-11-04T18:45:17.014293] [INFO ] [MAIN ] EQH: Fetched primary data is valid. [2025-11-04T18:45:17.014357] [DEBUG] [LOCAL ] EQH: Saving primary data. [2025-11-04T18:45:17.015937] [DEBUG] [LOCAL ] EQH: Trimmed 76589 records to 52650 (keeping 200 days of data) [2025-11-04T18:45:17.326501] [DEBUG] [LOCAL ] EQH: Primary data saved (52650 records). [2025-11-04T18:45:17.326618] [INFO ] [MAIN ] EQH: Full history saved. [2025-11-04T18:45:17.326739] [INFO ] [MAIN ] Update Progress: 757/2367 symbols completed. EQH took 34.2s. Estimated remaining: 3h 43m 49s. Rate: 7.2 symbols/min [2025-11-04T18:45:17.537032] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T18:45:17.767095] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-11. [2025-11-04T18:45:18.556914] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-11-04T18:45:18.567309] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-05. [2025-11-04T18:45:19.036681] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-11 (1.27 seconds). [2025-11-04T18:45:19.038691] [INFO ] [REMOTE ] EPRT: Full fetch complete (70017 records). [2025-11-04T18:45:19.038817] [DEBUG] [VALIDATOR ] EPRT: Validating primary data. [2025-11-04T18:45:19.382395] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-10. [2025-11-04T18:45:19.442589] [DEBUG] [VALIDATOR ] EPRT: All expected trading days are present. [2025-11-04T18:45:19.442716] [DEBUG] [VALIDATOR ] EPRT: Primary data validation passed. [2025-11-04T18:45:19.443639] [INFO ] [MAIN ] EPRT: Fetched primary data is valid. [2025-11-04T18:45:19.443730] [DEBUG] [LOCAL ] EPRT: Saving primary data. [2025-11-04T18:45:19.445152] [DEBUG] [LOCAL ] EPRT: Trimmed 70017 records to 48427 (keeping 200 days of data) [2025-11-04T18:45:19.734253] [DEBUG] [LOCAL ] EPRT: Primary data saved (48427 records). [2025-11-04T18:45:19.734388] [INFO ] [MAIN ] EPRT: Full history saved. [2025-11-04T18:45:19.734459] [INFO ] [MAIN ] Update Progress: 758/2367 symbols completed. EPRT took 48.1s. Estimated remaining: 3h 43m 28s. Rate: 7.2 symbols/min [2025-11-04T18:45:20.075773] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-11-04T18:45:20.167063] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-06. [2025-11-04T18:45:21.130331] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-11-04T18:45:21.143563] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-11. [2025-11-04T18:45:21.613538] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T18:45:21.767041] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-07. [2025-11-04T18:45:23.044023] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-11 (1.90 seconds). [2025-11-04T18:45:23.046075] [INFO ] [REMOTE ] EQNR: Full fetch complete (76991 records). [2025-11-04T18:45:23.046213] [DEBUG] [VALIDATOR ] EQNR: Validating primary data. [2025-11-04T18:45:23.541166] [DEBUG] [VALIDATOR ] EQNR: All expected trading days are present. [2025-11-04T18:45:23.541420] [DEBUG] [VALIDATOR ] EQNR: Primary data validation passed. [2025-11-04T18:45:23.542458] [INFO ] [MAIN ] EQNR: Fetched primary data is valid. [2025-11-04T18:45:23.542552] [DEBUG] [LOCAL ] EQNR: Saving primary data. [2025-11-04T18:45:23.584748] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-11-04T18:45:23.595686] [DEBUG] [LOCAL ] EQNR: Trimmed 76991 records to 53013 (keeping 200 days of data) [2025-11-04T18:45:23.597171] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-08. [2025-11-04T18:45:23.911852] [DEBUG] [LOCAL ] EQNR: Primary data saved (53013 records). [2025-11-04T18:45:23.911982] [INFO ] [MAIN ] EQNR: Full history saved. [2025-11-04T18:45:23.912058] [INFO ] [MAIN ] Update Progress: 759/2367 symbols completed. EQNR took 26.8s. Estimated remaining: 3h 43m 11s. Rate: 7.2 symbols/min [2025-11-04T18:45:25.144662] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-11-04T18:45:25.157507] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-09. [2025-11-04T18:45:26.585231] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T18:45:26.597380] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-10. [2025-11-04T18:45:28.136843] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-11-04T18:45:28.149236] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-11. [2025-11-04T18:45:29.502380] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-11 (1.35 seconds). [2025-11-04T18:45:29.507370] [INFO ] [REMOTE ] EQR: Full fetch complete (74743 records). [2025-11-04T18:45:29.507604] [DEBUG] [VALIDATOR ] EQR: Validating primary data. [2025-11-04T18:45:30.012309] [DEBUG] [VALIDATOR ] EQR: All expected trading days are present. [2025-11-04T18:45:30.012411] [DEBUG] [VALIDATOR ] EQR: Primary data validation passed. [2025-11-04T18:45:30.013274] [INFO ] [MAIN ] EQR: Fetched primary data is valid. [2025-11-04T18:45:30.013340] [DEBUG] [LOCAL ] EQR: Saving primary data. [2025-11-04T18:45:30.014811] [DEBUG] [LOCAL ] EQR: Trimmed 74743 records to 51837 (keeping 200 days of data) [2025-11-04T18:45:30.333078] [DEBUG] [LOCAL ] EQR: Primary data saved (51837 records). [2025-11-04T18:45:30.333208] [INFO ] [MAIN ] EQR: Full history saved. [2025-11-04T18:45:30.333283] [INFO ] [MAIN ] Update Progress: 760/2367 symbols completed. EQR took 26.7s. Estimated remaining: 3h 42m 58s. Rate: 7.2 symbols/min [2025-11-04T18:45:31.003900] [INFO ] [MAIN ] ERO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:45:31.043621] [INFO ] [MAIN ] EQT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:45:31.052528] [INFO ] [MAIN ] ES: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:45:31.073211] [INFO ] [MAIN ] EQS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:45:31.069325] [DEBUG] [LOCAL ] EQT: No existing data file found in primary storage. [2025-11-04T18:45:31.073578] [WARN ] [MAIN ] EQT: No existing data found, performing full refresh instead. [2025-11-04T18:45:31.073751] [DEBUG] [LOCAL ] EQS: No existing data file found in primary storage. [2025-11-04T18:45:31.073978] [INFO ] [MAIN ] EQT: Wiping existing primary data before full refresh. [2025-11-04T18:45:31.074071] [WARN ] [MAIN ] EQS: No existing data found, performing full refresh instead. [2025-11-04T18:45:31.074515] [INFO ] [LOCAL ] EQT: Starting deletion of primary data. [2025-11-04T18:45:31.074776] [DEBUG] [LOCAL ] EQT: No primary data file found to delete. [2025-11-04T18:45:31.074690] [INFO ] [MAIN ] EQS: Wiping existing primary data before full refresh. [2025-11-04T18:45:31.074951] [INFO ] [LOCAL ] EQS: Starting deletion of primary data. [2025-11-04T18:45:31.074893] [INFO ] [MAIN ] EQT: Starting full primary refresh. [2025-11-04T18:45:31.075141] [DEBUG] [LOCAL ] EQS: No primary data file found to delete. [2025-11-04T18:45:31.080577] [INFO ] [MAIN ] EQS: Starting full primary refresh. [2025-11-04T18:45:31.578420] [DEBUG] [LOCAL ] ERO: Loaded 41882 existing records from primary storage. [2025-11-04T18:45:31.650117] [INFO ] [MAIN ] ERO: Loaded 41882 existing records. [2025-11-04T18:45:31.856389] [DEBUG] [LOCAL ] ES: Loaded 52647 existing records from primary storage. [2025-11-04T18:45:31.903796] [INFO ] [REMOTE ] EQT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:45:31.936068] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-01. [2025-11-04T18:45:31.942205] [INFO ] [MAIN ] ES: Loaded 52647 existing records. [2025-11-04T18:45:32.120247] [INFO ] [REMOTE ] EQS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:45:32.138473] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-01. [2025-11-04T18:45:32.184770] [WARN ] [MAIN ] ERO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:45:32.198980] [INFO ] [MAIN ] ERO: Wiping existing primary data before full refresh. [2025-11-04T18:45:32.199047] [INFO ] [LOCAL ] ERO: Starting deletion of primary data. [2025-11-04T18:45:32.202955] [DEBUG] [LOCAL ] ERO: Deleted primary data file. [2025-11-04T18:45:32.203271] [INFO ] [MAIN ] ERO: Starting full primary refresh. [2025-11-04T18:45:32.382685] [WARN ] [MAIN ] ES: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:45:32.398426] [INFO ] [MAIN ] ES: Wiping existing primary data before full refresh. [2025-11-04T18:45:32.403631] [INFO ] [LOCAL ] ES: Starting deletion of primary data. [2025-11-04T18:45:32.425778] [DEBUG] [LOCAL ] ES: Deleted primary data file. [2025-11-04T18:45:32.439256] [INFO ] [MAIN ] ES: Starting full primary refresh. [2025-11-04T18:45:32.594114] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-11-04T18:45:32.632892] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-02. [2025-11-04T18:45:32.927385] [INFO ] [REMOTE ] ERO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:45:32.938108] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-01. [2025-11-04T18:45:32.957093] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T18:45:32.976902] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-03. [2025-11-04T18:45:33.013846] [INFO ] [REMOTE ] ES: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:45:33.014891] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-01. [2025-11-04T18:45:33.245393] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-03 (0.27 seconds). [2025-11-04T18:45:33.246397] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-04. [2025-11-04T18:45:33.450580] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-04 (0.20 seconds). [2025-11-04T18:45:33.451696] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-05. [2025-11-04T18:45:33.712594] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-05 (0.26 seconds). [2025-11-04T18:45:33.713553] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-06. [2025-11-04T18:45:33.807204] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-11-04T18:45:33.817562] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-02. [2025-11-04T18:45:33.946740] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-06 (0.23 seconds). [2025-11-04T18:45:33.947680] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-07. [2025-11-04T18:45:34.174885] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-11-04T18:45:34.216524] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T18:45:34.325748] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-11-04T18:45:34.567033] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-08. [2025-11-04T18:45:34.965090] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-11-04T18:45:35.361000] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-11-04T18:45:35.371292] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-02. [2025-11-04T18:45:36.167107] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-03. [2025-11-04T18:45:36.470292] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T18:45:36.967079] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-03. [2025-11-04T18:45:37.767077] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-02. [2025-11-04T18:45:37.990110] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-11-04T18:45:38.208554] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T18:45:38.567059] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-09. [2025-11-04T18:45:38.957180] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-11-04T18:45:39.194883] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-11-04T18:45:39.367044] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-04. [2025-11-04T18:45:40.167034] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-10. [2025-11-04T18:45:40.445100] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-10 (0.28 seconds). [2025-11-04T18:45:40.954300] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-11-04T18:45:40.967001] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-05. [2025-11-04T18:45:41.767030] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-03. [2025-11-04T18:45:42.545956] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-11-04T18:45:42.567003] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-06. [2025-11-04T18:45:43.290890] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T18:45:43.367050] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-11. [2025-11-04T18:45:43.903178] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T18:45:43.903848] [INFO ] [REMOTE ] EQS: Full fetch complete (2354 records). [2025-11-04T18:45:43.903917] [DEBUG] [VALIDATOR ] EQS: Validating primary data. [2025-11-04T18:45:44.088328] [WARN ] [VALIDATOR ] EQS: Validation failed - insufficient historical data. Have 189 unique days, need 200 days [2025-11-04T18:45:44.088709] [FATAL] [MAIN ] EQS: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:45:44.088866] [WARN ] [MAIN ] EQS: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:45:44.088944] [INFO ] [LOCAL ] EQS: Starting deletion of primary data. [2025-11-04T18:45:44.089022] [DEBUG] [LOCAL ] EQS: No primary data file found to delete. [2025-11-04T18:45:44.089077] [INFO ] [LOCAL ] EQS: Starting deletion of derivative data. [2025-11-04T18:45:44.089121] [DEBUG] [LOCAL ] EQS: No derivative data file found to delete. [2025-11-04T18:45:44.089182] [INFO ] [MAIN ] Update Progress: 761/2367 symbols completed. EQS took 13.8s. Estimated remaining: 3h 43m 1s. Rate: 7.2 symbols/min [2025-11-04T18:45:44.168976] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-04. [2025-11-04T18:45:44.261242] [INFO ] [MAIN ] ESAB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:45:44.261770] [DEBUG] [LOCAL ] ESAB: No existing data file found in primary storage. [2025-11-04T18:45:44.261877] [WARN ] [MAIN ] ESAB: No existing data found, performing full refresh instead. [2025-11-04T18:45:44.262129] [INFO ] [MAIN ] ESAB: Wiping existing primary data before full refresh. [2025-11-04T18:45:44.262237] [INFO ] [LOCAL ] ESAB: Starting deletion of primary data. [2025-11-04T18:45:44.262331] [DEBUG] [LOCAL ] ESAB: No primary data file found to delete. [2025-11-04T18:45:44.262461] [INFO ] [MAIN ] ESAB: Starting full primary refresh. [2025-11-04T18:45:44.647017] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-06 (2.08 seconds). [2025-11-04T18:45:44.651526] [INFO ] [REMOTE ] ESAB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:45:44.967032] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-04. [2025-11-04T18:45:45.610547] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T18:45:45.767043] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-05. [2025-11-04T18:45:46.435836] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-11-04T18:45:46.567037] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-05. [2025-11-04T18:45:47.276266] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-11-04T18:45:47.367033] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-06. [2025-11-04T18:45:47.621600] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T18:45:48.167042] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-01. [2025-11-04T18:45:48.881565] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-11-04T18:45:48.967008] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-06. [2025-11-04T18:45:49.026095] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T18:45:49.767034] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-02. [2025-11-04T18:45:50.037138] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T18:45:50.567016] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-07. [2025-11-04T18:45:50.589081] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-11-04T18:45:51.367081] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-07. [2025-11-04T18:45:52.041978] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-11-04T18:45:52.167094] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-03. [2025-11-04T18:45:52.967041] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-08. [2025-11-04T18:45:53.000593] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-11-04T18:45:53.127684] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T18:45:53.767093] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-07. [2025-11-04T18:45:54.546579] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-11-04T18:45:54.566995] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-09. [2025-11-04T18:45:54.907911] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T18:45:55.367091] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-04. [2025-11-04T18:45:55.949973] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T18:45:56.172301] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-08. [2025-11-04T18:45:56.183047] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T18:45:56.967062] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-05. [2025-11-04T18:45:57.768827] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-10. [2025-11-04T18:45:57.773507] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T18:45:57.823353] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-11-04T18:45:58.567055] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-06. [2025-11-04T18:45:59.362493] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T18:45:59.373572] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-08. [2025-11-04T18:45:59.525150] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-11-04T18:46:00.167060] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-07. [2025-11-04T18:46:00.967033] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-11. [2025-11-04T18:46:01.053092] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-11-04T18:46:01.245417] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T18:46:01.586135] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T18:46:01.588237] [INFO ] [REMOTE ] ES: Full fetch complete (76451 records). [2025-11-04T18:46:01.588341] [DEBUG] [VALIDATOR ] ES: Validating primary data. [2025-11-04T18:46:01.779931] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-09. [2025-11-04T18:46:01.977398] [DEBUG] [VALIDATOR ] ES: All expected trading days are present. [2025-11-04T18:46:01.977477] [DEBUG] [VALIDATOR ] ES: Primary data validation passed. [2025-11-04T18:46:01.978334] [INFO ] [MAIN ] ES: Fetched primary data is valid. [2025-11-04T18:46:01.978397] [DEBUG] [LOCAL ] ES: Saving primary data. [2025-11-04T18:46:01.979800] [DEBUG] [LOCAL ] ES: Trimmed 76451 records to 52658 (keeping 200 days of data) [2025-11-04T18:46:02.296149] [DEBUG] [LOCAL ] ES: Primary data saved (52658 records). [2025-11-04T18:46:02.296282] [INFO ] [MAIN ] ES: Full history saved. [2025-11-04T18:46:02.296380] [INFO ] [MAIN ] Update Progress: 762/2367 symbols completed. ES took 32.0s. Estimated remaining: 3h 43m 14s. Rate: 7.2 symbols/min [2025-11-04T18:46:02.462897] [INFO ] [MAIN ] ESE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:46:02.479809] [DEBUG] [LOCAL ] ESE: Loaded 16143 existing records from primary storage. [2025-11-04T18:46:02.480043] [INFO ] [MAIN ] ESE: Loaded 16143 existing records. [2025-11-04T18:46:02.572160] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-08. [2025-11-04T18:46:02.659693] [WARN ] [MAIN ] ESE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:46:02.659935] [INFO ] [MAIN ] ESE: Wiping existing primary data before full refresh. [2025-11-04T18:46:02.659991] [INFO ] [LOCAL ] ESE: Starting deletion of primary data. [2025-11-04T18:46:02.660153] [DEBUG] [LOCAL ] ESE: Deleted primary data file. [2025-11-04T18:46:02.660207] [INFO ] [MAIN ] ESE: Starting full primary refresh. [2025-11-04T18:46:03.016114] [INFO ] [REMOTE ] ESE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:46:03.367086] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-09. [2025-11-04T18:46:03.498154] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-11-04T18:46:03.534510] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-11-04T18:46:04.167051] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-10. [2025-11-04T18:46:04.635761] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-11-04T18:46:04.967026] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-09. [2025-11-04T18:46:05.767052] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-01. [2025-11-04T18:46:06.024300] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T18:46:06.073612] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-11-04T18:46:06.347983] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-11-04T18:46:06.567042] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-11. [2025-11-04T18:46:06.837064] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-11 (0.27 seconds). [2025-11-04T18:46:06.839125] [INFO ] [REMOTE ] EQT: Full fetch complete (77860 records). [2025-11-04T18:46:06.839274] [DEBUG] [VALIDATOR ] EQT: Validating primary data. [2025-11-04T18:46:07.230737] [DEBUG] [VALIDATOR ] EQT: All expected trading days are present. [2025-11-04T18:46:07.230861] [DEBUG] [VALIDATOR ] EQT: Primary data validation passed. [2025-11-04T18:46:07.231841] [INFO ] [MAIN ] EQT: Fetched primary data is valid. [2025-11-04T18:46:07.231924] [DEBUG] [LOCAL ] EQT: Saving primary data. [2025-11-04T18:46:07.233722] [DEBUG] [LOCAL ] EQT: Trimmed 77860 records to 53684 (keeping 200 days of data) [2025-11-04T18:46:07.368028] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-02. [2025-11-04T18:46:07.613518] [DEBUG] [LOCAL ] EQT: Primary data saved (53684 records). [2025-11-04T18:46:07.613710] [INFO ] [MAIN ] EQT: Full history saved. [2025-11-04T18:46:07.613822] [INFO ] [MAIN ] Update Progress: 763/2367 symbols completed. EQT took 37.2s. Estimated remaining: 3h 42m 59s. Rate: 7.2 symbols/min [2025-11-04T18:46:07.786850] [INFO ] [MAIN ] ESI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:46:07.832006] [DEBUG] [LOCAL ] ESI: Loaded 50571 existing records from primary storage. [2025-11-04T18:46:07.832302] [INFO ] [MAIN ] ESI: Loaded 50571 existing records. [2025-11-04T18:46:08.011687] [WARN ] [MAIN ] ESI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:46:08.012009] [INFO ] [MAIN ] ESI: Wiping existing primary data before full refresh. [2025-11-04T18:46:08.012076] [INFO ] [LOCAL ] ESI: Starting deletion of primary data. [2025-11-04T18:46:08.012441] [DEBUG] [LOCAL ] ESI: Deleted primary data file. [2025-11-04T18:46:08.012496] [INFO ] [MAIN ] ESI: Starting full primary refresh. [2025-11-04T18:46:08.170408] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-10. [2025-11-04T18:46:08.245619] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-11-04T18:46:08.404825] [INFO ] [REMOTE ] ESI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:46:08.967083] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-03. [2025-11-04T18:46:09.726238] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T18:46:09.761746] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T18:46:09.772251] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-04. [2025-11-04T18:46:10.524765] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T18:46:10.567000] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-05. [2025-11-04T18:46:11.296450] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T18:46:11.367005] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-06. [2025-11-04T18:46:12.110671] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-11-04T18:46:12.167005] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-07. [2025-11-04T18:46:12.835863] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T18:46:12.967064] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-11. [2025-11-04T18:46:13.767038] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-08. [2025-11-04T18:46:14.100603] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T18:46:14.102374] [INFO ] [REMOTE ] ERO: Full fetch complete (60995 records). [2025-11-04T18:46:14.102449] [DEBUG] [VALIDATOR ] ERO: Validating primary data. [2025-11-04T18:46:14.483985] [DEBUG] [VALIDATOR ] ERO: All expected trading days are present. [2025-11-04T18:46:14.484135] [DEBUG] [VALIDATOR ] ERO: Primary data validation passed. [2025-11-04T18:46:14.484962] [INFO ] [MAIN ] ERO: Fetched primary data is valid. [2025-11-04T18:46:14.485024] [DEBUG] [LOCAL ] ERO: Saving primary data. [2025-11-04T18:46:14.486244] [DEBUG] [LOCAL ] ERO: Trimmed 60995 records to 41984 (keeping 200 days of data) [2025-11-04T18:46:14.575016] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-10. [2025-11-04T18:46:14.605968] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-11-04T18:46:14.740476] [DEBUG] [LOCAL ] ERO: Primary data saved (41984 records). [2025-11-04T18:46:14.740599] [INFO ] [MAIN ] ERO: Full history saved. [2025-11-04T18:46:14.740695] [INFO ] [MAIN ] Update Progress: 764/2367 symbols completed. ERO took 44.4s. Estimated remaining: 3h 42m 48s. Rate: 7.2 symbols/min [2025-11-04T18:46:14.921781] [INFO ] [MAIN ] ESNT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:46:14.955559] [DEBUG] [LOCAL ] ESNT: Loaded 37402 existing records from primary storage. [2025-11-04T18:46:14.955837] [INFO ] [MAIN ] ESNT: Loaded 37402 existing records. [2025-11-04T18:46:15.132566] [WARN ] [MAIN ] ESNT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:46:15.132822] [INFO ] [MAIN ] ESNT: Wiping existing primary data before full refresh. [2025-11-04T18:46:15.132879] [INFO ] [LOCAL ] ESNT: Starting deletion of primary data. [2025-11-04T18:46:15.133076] [DEBUG] [LOCAL ] ESNT: Deleted primary data file. [2025-11-04T18:46:15.133130] [INFO ] [MAIN ] ESNT: Starting full primary refresh. [2025-11-04T18:46:15.372203] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-09. [2025-11-04T18:46:15.486876] [INFO ] [REMOTE ] ESNT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:46:15.583316] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-11-04T18:46:16.167050] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-11. [2025-11-04T18:46:16.291547] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T18:46:16.967010] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-01. [2025-11-04T18:46:16.999321] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T18:46:17.000776] [INFO ] [REMOTE ] ESAB: Full fetch complete (37147 records). [2025-11-04T18:46:17.000879] [DEBUG] [VALIDATOR ] ESAB: Validating primary data. [2025-11-04T18:46:17.382792] [DEBUG] [VALIDATOR ] ESAB: All expected trading days are present. [2025-11-04T18:46:17.382870] [DEBUG] [VALIDATOR ] ESAB: Primary data validation passed. [2025-11-04T18:46:17.383391] [INFO ] [MAIN ] ESAB: Fetched primary data is valid. [2025-11-04T18:46:17.383431] [DEBUG] [LOCAL ] ESAB: Saving primary data. [2025-11-04T18:46:17.384289] [DEBUG] [LOCAL ] ESAB: Trimmed 37147 records to 26182 (keeping 200 days of data) [2025-11-04T18:46:17.544490] [DEBUG] [LOCAL ] ESAB: Primary data saved (26182 records). [2025-11-04T18:46:17.544613] [INFO ] [MAIN ] ESAB: Full history saved. [2025-11-04T18:46:17.544706] [INFO ] [MAIN ] Update Progress: 765/2367 symbols completed. ESAB took 33.5s. Estimated remaining: 3h 42m 28s. Rate: 7.2 symbols/min [2025-11-04T18:46:17.721963] [INFO ] [MAIN ] ESRT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:46:17.762860] [DEBUG] [LOCAL ] ESRT: Loaded 46420 existing records from primary storage. [2025-11-04T18:46:17.763176] [INFO ] [MAIN ] ESRT: Loaded 46420 existing records. [2025-11-04T18:46:17.774927] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-01. [2025-11-04T18:46:17.954023] [WARN ] [MAIN ] ESRT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:46:17.954246] [INFO ] [MAIN ] ESRT: Wiping existing primary data before full refresh. [2025-11-04T18:46:17.954303] [INFO ] [LOCAL ] ESRT: Starting deletion of primary data. [2025-11-04T18:46:17.954495] [DEBUG] [LOCAL ] ESRT: Deleted primary data file. [2025-11-04T18:46:17.954547] [INFO ] [MAIN ] ESRT: Starting full primary refresh. [2025-11-04T18:46:18.323718] [INFO ] [REMOTE ] ESRT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:46:18.351237] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T18:46:18.567036] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-01. [2025-11-04T18:46:18.956434] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T18:46:19.367074] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-02. [2025-11-04T18:46:19.645081] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T18:46:20.167085] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-02. [2025-11-04T18:46:20.433683] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T18:46:20.967046] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-02. [2025-11-04T18:46:21.360594] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T18:46:21.767083] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-10. [2025-11-04T18:46:22.179503] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T18:46:22.498485] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-11-04T18:46:22.567023] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-03. [2025-11-04T18:46:23.367051] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-03. [2025-11-04T18:46:24.048793] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-11-04T18:46:24.167016] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-04. [2025-11-04T18:46:24.645339] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T18:46:24.967020] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-11. [2025-11-04T18:46:25.558333] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T18:46:25.673078] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T18:46:25.674056] [INFO ] [REMOTE ] ESE: Full fetch complete (23150 records). [2025-11-04T18:46:25.674171] [DEBUG] [VALIDATOR ] ESE: Validating primary data. [2025-11-04T18:46:25.772120] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-05. [2025-11-04T18:46:26.034914] [DEBUG] [VALIDATOR ] ESE: All expected trading days are present. [2025-11-04T18:46:26.034990] [DEBUG] [VALIDATOR ] ESE: Primary data validation passed. [2025-11-04T18:46:26.035420] [INFO ] [MAIN ] ESE: Fetched primary data is valid. [2025-11-04T18:46:26.035457] [DEBUG] [LOCAL ] ESE: Saving primary data. [2025-11-04T18:46:26.036055] [DEBUG] [LOCAL ] ESE: Trimmed 23150 records to 16121 (keeping 200 days of data) [2025-11-04T18:46:26.135398] [DEBUG] [LOCAL ] ESE: Primary data saved (16121 records). [2025-11-04T18:46:26.135520] [INFO ] [MAIN ] ESE: Full history saved. [2025-11-04T18:46:26.135594] [INFO ] [MAIN ] Update Progress: 766/2367 symbols completed. ESE took 23.8s. Estimated remaining: 3h 42m 21s. Rate: 7.2 symbols/min [2025-11-04T18:46:26.301113] [INFO ] [MAIN ] ESS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:46:26.329741] [DEBUG] [LOCAL ] ESS: Loaded 30143 existing records from primary storage. [2025-11-04T18:46:26.329991] [INFO ] [MAIN ] ESS: Loaded 30143 existing records. [2025-11-04T18:46:26.502927] [WARN ] [MAIN ] ESS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:46:26.503120] [INFO ] [MAIN ] ESS: Wiping existing primary data before full refresh. [2025-11-04T18:46:26.503167] [INFO ] [LOCAL ] ESS: Starting deletion of primary data. [2025-11-04T18:46:26.503357] [DEBUG] [LOCAL ] ESS: Deleted primary data file. [2025-11-04T18:46:26.503433] [INFO ] [MAIN ] ESS: Starting full primary refresh. [2025-11-04T18:46:26.576278] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-04. [2025-11-04T18:46:26.858321] [INFO ] [REMOTE ] ESS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:46:27.225950] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T18:46:27.367024] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-06. [2025-11-04T18:46:27.787703] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T18:46:28.167068] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-03. [2025-11-04T18:46:28.491329] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T18:46:28.967032] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-07. [2025-11-04T18:46:29.492450] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T18:46:29.767045] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-04. [2025-11-04T18:46:30.312285] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T18:46:30.567087] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-01. [2025-11-04T18:46:31.032924] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T18:46:31.367056] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-05. [2025-11-04T18:46:31.574129] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T18:46:32.167045] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-08. [2025-11-04T18:46:32.363895] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-11-04T18:46:32.967083] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-02. [2025-11-04T18:46:33.400962] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T18:46:33.767081] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-05. [2025-11-04T18:46:34.001564] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T18:46:34.567048] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-03. [2025-11-04T18:46:34.836617] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T18:46:35.367060] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-06. [2025-11-04T18:46:35.610117] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T18:46:36.167039] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-06. [2025-11-04T18:46:36.628331] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-11-04T18:46:36.967084] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-07. [2025-11-04T18:46:37.107938] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-11-04T18:46:37.767080] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-07. [2025-11-04T18:46:38.251091] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T18:46:38.567059] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-04. [2025-11-04T18:46:39.367182] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-08. [2025-11-04T18:46:39.691767] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T18:46:40.167056] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-05. [2025-11-04T18:46:40.472863] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-07 (2.71 seconds). [2025-11-04T18:46:40.669186] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T18:46:40.967009] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-09. [2025-11-04T18:46:41.080788] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-11-04T18:46:41.767040] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-09. [2025-11-04T18:46:42.131766] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T18:46:42.567110] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-06. [2025-11-04T18:46:42.896504] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-11-04T18:46:43.367009] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-08. [2025-11-04T18:46:43.570223] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T18:46:44.167054] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-07. [2025-11-04T18:46:44.516310] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-11-04T18:46:44.967034] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-10. [2025-11-04T18:46:45.311404] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T18:46:45.767044] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-09. [2025-11-04T18:46:46.281663] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T18:46:46.567047] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-10. [2025-11-04T18:46:46.909313] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T18:46:47.367058] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-10. [2025-11-04T18:46:48.037518] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T18:46:48.167098] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-08. [2025-11-04T18:46:48.345055] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-11-04T18:46:48.967036] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-11. [2025-11-04T18:46:49.302193] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T18:46:49.767030] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-09. [2025-11-04T18:46:50.125405] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T18:46:50.127386] [INFO ] [REMOTE ] ESRT: Full fetch complete (66954 records). [2025-11-04T18:46:50.127477] [DEBUG] [VALIDATOR ] ESRT: Validating primary data. [2025-11-04T18:46:50.503463] [DEBUG] [VALIDATOR ] ESRT: All expected trading days are present. [2025-11-04T18:46:50.503544] [DEBUG] [VALIDATOR ] ESRT: Primary data validation passed. [2025-11-04T18:46:50.504327] [INFO ] [MAIN ] ESRT: Fetched primary data is valid. [2025-11-04T18:46:50.504392] [DEBUG] [LOCAL ] ESRT: Saving primary data. [2025-11-04T18:46:50.505725] [DEBUG] [LOCAL ] ESRT: Trimmed 66954 records to 46417 (keeping 200 days of data) [2025-11-04T18:46:50.587852] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-11. [2025-11-04T18:46:50.789156] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T18:46:50.808815] [DEBUG] [LOCAL ] ESRT: Primary data saved (46417 records). [2025-11-04T18:46:50.808940] [INFO ] [MAIN ] ESRT: Full history saved. [2025-11-04T18:46:50.809013] [INFO ] [MAIN ] Update Progress: 767/2367 symbols completed. ESRT took 33.3s. Estimated remaining: 3h 42m 46s. Rate: 7.2 symbols/min [2025-11-04T18:46:50.979358] [INFO ] [MAIN ] ESTC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:46:51.024241] [DEBUG] [LOCAL ] ESTC: Loaded 50206 existing records from primary storage. [2025-11-04T18:46:51.024698] [INFO ] [MAIN ] ESTC: Loaded 50206 existing records. [2025-11-04T18:46:51.227942] [WARN ] [MAIN ] ESTC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:46:51.228146] [INFO ] [MAIN ] ESTC: Wiping existing primary data before full refresh. [2025-11-04T18:46:51.228203] [INFO ] [LOCAL ] ESTC: Starting deletion of primary data. [2025-11-04T18:46:51.228462] [DEBUG] [LOCAL ] ESTC: Deleted primary data file. [2025-11-04T18:46:51.228520] [INFO ] [MAIN ] ESTC: Starting full primary refresh. [2025-11-04T18:46:51.375892] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-11. [2025-11-04T18:46:51.605993] [INFO ] [REMOTE ] ESTC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:46:51.800826] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-11 (1.21 seconds). [2025-11-04T18:46:51.802848] [INFO ] [REMOTE ] ESI: Full fetch complete (73513 records). [2025-11-04T18:46:51.802960] [DEBUG] [VALIDATOR ] ESI: Validating primary data. [2025-11-04T18:46:52.185240] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-10. [2025-11-04T18:46:52.208334] [DEBUG] [VALIDATOR ] ESI: All expected trading days are present. [2025-11-04T18:46:52.208403] [DEBUG] [VALIDATOR ] ESI: Primary data validation passed. [2025-11-04T18:46:52.209245] [INFO ] [MAIN ] ESI: Fetched primary data is valid. [2025-11-04T18:46:52.209394] [DEBUG] [LOCAL ] ESI: Saving primary data. [2025-11-04T18:46:52.210776] [DEBUG] [LOCAL ] ESI: Trimmed 73513 records to 50576 (keeping 200 days of data) [2025-11-04T18:46:52.443328] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T18:46:52.446230] [INFO ] [REMOTE ] ESNT: Full fetch complete (56410 records). [2025-11-04T18:46:52.447402] [DEBUG] [VALIDATOR ] ESNT: Validating primary data. [2025-11-04T18:46:52.687815] [DEBUG] [LOCAL ] ESI: Primary data saved (50576 records). [2025-11-04T18:46:52.698271] [INFO ] [MAIN ] ESI: Full history saved. [2025-11-04T18:46:52.698409] [INFO ] [MAIN ] Update Progress: 768/2367 symbols completed. ESI took 45.1s. Estimated remaining: 3h 42m 25s. Rate: 7.2 symbols/min [2025-11-04T18:46:53.054388] [INFO ] [MAIN ] ET: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:46:52.975634] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-01. [2025-11-04T18:46:53.091037] [DEBUG] [VALIDATOR ] ESNT: All expected trading days are present. [2025-11-04T18:46:53.091123] [DEBUG] [VALIDATOR ] ESNT: Primary data validation passed. [2025-11-04T18:46:53.091851] [INFO ] [MAIN ] ESNT: Fetched primary data is valid. [2025-11-04T18:46:53.092020] [DEBUG] [LOCAL ] ESNT: Saving primary data. [2025-11-04T18:46:53.093277] [DEBUG] [LOCAL ] ESNT: Trimmed 56410 records to 37308 (keeping 200 days of data) [2025-11-04T18:46:53.341748] [DEBUG] [LOCAL ] ESNT: Primary data saved (37308 records). [2025-11-04T18:46:53.343584] [INFO ] [MAIN ] ESNT: Full history saved. [2025-11-04T18:46:53.343865] [INFO ] [MAIN ] Update Progress: 769/2367 symbols completed. ESNT took 38.6s. Estimated remaining: 3h 42m 0s. Rate: 7.2 symbols/min [2025-11-04T18:46:53.468661] [DEBUG] [LOCAL ] ET: Loaded 53789 existing records from primary storage. [2025-11-04T18:46:53.477200] [INFO ] [MAIN ] ET: Loaded 53789 existing records. [2025-11-04T18:46:53.627269] [INFO ] [MAIN ] ETB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:46:53.775221] [WARN ] [MAIN ] ET: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:46:53.776089] [INFO ] [MAIN ] ET: Wiping existing primary data before full refresh. [2025-11-04T18:46:53.776396] [INFO ] [LOCAL ] ET: Starting deletion of primary data. [2025-11-04T18:46:53.780443] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-11-04T18:46:53.780702] [DEBUG] [LOCAL ] ET: Deleted primary data file. [2025-11-04T18:46:53.785459] [INFO ] [MAIN ] ET: Starting full primary refresh. [2025-11-04T18:46:53.827141] [DEBUG] [LOCAL ] ETB: Loaded 11270 existing records from primary storage. [2025-11-04T18:46:53.838490] [INFO ] [MAIN ] ETB: Loaded 11270 existing records. [2025-11-04T18:46:53.877778] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-11. [2025-11-04T18:46:54.294994] [WARN ] [MAIN ] ETB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:46:54.295355] [INFO ] [MAIN ] ETB: Wiping existing primary data before full refresh. [2025-11-04T18:46:54.295449] [INFO ] [LOCAL ] ETB: Starting deletion of primary data. [2025-11-04T18:46:54.300968] [DEBUG] [LOCAL ] ETB: Deleted primary data file. [2025-11-04T18:46:54.317265] [INFO ] [MAIN ] ETB: Starting full primary refresh. [2025-11-04T18:46:54.640014] [INFO ] [REMOTE ] ET: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:46:54.672470] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-01. [2025-11-04T18:46:54.887547] [INFO ] [REMOTE ] ETB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:46:54.889678] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-11-04T18:46:55.063224] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T18:46:55.065179] [INFO ] [REMOTE ] ESS: Full fetch complete (44241 records). [2025-11-04T18:46:55.065265] [DEBUG] [VALIDATOR ] ESS: Validating primary data. [2025-11-04T18:46:55.372240] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-01. [2025-11-04T18:46:55.531624] [DEBUG] [VALIDATOR ] ESS: All expected trading days are present. [2025-11-04T18:46:55.531753] [DEBUG] [VALIDATOR ] ESS: Primary data validation passed. [2025-11-04T18:46:55.532505] [INFO ] [MAIN ] ESS: Fetched primary data is valid. [2025-11-04T18:46:55.532571] [DEBUG] [LOCAL ] ESS: Saving primary data. [2025-11-04T18:46:55.533684] [DEBUG] [LOCAL ] ESS: Trimmed 44241 records to 30180 (keeping 200 days of data) [2025-11-04T18:46:55.743540] [DEBUG] [LOCAL ] ESS: Primary data saved (30180 records). [2025-11-04T18:46:55.743703] [INFO ] [MAIN ] ESS: Full history saved. [2025-11-04T18:46:55.743797] [INFO ] [MAIN ] Update Progress: 770/2367 symbols completed. ESS took 29.6s. Estimated remaining: 3h 41m 40s. Rate: 7.2 symbols/min [2025-11-04T18:46:55.959220] [INFO ] [MAIN ] ETD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:46:55.967002] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-11-04T18:46:55.999537] [DEBUG] [LOCAL ] ETD: Loaded 26849 existing records from primary storage. [2025-11-04T18:46:55.999933] [INFO ] [MAIN ] ETD: Loaded 26849 existing records. [2025-11-04T18:46:56.172243] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-02. [2025-11-04T18:46:56.216514] [WARN ] [MAIN ] ETD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:46:56.216768] [INFO ] [MAIN ] ETD: Wiping existing primary data before full refresh. [2025-11-04T18:46:56.216824] [INFO ] [LOCAL ] ETD: Starting deletion of primary data. [2025-11-04T18:46:56.217015] [DEBUG] [LOCAL ] ETD: Deleted primary data file. [2025-11-04T18:46:56.217076] [INFO ] [MAIN ] ETD: Starting full primary refresh. [2025-11-04T18:46:56.672714] [INFO ] [REMOTE ] ETD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:46:56.705365] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-01 (2.03 seconds). [2025-11-04T18:46:56.778802] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-11-04T18:46:56.967048] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-02. [2025-11-04T18:46:57.767039] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-03. [2025-11-04T18:46:58.424758] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-11-04T18:46:58.567040] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-04. [2025-11-04T18:46:58.692767] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-11-04T18:46:59.080622] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-11-04T18:46:59.367112] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-02. [2025-11-04T18:47:00.167094] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-05. [2025-11-04T18:47:00.735057] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-11-04T18:47:00.772577] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-11-04T18:47:00.967003] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-03. [2025-11-04T18:47:01.767075] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-01. [2025-11-04T18:47:02.348452] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T18:47:02.567013] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-04. [2025-11-04T18:47:02.606560] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-11-04T18:47:03.367018] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-03. [2025-11-04T18:47:04.052402] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-11-04T18:47:04.167018] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-05. [2025-11-04T18:47:04.967032] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-02. [2025-11-04T18:47:05.254362] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-11-04T18:47:05.767031] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-06. [2025-11-04T18:47:05.798163] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T18:47:06.315241] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-11-04T18:47:06.572148] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-04. [2025-11-04T18:47:06.594463] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-05 (2.43 seconds). [2025-11-04T18:47:07.367040] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-06. [2025-11-04T18:47:08.167088] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-07. [2025-11-04T18:47:08.754618] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-11-04T18:47:08.852320] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-11-04T18:47:08.878425] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-11-04T18:47:08.967007] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-08. [2025-11-04T18:47:09.556627] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-11-04T18:47:09.767087] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-03. [2025-11-04T18:47:10.567016] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-09. [2025-11-04T18:47:10.594197] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-11-04T18:47:11.063580] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-11-04T18:47:11.367050] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-04. [2025-11-04T18:47:12.167053] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-07. [2025-11-04T18:47:12.455304] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T18:47:12.967069] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-05. [2025-11-04T18:47:13.488722] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T18:47:13.658603] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-11-04T18:47:13.767015] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-10. [2025-11-04T18:47:14.469847] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-11-04T18:47:14.567006] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-11. [2025-11-04T18:47:15.143225] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-11 (0.58 seconds). [2025-11-04T18:47:15.144070] [INFO ] [REMOTE ] ETB: Full fetch complete (18537 records). [2025-11-04T18:47:15.144141] [DEBUG] [VALIDATOR ] ETB: Validating primary data. [2025-11-04T18:47:15.370691] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-06. [2025-11-04T18:47:15.516191] [DEBUG] [VALIDATOR ] ETB: All expected trading days are present. [2025-11-04T18:47:15.516269] [DEBUG] [VALIDATOR ] ETB: Primary data validation passed. [2025-11-04T18:47:15.516666] [INFO ] [MAIN ] ETB: Fetched primary data is valid. [2025-11-04T18:47:15.516757] [DEBUG] [LOCAL ] ETB: Saving primary data. [2025-11-04T18:47:15.517324] [DEBUG] [LOCAL ] ETB: Trimmed 18537 records to 11215 (keeping 200 days of data) [2025-11-04T18:47:15.587686] [DEBUG] [LOCAL ] ETB: Primary data saved (11215 records). [2025-11-04T18:47:15.587875] [INFO ] [MAIN ] ETB: Full history saved. [2025-11-04T18:47:15.587972] [INFO ] [MAIN ] Update Progress: 771/2367 symbols completed. ETB took 22.2s. Estimated remaining: 3h 41m 55s. Rate: 7.2 symbols/min [2025-11-04T18:47:15.758945] [INFO ] [MAIN ] ETG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:47:15.775833] [DEBUG] [LOCAL ] ETG: Loaded 16056 existing records from primary storage. [2025-11-04T18:47:15.776185] [INFO ] [MAIN ] ETG: Loaded 16056 existing records. [2025-11-04T18:47:15.950582] [WARN ] [MAIN ] ETG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:47:15.950829] [INFO ] [MAIN ] ETG: Wiping existing primary data before full refresh. [2025-11-04T18:47:15.950881] [INFO ] [LOCAL ] ETG: Starting deletion of primary data. [2025-11-04T18:47:15.951044] [DEBUG] [LOCAL ] ETG: Deleted primary data file. [2025-11-04T18:47:15.951099] [INFO ] [MAIN ] ETG: Starting full primary refresh. [2025-11-04T18:47:16.176313] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-05. [2025-11-04T18:47:16.299712] [INFO ] [REMOTE ] ETG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:47:16.393139] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T18:47:16.967065] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-07. [2025-11-04T18:47:17.767002] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-08. [2025-11-04T18:47:17.941849] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-11-04T18:47:18.160564] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-11-04T18:47:18.567033] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-08. [2025-11-04T18:47:19.013499] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T18:47:19.367045] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-06. [2025-11-04T18:47:19.465195] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-11-04T18:47:20.167056] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-01. [2025-11-04T18:47:20.933993] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-11-04T18:47:20.967020] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-02. [2025-11-04T18:47:21.173169] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-11-04T18:47:21.684869] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T18:47:21.767030] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-03. [2025-11-04T18:47:22.522803] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T18:47:22.567001] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-04. [2025-11-04T18:47:23.300169] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T18:47:23.367009] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-09. [2025-11-04T18:47:24.096472] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-11-04T18:47:24.167010] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-10. [2025-11-04T18:47:24.962402] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T18:47:24.968155] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-07. [2025-11-04T18:47:25.767060] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-09. [2025-11-04T18:47:26.567066] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-05. [2025-11-04T18:47:26.761571] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-11-04T18:47:27.237828] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T18:47:27.323552] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-11-04T18:47:27.367002] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-10. [2025-11-04T18:47:28.167052] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-11. [2025-11-04T18:47:28.750023] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T18:47:28.957446] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-11 (0.79 seconds). [2025-11-04T18:47:28.958953] [INFO ] [REMOTE ] ETD: Full fetch complete (38728 records). [2025-11-04T18:47:28.959031] [DEBUG] [VALIDATOR ] ETD: Validating primary data. [2025-11-04T18:47:28.972112] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-08. [2025-11-04T18:47:29.372058] [DEBUG] [VALIDATOR ] ETD: All expected trading days are present. [2025-11-04T18:47:29.372145] [DEBUG] [VALIDATOR ] ETD: Primary data validation passed. [2025-11-04T18:47:29.372721] [INFO ] [MAIN ] ETD: Fetched primary data is valid. [2025-11-04T18:47:29.372783] [DEBUG] [LOCAL ] ETD: Saving primary data. [2025-11-04T18:47:29.373763] [DEBUG] [LOCAL ] ETD: Trimmed 38728 records to 26931 (keeping 200 days of data) [2025-11-04T18:47:29.541101] [DEBUG] [LOCAL ] ETD: Primary data saved (26931 records). [2025-11-04T18:47:29.541255] [INFO ] [MAIN ] ETD: Full history saved. [2025-11-04T18:47:29.541353] [INFO ] [MAIN ] Update Progress: 772/2367 symbols completed. ETD took 33.8s. Estimated remaining: 3h 41m 58s. Rate: 7.2 symbols/min [2025-11-04T18:47:29.724216] [INFO ] [MAIN ] ETJ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:47:29.740179] [DEBUG] [LOCAL ] ETJ: Loaded 15244 existing records from primary storage. [2025-11-04T18:47:29.740412] [INFO ] [MAIN ] ETJ: Loaded 15244 existing records. [2025-11-04T18:47:29.774608] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-11. [2025-11-04T18:47:29.922488] [WARN ] [MAIN ] ETJ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:47:29.922743] [INFO ] [MAIN ] ETJ: Wiping existing primary data before full refresh. [2025-11-04T18:47:29.922797] [INFO ] [LOCAL ] ETJ: Starting deletion of primary data. [2025-11-04T18:47:29.922956] [DEBUG] [LOCAL ] ETJ: Deleted primary data file. [2025-11-04T18:47:29.923011] [INFO ] [MAIN ] ETJ: Starting full primary refresh. [2025-11-04T18:47:30.272171] [INFO ] [REMOTE ] ETJ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:47:30.567015] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-01. [2025-11-04T18:47:30.622541] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-11-04T18:47:31.179342] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-11 (1.40 seconds). [2025-11-04T18:47:31.181308] [INFO ] [REMOTE ] ESTC: Full fetch complete (72569 records). [2025-11-04T18:47:31.181401] [DEBUG] [VALIDATOR ] ESTC: Validating primary data. [2025-11-04T18:47:31.347391] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-11-04T18:47:31.377001] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-02. [2025-11-04T18:47:31.598324] [DEBUG] [VALIDATOR ] ESTC: All expected trading days are present. [2025-11-04T18:47:31.598403] [DEBUG] [VALIDATOR ] ESTC: Primary data validation passed. [2025-11-04T18:47:31.599242] [INFO ] [MAIN ] ESTC: Fetched primary data is valid. [2025-11-04T18:47:31.599305] [DEBUG] [LOCAL ] ESTC: Saving primary data. [2025-11-04T18:47:31.600639] [DEBUG] [LOCAL ] ESTC: Trimmed 72569 records to 50272 (keeping 200 days of data) [2025-11-04T18:47:31.891409] [DEBUG] [LOCAL ] ESTC: Primary data saved (50272 records). [2025-11-04T18:47:31.891523] [INFO ] [MAIN ] ESTC: Full history saved. [2025-11-04T18:47:31.891599] [INFO ] [MAIN ] Update Progress: 773/2367 symbols completed. ESTC took 41.1s. Estimated remaining: 3h 41m 38s. Rate: 7.2 symbols/min [2025-11-04T18:47:32.061140] [INFO ] [MAIN ] ETN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:47:32.111180] [DEBUG] [LOCAL ] ETN: Loaded 52483 existing records from primary storage. [2025-11-04T18:47:32.111813] [INFO ] [MAIN ] ETN: Loaded 52483 existing records. [2025-11-04T18:47:32.172212] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-06. [2025-11-04T18:47:32.264608] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T18:47:32.329437] [WARN ] [MAIN ] ETN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:47:32.329701] [INFO ] [MAIN ] ETN: Wiping existing primary data before full refresh. [2025-11-04T18:47:32.329758] [INFO ] [LOCAL ] ETN: Starting deletion of primary data. [2025-11-04T18:47:32.330013] [DEBUG] [LOCAL ] ETN: Deleted primary data file. [2025-11-04T18:47:32.330068] [INFO ] [MAIN ] ETN: Starting full primary refresh. [2025-11-04T18:47:32.695752] [INFO ] [REMOTE ] ETN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:47:32.927197] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-11-04T18:47:32.967005] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-03. [2025-11-04T18:47:33.767028] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-09. [2025-11-04T18:47:33.823849] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-11-04T18:47:34.567045] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-01. [2025-11-04T18:47:35.367028] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-04. [2025-11-04T18:47:35.443451] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-11-04T18:47:36.091404] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-11-04T18:47:36.172118] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-07. [2025-11-04T18:47:36.189063] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-11-04T18:47:36.802573] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-11-04T18:47:36.967039] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-10. [2025-11-04T18:47:37.767110] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-05. [2025-11-04T18:47:38.215408] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-11-04T18:47:38.567060] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-06. [2025-11-04T18:47:38.999271] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-10 (2.03 seconds). [2025-11-04T18:47:39.152377] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-11-04T18:47:39.367038] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-07. [2025-11-04T18:47:39.976455] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-11-04T18:47:40.167004] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-02. [2025-11-04T18:47:40.967034] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-11. [2025-11-04T18:47:41.720152] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-11-04T18:47:41.767016] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-08. [2025-11-04T18:47:42.499925] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-11-04T18:47:42.567039] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-08. [2025-11-04T18:47:42.703746] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-11 (1.74 seconds). [2025-11-04T18:47:42.705810] [INFO ] [REMOTE ] ET: Full fetch complete (77969 records). [2025-11-04T18:47:42.705925] [DEBUG] [VALIDATOR ] ET: Validating primary data. [2025-11-04T18:47:43.097810] [DEBUG] [VALIDATOR ] ET: All expected trading days are present. [2025-11-04T18:47:43.097898] [DEBUG] [VALIDATOR ] ET: Primary data validation passed. [2025-11-04T18:47:43.098770] [INFO ] [MAIN ] ET: Fetched primary data is valid. [2025-11-04T18:47:43.098834] [DEBUG] [LOCAL ] ET: Saving primary data. [2025-11-04T18:47:43.100279] [DEBUG] [LOCAL ] ET: Trimmed 77969 records to 53789 (keeping 200 days of data) [2025-11-04T18:47:43.198884] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-11-04T18:47:43.391065] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-03. [2025-11-04T18:47:43.442725] [DEBUG] [LOCAL ] ET: Primary data saved (53789 records). [2025-11-04T18:47:43.442859] [INFO ] [MAIN ] ET: Full history saved. [2025-11-04T18:47:43.442935] [INFO ] [MAIN ] Update Progress: 774/2367 symbols completed. ET took 50.7s. Estimated remaining: 3h 41m 36s. Rate: 7.2 symbols/min [2025-11-04T18:47:43.624552] [INFO ] [MAIN ] ETO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:47:43.636036] [DEBUG] [LOCAL ] ETO: Loaded 6655 existing records from primary storage. [2025-11-04T18:47:43.636285] [INFO ] [MAIN ] ETO: Loaded 6655 existing records. [2025-11-04T18:47:43.826871] [WARN ] [MAIN ] ETO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:47:43.827119] [INFO ] [MAIN ] ETO: Wiping existing primary data before full refresh. [2025-11-04T18:47:43.827175] [INFO ] [LOCAL ] ETO: Starting deletion of primary data. [2025-11-04T18:47:43.827306] [DEBUG] [LOCAL ] ETO: Deleted primary data file. [2025-11-04T18:47:43.827344] [INFO ] [MAIN ] ETO: Starting full primary refresh. [2025-11-04T18:47:44.172130] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-09. [2025-11-04T18:47:44.201708] [INFO ] [REMOTE ] ETO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:47:44.785346] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-11-04T18:47:44.967023] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-09. [2025-11-04T18:47:45.043255] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-11-04T18:47:45.695554] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-11-04T18:47:45.766995] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-10. [2025-11-04T18:47:46.364127] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-11-04T18:47:46.567135] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-11. [2025-11-04T18:47:47.096454] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-11 (0.53 seconds). [2025-11-04T18:47:47.097474] [INFO ] [REMOTE ] ETJ: Full fetch complete (25034 records). [2025-11-04T18:47:47.097547] [DEBUG] [VALIDATOR ] ETJ: Validating primary data. [2025-11-04T18:47:47.372161] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-04. [2025-11-04T18:47:47.468540] [DEBUG] [VALIDATOR ] ETJ: All expected trading days are present. [2025-11-04T18:47:47.468613] [DEBUG] [VALIDATOR ] ETJ: Primary data validation passed. [2025-11-04T18:47:47.469115] [INFO ] [MAIN ] ETJ: Fetched primary data is valid. [2025-11-04T18:47:47.469174] [DEBUG] [LOCAL ] ETJ: Saving primary data. [2025-11-04T18:47:47.469873] [DEBUG] [LOCAL ] ETJ: Trimmed 25034 records to 15223 (keeping 200 days of data) [2025-11-04T18:47:47.560273] [DEBUG] [LOCAL ] ETJ: Primary data saved (15223 records). [2025-11-04T18:47:47.560412] [INFO ] [MAIN ] ETJ: Full history saved. [2025-11-04T18:47:47.560487] [INFO ] [MAIN ] Update Progress: 775/2367 symbols completed. ETJ took 18.0s. Estimated remaining: 3h 41m 19s. Rate: 7.2 symbols/min [2025-11-04T18:47:47.731479] [INFO ] [MAIN ] ETR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:47:47.779053] [DEBUG] [LOCAL ] ETR: Loaded 53174 existing records from primary storage. [2025-11-04T18:47:47.779442] [INFO ] [MAIN ] ETR: Loaded 53174 existing records. [2025-11-04T18:47:47.976105] [WARN ] [MAIN ] ETR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:47:47.976305] [INFO ] [MAIN ] ETR: Wiping existing primary data before full refresh. [2025-11-04T18:47:47.976359] [INFO ] [LOCAL ] ETR: Starting deletion of primary data. [2025-11-04T18:47:47.976548] [DEBUG] [LOCAL ] ETR: Deleted primary data file. [2025-11-04T18:47:47.976592] [INFO ] [MAIN ] ETR: Starting full primary refresh. [2025-11-04T18:47:48.172170] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-10. [2025-11-04T18:47:48.331903] [INFO ] [REMOTE ] ETR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:47:48.928106] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T18:47:48.949554] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-11-04T18:47:48.966985] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-11. [2025-11-04T18:47:49.613914] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-11 (0.65 seconds). [2025-11-04T18:47:49.615028] [INFO ] [REMOTE ] ETG: Full fetch complete (24391 records). [2025-11-04T18:47:49.615096] [DEBUG] [VALIDATOR ] ETG: Validating primary data. [2025-11-04T18:47:49.769441] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-01. [2025-11-04T18:47:49.976765] [DEBUG] [VALIDATOR ] ETG: All expected trading days are present. [2025-11-04T18:47:49.976857] [DEBUG] [VALIDATOR ] ETG: Primary data validation passed. [2025-11-04T18:47:49.977289] [INFO ] [MAIN ] ETG: Fetched primary data is valid. [2025-11-04T18:47:49.977330] [DEBUG] [LOCAL ] ETG: Saving primary data. [2025-11-04T18:47:49.977953] [DEBUG] [LOCAL ] ETG: Trimmed 24391 records to 16133 (keeping 200 days of data) [2025-11-04T18:47:50.071570] [DEBUG] [LOCAL ] ETG: Primary data saved (16133 records). [2025-11-04T18:47:50.071721] [INFO ] [MAIN ] ETG: Full history saved. [2025-11-04T18:47:50.071806] [INFO ] [MAIN ] Update Progress: 776/2367 symbols completed. ETG took 34.5s. Estimated remaining: 3h 40m 59s. Rate: 7.2 symbols/min [2025-11-04T18:47:50.235625] [INFO ] [MAIN ] ETSY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:47:50.283548] [DEBUG] [LOCAL ] ETSY: Loaded 53556 existing records from primary storage. [2025-11-04T18:47:50.283935] [INFO ] [MAIN ] ETSY: Loaded 53556 existing records. [2025-11-04T18:47:50.459594] [WARN ] [MAIN ] ETSY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:47:50.459833] [INFO ] [MAIN ] ETSY: Wiping existing primary data before full refresh. [2025-11-04T18:47:50.459881] [INFO ] [LOCAL ] ETSY: Starting deletion of primary data. [2025-11-04T18:47:50.460083] [DEBUG] [LOCAL ] ETSY: Deleted primary data file. [2025-11-04T18:47:50.460153] [INFO ] [MAIN ] ETSY: Starting full primary refresh. [2025-11-04T18:47:50.572176] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-05. [2025-11-04T18:47:50.814002] [INFO ] [REMOTE ] ETSY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:47:51.132540] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T18:47:51.367069] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-02. [2025-11-04T18:47:52.167100] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-01. [2025-11-04T18:47:52.272239] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-11-04T18:47:52.604129] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T18:47:52.967049] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-01. [2025-11-04T18:47:53.311513] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-11-04T18:47:53.730242] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-11-04T18:47:53.767017] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-02. [2025-11-04T18:47:54.567068] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-03. [2025-11-04T18:47:55.032203] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T18:47:55.367087] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-02. [2025-11-04T18:47:55.773712] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-11-04T18:47:56.171939] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-03. [2025-11-04T18:47:56.174742] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-11-04T18:47:56.967058] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-04. [2025-11-04T18:47:57.531375] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T18:47:57.767010] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-04. [2025-11-04T18:47:58.514406] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-11-04T18:47:58.566998] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-05. [2025-11-04T18:47:59.367084] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-06. [2025-11-04T18:47:59.723121] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-11-04T18:47:59.864300] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T18:48:00.167088] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-03. [2025-11-04T18:48:00.808239] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T18:48:00.967035] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-05. [2025-11-04T18:48:01.446240] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-06 (2.08 seconds). [2025-11-04T18:48:01.767057] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-04. [2025-11-04T18:48:02.292811] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T18:48:02.426745] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-11-04T18:48:02.567040] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-05. [2025-11-04T18:48:03.039048] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-11-04T18:48:03.367079] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-06. [2025-11-04T18:48:03.859234] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-11-04T18:48:04.167071] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-06. [2025-11-04T18:48:04.967131] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-07. [2025-11-04T18:48:05.767038] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-07. [2025-11-04T18:48:06.000372] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-11-04T18:48:06.508381] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T18:48:06.550776] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-11-04T18:48:06.567002] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-08. [2025-11-04T18:48:07.367045] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-07. [2025-11-04T18:48:08.172638] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-08. [2025-11-04T18:48:08.179055] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-11-04T18:48:08.627601] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-11-04T18:48:08.680547] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T18:48:08.967070] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-09. [2025-11-04T18:48:09.412365] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-11-04T18:48:09.767083] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-08. [2025-11-04T18:48:10.567059] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-06. [2025-11-04T18:48:11.252364] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T18:48:11.367034] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-09. [2025-11-04T18:48:12.012324] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T18:48:12.167015] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-09. [2025-11-04T18:48:12.967046] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-07. [2025-11-04T18:48:13.641557] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-09 (2.27 seconds). [2025-11-04T18:48:13.767018] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-10. [2025-11-04T18:48:13.813696] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-11-04T18:48:14.468350] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-11-04T18:48:14.567024] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-08. [2025-11-04T18:48:15.372175] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-10. [2025-11-04T18:48:15.387523] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-11-04T18:48:15.857143] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-11-04T18:48:15.971725] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T18:48:16.167035] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-09. [2025-11-04T18:48:16.967119] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-11. [2025-11-04T18:48:17.576032] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T18:48:17.577492] [INFO ] [REMOTE ] ETO: Full fetch complete (9722 records). [2025-11-04T18:48:17.577588] [DEBUG] [VALIDATOR ] ETO: Validating primary data. [2025-11-04T18:48:17.772199] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-10. [2025-11-04T18:48:18.062978] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-09 (1.90 seconds). [2025-11-04T18:48:18.082556] [DEBUG] [VALIDATOR ] ETO: All expected trading days are present. [2025-11-04T18:48:18.082990] [DEBUG] [VALIDATOR ] ETO: Primary data validation passed. [2025-11-04T18:48:18.083468] [INFO ] [MAIN ] ETO: Fetched primary data is valid. [2025-11-04T18:48:18.083980] [DEBUG] [LOCAL ] ETO: Saving primary data. [2025-11-04T18:48:18.084681] [DEBUG] [LOCAL ] ETO: Trimmed 9722 records to 6671 (keeping 200 days of data) [2025-11-04T18:48:18.127772] [DEBUG] [LOCAL ] ETO: Primary data saved (6671 records). [2025-11-04T18:48:18.127912] [INFO ] [MAIN ] ETO: Full history saved. [2025-11-04T18:48:18.128052] [INFO ] [MAIN ] Update Progress: 777/2367 symbols completed. ETO took 34.7s. Estimated remaining: 3h 41m 31s. Rate: 7.2 symbols/min [2025-11-04T18:48:18.300423] [INFO ] [MAIN ] ETV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:48:18.322574] [DEBUG] [LOCAL ] ETV: Loaded 21258 existing records from primary storage. [2025-11-04T18:48:18.322860] [INFO ] [MAIN ] ETV: Loaded 21258 existing records. [2025-11-04T18:48:18.521559] [WARN ] [MAIN ] ETV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:48:18.521791] [INFO ] [MAIN ] ETV: Wiping existing primary data before full refresh. [2025-11-04T18:48:18.521841] [INFO ] [LOCAL ] ETV: Starting deletion of primary data. [2025-11-04T18:48:18.522046] [DEBUG] [LOCAL ] ETV: Deleted primary data file. [2025-11-04T18:48:18.522103] [INFO ] [MAIN ] ETV: Starting full primary refresh. [2025-11-04T18:48:18.572190] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-10. [2025-11-04T18:48:18.887260] [INFO ] [REMOTE ] ETV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:48:19.368678] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-11. [2025-11-04T18:48:19.375295] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-11-04T18:48:20.167003] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-11. [2025-11-04T18:48:20.381263] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-11-04T18:48:20.960733] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-11 (1.59 seconds). [2025-11-04T18:48:20.962710] [INFO ] [REMOTE ] ETR: Full fetch complete (77265 records). [2025-11-04T18:48:20.962807] [DEBUG] [VALIDATOR ] ETR: Validating primary data. [2025-11-04T18:48:20.974663] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-01. [2025-11-04T18:48:21.353991] [DEBUG] [VALIDATOR ] ETR: All expected trading days are present. [2025-11-04T18:48:21.354094] [DEBUG] [VALIDATOR ] ETR: Primary data validation passed. [2025-11-04T18:48:21.355005] [INFO ] [MAIN ] ETR: Fetched primary data is valid. [2025-11-04T18:48:21.355071] [DEBUG] [LOCAL ] ETR: Saving primary data. [2025-11-04T18:48:21.356518] [DEBUG] [LOCAL ] ETR: Trimmed 77265 records to 53170 (keeping 200 days of data) [2025-11-04T18:48:21.588015] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-11 (1.42 seconds). [2025-11-04T18:48:21.592160] [INFO ] [REMOTE ] ETN: Full fetch complete (76601 records). [2025-11-04T18:48:21.593063] [DEBUG] [VALIDATOR ] ETN: Validating primary data. [2025-11-04T18:48:21.771207] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-11. [2025-11-04T18:48:21.945026] [DEBUG] [LOCAL ] ETR: Primary data saved (53170 records). [2025-11-04T18:48:21.957576] [INFO ] [MAIN ] ETR: Full history saved. [2025-11-04T18:48:21.972416] [INFO ] [MAIN ] Update Progress: 778/2367 symbols completed. ETR took 34.4s. Estimated remaining: 3h 41m 13s. Rate: 7.2 symbols/min [2025-11-04T18:48:22.261471] [DEBUG] [VALIDATOR ] ETN: All expected trading days are present. [2025-11-04T18:48:22.268497] [DEBUG] [VALIDATOR ] ETN: Primary data validation passed. [2025-11-04T18:48:22.278089] [INFO ] [MAIN ] ETN: Fetched primary data is valid. [2025-11-04T18:48:22.308763] [INFO ] [MAIN ] ETW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:48:22.278239] [DEBUG] [LOCAL ] ETN: Saving primary data. [2025-11-04T18:48:22.313268] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T18:48:22.314628] [DEBUG] [LOCAL ] ETN: Trimmed 76601 records to 52485 (keeping 200 days of data) [2025-11-04T18:48:22.524677] [DEBUG] [LOCAL ] ETW: Loaded 24469 existing records from primary storage. [2025-11-04T18:48:22.555611] [INFO ] [MAIN ] ETW: Loaded 24469 existing records. [2025-11-04T18:48:22.569408] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-02. [2025-11-04T18:48:22.828959] [DEBUG] [LOCAL ] ETN: Primary data saved (52485 records). [2025-11-04T18:48:22.831637] [INFO ] [MAIN ] ETN: Full history saved. [2025-11-04T18:48:22.831774] [INFO ] [MAIN ] Update Progress: 779/2367 symbols completed. ETN took 50.9s. Estimated remaining: 3h 40m 49s. Rate: 7.2 symbols/min [2025-11-04T18:48:22.931939] [WARN ] [MAIN ] ETW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:48:22.959377] [INFO ] [MAIN ] ETW: Wiping existing primary data before full refresh. [2025-11-04T18:48:22.959753] [INFO ] [LOCAL ] ETW: Starting deletion of primary data. [2025-11-04T18:48:22.975682] [DEBUG] [LOCAL ] ETW: Deleted primary data file. [2025-11-04T18:48:22.999058] [INFO ] [MAIN ] ETW: Starting full primary refresh. [2025-11-04T18:48:23.094909] [INFO ] [MAIN ] ETX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:48:23.188957] [DEBUG] [LOCAL ] ETX: Loaded 5345 existing records from primary storage. [2025-11-04T18:48:23.213815] [INFO ] [MAIN ] ETX: Loaded 5345 existing records. [2025-11-04T18:48:23.453352] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-11 (1.68 seconds). [2025-11-04T18:48:23.477332] [INFO ] [REMOTE ] ETSY: Full fetch complete (77672 records). [2025-11-04T18:48:23.536739] [DEBUG] [VALIDATOR ] ETSY: Validating primary data. [2025-11-04T18:48:23.650346] [WARN ] [MAIN ] ETX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:48:23.646906] [INFO ] [REMOTE ] ETW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:48:23.670606] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-01. [2025-11-04T18:48:23.715961] [INFO ] [MAIN ] ETX: Wiping existing primary data before full refresh. [2025-11-04T18:48:23.750393] [INFO ] [LOCAL ] ETX: Starting deletion of primary data. [2025-11-04T18:48:23.750232] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T18:48:23.766296] [DEBUG] [LOCAL ] ETX: Deleted primary data file. [2025-11-04T18:48:23.766447] [INFO ] [MAIN ] ETX: Starting full primary refresh. [2025-11-04T18:48:24.190351] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-03. [2025-11-04T18:48:24.265682] [DEBUG] [VALIDATOR ] ETSY: All expected trading days are present. [2025-11-04T18:48:24.293926] [DEBUG] [VALIDATOR ] ETSY: Primary data validation passed. [2025-11-04T18:48:24.332290] [INFO ] [MAIN ] ETSY: Fetched primary data is valid. [2025-11-04T18:48:24.343307] [DEBUG] [LOCAL ] ETSY: Saving primary data. [2025-11-04T18:48:24.410715] [DEBUG] [LOCAL ] ETSY: Trimmed 77672 records to 53556 (keeping 200 days of data) [2025-11-04T18:48:24.447725] [INFO ] [REMOTE ] ETX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:48:24.640496] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-11-04T18:48:24.782702] [DEBUG] [LOCAL ] ETSY: Primary data saved (53556 records). [2025-11-04T18:48:24.782841] [INFO ] [MAIN ] ETSY: Full history saved. [2025-11-04T18:48:24.782921] [INFO ] [MAIN ] Update Progress: 780/2367 symbols completed. ETSY took 34.7s. Estimated remaining: 3h 40m 28s. Rate: 7.2 symbols/min [2025-11-04T18:48:24.956630] [INFO ] [MAIN ] ETY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:48:24.968745] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-02. [2025-11-04T18:48:24.979786] [DEBUG] [LOCAL ] ETY: Loaded 21642 existing records from primary storage. [2025-11-04T18:48:24.980028] [INFO ] [MAIN ] ETY: Loaded 21642 existing records. [2025-11-04T18:48:25.119445] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T18:48:25.183974] [WARN ] [MAIN ] ETY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:48:25.184177] [INFO ] [MAIN ] ETY: Wiping existing primary data before full refresh. [2025-11-04T18:48:25.184223] [INFO ] [LOCAL ] ETY: Starting deletion of primary data. [2025-11-04T18:48:25.184383] [DEBUG] [LOCAL ] ETY: Deleted primary data file. [2025-11-04T18:48:25.184420] [INFO ] [MAIN ] ETY: Starting full primary refresh. [2025-11-04T18:48:25.539159] [INFO ] [REMOTE ] ETY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:48:25.756846] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T18:48:25.767003] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-03. [2025-11-04T18:48:26.577894] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-11-04T18:48:26.576599] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-01. [2025-11-04T18:48:26.937736] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-01 (0.36 seconds). [2025-11-04T18:48:27.367034] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-02. [2025-11-04T18:48:27.692505] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-11-04T18:48:28.167048] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-01. [2025-11-04T18:48:28.967045] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-04. [2025-11-04T18:48:28.985781] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-11-04T18:48:29.767045] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-03. [2025-11-04T18:48:29.918838] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-11-04T18:48:30.116020] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-11-04T18:48:30.567051] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-02. [2025-11-04T18:48:31.367011] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-04. [2025-11-04T18:48:31.430194] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-11-04T18:48:32.167027] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-04. [2025-11-04T18:48:32.198158] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-11-04T18:48:32.527816] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T18:48:32.967039] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-05. [2025-11-04T18:48:33.261177] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-05 (0.29 seconds). [2025-11-04T18:48:33.767105] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-06. [2025-11-04T18:48:34.305024] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-11-04T18:48:34.567043] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-05. [2025-11-04T18:48:35.367021] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-07. [2025-11-04T18:48:35.443919] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T18:48:35.759214] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-11-04T18:48:36.167044] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-05. [2025-11-04T18:48:36.929625] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-11-04T18:48:36.967034] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-03. [2025-11-04T18:48:37.767073] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-06. [2025-11-04T18:48:37.989754] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-11-04T18:48:38.567058] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-08. [2025-11-04T18:48:38.609206] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T18:48:39.073684] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-11-04T18:48:39.367063] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-06. [2025-11-04T18:48:40.000829] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-11-04T18:48:40.167084] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-04. [2025-11-04T18:48:40.967078] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-07. [2025-11-04T18:48:41.080764] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-11-04T18:48:41.767076] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-09. [2025-11-04T18:48:41.994300] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T18:48:42.071873] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-09 (0.30 seconds). [2025-11-04T18:48:42.567038] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-07. [2025-11-04T18:48:43.367536] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-10. [2025-11-04T18:48:43.370804] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-11-04T18:48:43.803412] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-11-04T18:48:44.167036] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-08. [2025-11-04T18:48:44.967010] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-05. [2025-11-04T18:48:45.193624] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-11-04T18:48:45.767018] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-08. [2025-11-04T18:48:45.821458] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-11-04T18:48:46.553509] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T18:48:46.567004] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-09. [2025-11-04T18:48:47.153902] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-11-04T18:48:47.367063] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-10. [2025-11-04T18:48:48.167085] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-11. [2025-11-04T18:48:48.204907] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-11-04T18:48:48.587860] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T18:48:48.588542] [INFO ] [REMOTE ] ETX: Full fetch complete (6835 records). [2025-11-04T18:48:48.588609] [DEBUG] [VALIDATOR ] ETX: Validating primary data. [2025-11-04T18:48:48.951376] [DEBUG] [VALIDATOR ] ETX: All expected trading days are present. [2025-11-04T18:48:48.951485] [DEBUG] [VALIDATOR ] ETX: Primary data validation passed. [2025-11-04T18:48:48.951712] [INFO ] [MAIN ] ETX: Fetched primary data is valid. [2025-11-04T18:48:48.951779] [DEBUG] [LOCAL ] ETX: Saving primary data. [2025-11-04T18:48:48.952166] [DEBUG] [LOCAL ] ETX: Trimmed 6835 records to 5325 (keeping 200 days of data) [2025-11-04T18:48:48.968323] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-09. [2025-11-04T18:48:48.985972] [DEBUG] [LOCAL ] ETX: Primary data saved (5325 records). [2025-11-04T18:48:48.986090] [INFO ] [MAIN ] ETX: Full history saved. [2025-11-04T18:48:48.986170] [INFO ] [MAIN ] Update Progress: 781/2367 symbols completed. ETX took 26.2s. Estimated remaining: 3h 40m 52s. Rate: 7.2 symbols/min [2025-11-04T18:48:49.159773] [INFO ] [MAIN ] EVC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:48:49.176873] [DEBUG] [LOCAL ] EVC: Loaded 17197 existing records from primary storage. [2025-11-04T18:48:49.177108] [INFO ] [MAIN ] EVC: Loaded 17197 existing records. [2025-11-04T18:48:49.351250] [WARN ] [MAIN ] EVC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:48:49.351501] [INFO ] [MAIN ] EVC: Wiping existing primary data before full refresh. [2025-11-04T18:48:49.351566] [INFO ] [LOCAL ] EVC: Starting deletion of primary data. [2025-11-04T18:48:49.351784] [DEBUG] [LOCAL ] EVC: Deleted primary data file. [2025-11-04T18:48:49.351839] [INFO ] [MAIN ] EVC: Starting full primary refresh. [2025-11-04T18:48:49.702853] [INFO ] [REMOTE ] EVC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:48:49.767009] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-01. [2025-11-04T18:48:49.960465] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T18:48:50.567014] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-10. [2025-11-04T18:48:50.681283] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-11-04T18:48:51.267212] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-11-04T18:48:51.367020] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-11. [2025-11-04T18:48:52.036384] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T18:48:52.037604] [INFO ] [REMOTE ] ETW: Full fetch complete (36360 records). [2025-11-04T18:48:52.037714] [DEBUG] [VALIDATOR ] ETW: Validating primary data. [2025-11-04T18:48:52.182005] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-11. [2025-11-04T18:48:52.414825] [DEBUG] [VALIDATOR ] ETW: All expected trading days are present. [2025-11-04T18:48:52.414907] [DEBUG] [VALIDATOR ] ETW: Primary data validation passed. [2025-11-04T18:48:52.415441] [INFO ] [MAIN ] ETW: Fetched primary data is valid. [2025-11-04T18:48:52.415480] [DEBUG] [LOCAL ] ETW: Saving primary data. [2025-11-04T18:48:52.416254] [DEBUG] [LOCAL ] ETW: Trimmed 36360 records to 24376 (keeping 200 days of data) [2025-11-04T18:48:52.555035] [DEBUG] [LOCAL ] ETW: Primary data saved (24376 records). [2025-11-04T18:48:52.555158] [INFO ] [MAIN ] ETW: Full history saved. [2025-11-04T18:48:52.555233] [INFO ] [MAIN ] Update Progress: 782/2367 symbols completed. ETW took 30.6s. Estimated remaining: 3h 40m 34s. Rate: 7.2 symbols/min [2025-11-04T18:48:52.722512] [INFO ] [MAIN ] EVEX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:48:52.754257] [DEBUG] [LOCAL ] EVEX: Loaded 34067 existing records from primary storage. [2025-11-04T18:48:52.754518] [INFO ] [MAIN ] EVEX: Loaded 34067 existing records. [2025-11-04T18:48:52.933902] [WARN ] [MAIN ] EVEX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:48:52.934122] [INFO ] [MAIN ] EVEX: Wiping existing primary data before full refresh. [2025-11-04T18:48:52.934178] [INFO ] [LOCAL ] EVEX: Starting deletion of primary data. [2025-11-04T18:48:52.934450] [DEBUG] [LOCAL ] EVEX: Deleted primary data file. [2025-11-04T18:48:52.934505] [INFO ] [MAIN ] EVEX: Starting full primary refresh. [2025-11-04T18:48:52.972737] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-02. [2025-11-04T18:48:53.012243] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T18:48:53.041845] [INFO ] [REMOTE ] ETV: Full fetch complete (31547 records). [2025-11-04T18:48:53.080154] [DEBUG] [VALIDATOR ] ETV: Validating primary data. [2025-11-04T18:48:53.479363] [INFO ] [REMOTE ] EVEX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:48:53.707142] [DEBUG] [VALIDATOR ] ETV: All expected trading days are present. [2025-11-04T18:48:53.707258] [DEBUG] [VALIDATOR ] ETV: Primary data validation passed. [2025-11-04T18:48:53.708024] [INFO ] [MAIN ] ETV: Fetched primary data is valid. [2025-11-04T18:48:53.708095] [DEBUG] [LOCAL ] ETV: Saving primary data. [2025-11-04T18:48:53.709319] [DEBUG] [LOCAL ] ETV: Trimmed 31547 records to 21111 (keeping 200 days of data) [2025-11-04T18:48:53.773894] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-06. [2025-11-04T18:48:53.854029] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-11-04T18:48:53.870673] [DEBUG] [LOCAL ] ETV: Primary data saved (21111 records). [2025-11-04T18:48:53.871321] [INFO ] [MAIN ] ETV: Full history saved. [2025-11-04T18:48:53.871456] [INFO ] [MAIN ] Update Progress: 783/2367 symbols completed. ETV took 35.7s. Estimated remaining: 3h 40m 11s. Rate: 7.2 symbols/min [2025-11-04T18:48:54.057989] [INFO ] [MAIN ] EVF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:48:54.070331] [DEBUG] [LOCAL ] EVF: Loaded 9314 existing records from primary storage. [2025-11-04T18:48:54.070732] [INFO ] [MAIN ] EVF: Loaded 9314 existing records. [2025-11-04T18:48:54.272454] [WARN ] [MAIN ] EVF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:48:54.272810] [INFO ] [MAIN ] EVF: Wiping existing primary data before full refresh. [2025-11-04T18:48:54.272868] [INFO ] [LOCAL ] EVF: Starting deletion of primary data. [2025-11-04T18:48:54.273052] [DEBUG] [LOCAL ] EVF: Deleted primary data file. [2025-11-04T18:48:54.273141] [INFO ] [MAIN ] EVF: Starting full primary refresh. [2025-11-04T18:48:54.574989] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-01. [2025-11-04T18:48:54.682276] [INFO ] [REMOTE ] EVF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:48:54.684386] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T18:48:55.367025] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-03. [2025-11-04T18:48:55.456378] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T18:48:56.167072] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-07. [2025-11-04T18:48:56.351511] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T18:48:56.967073] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-01. [2025-11-04T18:48:56.990778] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T18:48:57.409899] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-11-04T18:48:57.767085] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-02. [2025-11-04T18:48:58.158228] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-11-04T18:48:58.567039] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-08. [2025-11-04T18:48:59.295996] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-11-04T18:48:59.367087] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-04. [2025-11-04T18:49:00.151247] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-11-04T18:49:00.167025] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-05. [2025-11-04T18:49:00.967012] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-03. [2025-11-04T18:49:01.033340] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T18:49:01.535509] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T18:49:01.767098] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-09. [2025-11-04T18:49:02.567038] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-06. [2025-11-04T18:49:02.604867] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-11-04T18:49:03.367012] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-10. [2025-11-04T18:49:03.459020] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T18:49:04.167031] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-07. [2025-11-04T18:49:04.248390] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-11-04T18:49:04.906875] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T18:49:04.967131] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-11. [2025-11-04T18:49:05.759243] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-11 (0.79 seconds). [2025-11-04T18:49:05.760307] [INFO ] [REMOTE ] ETY: Full fetch complete (33536 records). [2025-11-04T18:49:05.760375] [DEBUG] [VALIDATOR ] ETY: Validating primary data. [2025-11-04T18:49:05.772247] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-04. [2025-11-04T18:49:06.127516] [DEBUG] [VALIDATOR ] ETY: All expected trading days are present. [2025-11-04T18:49:06.127614] [DEBUG] [VALIDATOR ] ETY: Primary data validation passed. [2025-11-04T18:49:06.128209] [INFO ] [MAIN ] ETY: Fetched primary data is valid. [2025-11-04T18:49:06.128272] [DEBUG] [LOCAL ] ETY: Saving primary data. [2025-11-04T18:49:06.129209] [DEBUG] [LOCAL ] ETY: Trimmed 33536 records to 21586 (keeping 200 days of data) [2025-11-04T18:49:06.256021] [DEBUG] [LOCAL ] ETY: Primary data saved (21586 records). [2025-11-04T18:49:06.256138] [INFO ] [MAIN ] ETY: Full history saved. [2025-11-04T18:49:06.256208] [INFO ] [MAIN ] Update Progress: 784/2367 symbols completed. ETY took 41.5s. Estimated remaining: 3h 40m 11s. Rate: 7.2 symbols/min [2025-11-04T18:49:06.426133] [INFO ] [MAIN ] EVG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:49:06.433456] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-11-04T18:49:06.442149] [DEBUG] [LOCAL ] EVG: Loaded 5857 existing records from primary storage. [2025-11-04T18:49:06.442319] [INFO ] [MAIN ] EVG: Loaded 5857 existing records. [2025-11-04T18:49:06.577510] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-08. [2025-11-04T18:49:06.616565] [WARN ] [MAIN ] EVG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:49:06.617157] [INFO ] [MAIN ] EVG: Wiping existing primary data before full refresh. [2025-11-04T18:49:06.617211] [INFO ] [LOCAL ] EVG: Starting deletion of primary data. [2025-11-04T18:49:06.617341] [DEBUG] [LOCAL ] EVG: Deleted primary data file. [2025-11-04T18:49:06.617378] [INFO ] [MAIN ] EVG: Starting full primary refresh. [2025-11-04T18:49:06.975813] [INFO ] [REMOTE ] EVG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:49:07.343222] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T18:49:07.367038] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-02. [2025-11-04T18:49:08.176912] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-01. [2025-11-04T18:49:08.178316] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-11-04T18:49:08.620465] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-11-04T18:49:08.967016] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-02. [2025-11-04T18:49:09.330464] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-11-04T18:49:09.767094] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-05. [2025-11-04T18:49:10.182717] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-11-04T18:49:10.567056] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-03. [2025-11-04T18:49:11.339348] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-11-04T18:49:11.367015] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-03. [2025-11-04T18:49:11.779147] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-11-04T18:49:12.167051] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-04. [2025-11-04T18:49:12.675957] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-11-04T18:49:12.967089] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-05. [2025-11-04T18:49:13.366045] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-11-04T18:49:13.767074] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-04. [2025-11-04T18:49:14.567033] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-09. [2025-11-04T18:49:14.586569] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T18:49:15.221358] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-11-04T18:49:15.367085] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-10. [2025-11-04T18:49:16.019717] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-11-04T18:49:16.167008] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-11. [2025-11-04T18:49:16.740226] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T18:49:16.741220] [INFO ] [REMOTE ] EVC: Full fetch complete (27826 records). [2025-11-04T18:49:16.741289] [DEBUG] [VALIDATOR ] EVC: Validating primary data. [2025-11-04T18:49:16.973180] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-06. [2025-11-04T18:49:17.104767] [DEBUG] [VALIDATOR ] EVC: All expected trading days are present. [2025-11-04T18:49:17.104839] [DEBUG] [VALIDATOR ] EVC: Primary data validation passed. [2025-11-04T18:49:17.105313] [INFO ] [MAIN ] EVC: Fetched primary data is valid. [2025-11-04T18:49:17.105351] [DEBUG] [LOCAL ] EVC: Saving primary data. [2025-11-04T18:49:17.105985] [DEBUG] [LOCAL ] EVC: Trimmed 27826 records to 17123 (keeping 200 days of data) [2025-11-04T18:49:17.200524] [DEBUG] [LOCAL ] EVC: Primary data saved (17123 records). [2025-11-04T18:49:17.200666] [INFO ] [MAIN ] EVC: Full history saved. [2025-11-04T18:49:17.200761] [INFO ] [MAIN ] Update Progress: 785/2367 symbols completed. EVC took 28.2s. Estimated remaining: 3h 40m 8s. Rate: 7.2 symbols/min [2025-11-04T18:49:17.370969] [INFO ] [MAIN ] EVH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:49:17.400002] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-11-04T18:49:17.418700] [DEBUG] [LOCAL ] EVH: Loaded 49513 existing records from primary storage. [2025-11-04T18:49:17.419003] [INFO ] [MAIN ] EVH: Loaded 49513 existing records. [2025-11-04T18:49:17.595408] [WARN ] [MAIN ] EVH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:49:17.595586] [INFO ] [MAIN ] EVH: Wiping existing primary data before full refresh. [2025-11-04T18:49:17.595665] [INFO ] [LOCAL ] EVH: Starting deletion of primary data. [2025-11-04T18:49:17.595917] [DEBUG] [LOCAL ] EVH: Deleted primary data file. [2025-11-04T18:49:17.595981] [INFO ] [MAIN ] EVH: Starting full primary refresh. [2025-11-04T18:49:17.768480] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-05. [2025-11-04T18:49:17.957619] [INFO ] [REMOTE ] EVH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:49:18.567044] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-07. [2025-11-04T18:49:18.592600] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T18:49:19.169418] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T18:49:19.367036] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-06. [2025-11-04T18:49:20.167038] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-01. [2025-11-04T18:49:20.222096] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T18:49:20.967055] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-07. [2025-11-04T18:49:21.432977] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T18:49:21.767033] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-08. [2025-11-04T18:49:21.977897] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T18:49:22.179599] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-11-04T18:49:22.567048] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-09. [2025-11-04T18:49:23.105525] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T18:49:23.367041] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-10. [2025-11-04T18:49:23.890868] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-11-04T18:49:24.167049] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-08. [2025-11-04T18:49:24.967100] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-06. [2025-11-04T18:49:25.339211] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T18:49:25.354180] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-11-04T18:49:25.767055] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-09. [2025-11-04T18:49:26.567069] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-07. [2025-11-04T18:49:27.367033] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-02. [2025-11-04T18:49:27.521570] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-11-04T18:49:27.559081] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-11-04T18:49:28.167051] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-11. [2025-11-04T18:49:28.630124] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T18:49:28.702842] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T18:49:28.703806] [INFO ] [REMOTE ] EVF: Full fetch complete (12984 records). [2025-11-04T18:49:28.703877] [DEBUG] [VALIDATOR ] EVF: Validating primary data. [2025-11-04T18:49:28.972279] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-10. [2025-11-04T18:49:29.092173] [DEBUG] [VALIDATOR ] EVF: All expected trading days are present. [2025-11-04T18:49:29.092275] [DEBUG] [VALIDATOR ] EVF: Primary data validation passed. [2025-11-04T18:49:29.092825] [INFO ] [MAIN ] EVF: Fetched primary data is valid. [2025-11-04T18:49:29.092892] [DEBUG] [LOCAL ] EVF: Saving primary data. [2025-11-04T18:49:29.093472] [DEBUG] [LOCAL ] EVF: Trimmed 12984 records to 9283 (keeping 200 days of data) [2025-11-04T18:49:29.152601] [DEBUG] [LOCAL ] EVF: Primary data saved (9283 records). [2025-11-04T18:49:29.152825] [INFO ] [MAIN ] EVF: Full history saved. [2025-11-04T18:49:29.152930] [INFO ] [MAIN ] Update Progress: 786/2367 symbols completed. EVF took 35.3s. Estimated remaining: 3h 40m 7s. Rate: 7.2 symbols/min [2025-11-04T18:49:29.335970] [INFO ] [MAIN ] EVN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:49:29.350139] [DEBUG] [LOCAL ] EVN: Loaded 10855 existing records from primary storage. [2025-11-04T18:49:29.350404] [INFO ] [MAIN ] EVN: Loaded 10855 existing records. [2025-11-04T18:49:29.538791] [WARN ] [MAIN ] EVN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:49:29.539092] [INFO ] [MAIN ] EVN: Wiping existing primary data before full refresh. [2025-11-04T18:49:29.539190] [INFO ] [LOCAL ] EVN: Starting deletion of primary data. [2025-11-04T18:49:29.539393] [DEBUG] [LOCAL ] EVN: Deleted primary data file. [2025-11-04T18:49:29.539436] [INFO ] [MAIN ] EVN: Starting full primary refresh. [2025-11-04T18:49:29.776305] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-08. [2025-11-04T18:49:29.925014] [INFO ] [REMOTE ] EVN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:49:30.272195] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-11-04T18:49:30.271991] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-11-04T18:49:30.567085] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-09. [2025-11-04T18:49:30.973798] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-11-04T18:49:31.367037] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-11. [2025-11-04T18:49:32.167050] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-01. [2025-11-04T18:49:32.599128] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T18:49:32.600882] [INFO ] [REMOTE ] EVEX: Full fetch complete (44961 records). [2025-11-04T18:49:32.600952] [DEBUG] [VALIDATOR ] EVEX: Validating primary data. [2025-11-04T18:49:32.915241] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T18:49:32.970530] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-03. [2025-11-04T18:49:33.008927] [DEBUG] [VALIDATOR ] EVEX: All expected trading days are present. [2025-11-04T18:49:33.009013] [DEBUG] [VALIDATOR ] EVEX: Primary data validation passed. [2025-11-04T18:49:33.009707] [INFO ] [MAIN ] EVEX: Fetched primary data is valid. [2025-11-04T18:49:33.009777] [DEBUG] [LOCAL ] EVEX: Saving primary data. [2025-11-04T18:49:33.010759] [DEBUG] [LOCAL ] EVEX: Trimmed 44961 records to 34370 (keeping 200 days of data) [2025-11-04T18:49:33.195876] [DEBUG] [LOCAL ] EVEX: Primary data saved (34370 records). [2025-11-04T18:49:33.196007] [INFO ] [MAIN ] EVEX: Full history saved. [2025-11-04T18:49:33.196085] [INFO ] [MAIN ] Update Progress: 787/2367 symbols completed. EVEX took 40.6s. Estimated remaining: 3h 39m 50s. Rate: 7.2 symbols/min [2025-11-04T18:49:33.363222] [INFO ] [MAIN ] EVR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:49:33.396969] [DEBUG] [LOCAL ] EVR: Loaded 32761 existing records from primary storage. [2025-11-04T18:49:33.397238] [INFO ] [MAIN ] EVR: Loaded 32761 existing records. [2025-11-04T18:49:33.573223] [WARN ] [MAIN ] EVR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:49:33.573419] [INFO ] [MAIN ] EVR: Wiping existing primary data before full refresh. [2025-11-04T18:49:33.573464] [INFO ] [LOCAL ] EVR: Starting deletion of primary data. [2025-11-04T18:49:33.573711] [DEBUG] [LOCAL ] EVR: Deleted primary data file. [2025-11-04T18:49:33.573767] [INFO ] [MAIN ] EVR: Starting full primary refresh. [2025-11-04T18:49:33.774789] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-10. [2025-11-04T18:49:33.950598] [INFO ] [REMOTE ] EVR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:49:34.226897] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-11-04T18:49:34.342210] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-11-04T18:49:34.567089] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-11. [2025-11-04T18:49:34.990171] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T18:49:34.991361] [INFO ] [REMOTE ] EVG: Full fetch complete (8264 records). [2025-11-04T18:49:34.991439] [DEBUG] [VALIDATOR ] EVG: Validating primary data. [2025-11-04T18:49:35.368639] [DEBUG] [VALIDATOR ] EVG: All expected trading days are present. [2025-11-04T18:49:35.368195] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-02. [2025-11-04T18:49:35.368924] [DEBUG] [VALIDATOR ] EVG: Primary data validation passed. [2025-11-04T18:49:35.370200] [INFO ] [MAIN ] EVG: Fetched primary data is valid. [2025-11-04T18:49:35.370271] [DEBUG] [LOCAL ] EVG: Saving primary data. [2025-11-04T18:49:35.371086] [DEBUG] [LOCAL ] EVG: Trimmed 8264 records to 5925 (keeping 200 days of data) [2025-11-04T18:49:35.412131] [DEBUG] [LOCAL ] EVG: Primary data saved (5925 records). [2025-11-04T18:49:35.412318] [INFO ] [MAIN ] EVG: Full history saved. [2025-11-04T18:49:35.412437] [INFO ] [MAIN ] Update Progress: 788/2367 symbols completed. EVG took 29.2s. Estimated remaining: 3h 39m 29s. Rate: 7.2 symbols/min [2025-11-04T18:49:35.594721] [INFO ] [MAIN ] EVT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:49:35.614222] [DEBUG] [LOCAL ] EVT: Loaded 16290 existing records from primary storage. [2025-11-04T18:49:35.614535] [INFO ] [MAIN ] EVT: Loaded 16290 existing records. [2025-11-04T18:49:35.805918] [WARN ] [MAIN ] EVT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:49:35.806241] [INFO ] [MAIN ] EVT: Wiping existing primary data before full refresh. [2025-11-04T18:49:35.806295] [INFO ] [LOCAL ] EVT: Starting deletion of primary data. [2025-11-04T18:49:35.806504] [DEBUG] [LOCAL ] EVT: Deleted primary data file. [2025-11-04T18:49:35.806546] [INFO ] [MAIN ] EVT: Starting full primary refresh. [2025-11-04T18:49:36.081398] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T18:49:36.177371] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-01. [2025-11-04T18:49:36.211906] [INFO ] [REMOTE ] EVT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:49:36.967045] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-04. [2025-11-04T18:49:37.140338] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T18:49:37.767058] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-03. [2025-11-04T18:49:38.224712] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T18:49:38.359946] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-11-04T18:49:38.567035] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-04. [2025-11-04T18:49:39.042027] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-11-04T18:49:39.367037] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-02. [2025-11-04T18:49:40.167055] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-01. [2025-11-04T18:49:40.358924] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T18:49:40.850442] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-11-04T18:49:40.967049] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-05. [2025-11-04T18:49:41.401977] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T18:49:41.767011] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-06. [2025-11-04T18:49:42.240458] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-11-04T18:49:42.567053] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-05. [2025-11-04T18:49:43.367037] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-03. [2025-11-04T18:49:43.765623] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T18:49:44.167073] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-07. [2025-11-04T18:49:44.467523] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T18:49:44.657378] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-11-04T18:49:44.967091] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-02. [2025-11-04T18:49:45.570797] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-11-04T18:49:45.767070] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-06. [2025-11-04T18:49:46.567093] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-03. [2025-11-04T18:49:46.950205] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T18:49:47.314762] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-11-04T18:49:47.367001] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-04. [2025-11-04T18:49:48.157234] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-11-04T18:49:48.166989] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-05. [2025-11-04T18:49:48.814697] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-11-04T18:49:48.967060] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-07. [2025-11-04T18:49:49.767045] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-08. [2025-11-04T18:49:50.148502] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T18:49:50.385346] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-11-04T18:49:50.567106] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-04. [2025-11-04T18:49:51.367058] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-08. [2025-11-04T18:49:51.687163] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T18:49:52.167075] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-09. [2025-11-04T18:49:52.500716] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T18:49:52.579013] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-11-04T18:49:52.967114] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-06. [2025-11-04T18:49:53.619318] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-11-04T18:49:53.767024] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-09. [2025-11-04T18:49:54.567036] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-07. [2025-11-04T18:49:54.956171] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T18:49:55.278943] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-11-04T18:49:55.367035] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-10. [2025-11-04T18:49:55.876081] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-11-04T18:49:56.167051] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-05. [2025-11-04T18:49:56.967051] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-11. [2025-11-04T18:49:57.184392] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T18:49:57.499768] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-11 (0.53 seconds). [2025-11-04T18:49:57.500763] [INFO ] [REMOTE ] EVN: Full fetch complete (16632 records). [2025-11-04T18:49:57.500837] [DEBUG] [VALIDATOR ] EVN: Validating primary data. [2025-11-04T18:49:57.774917] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-10. [2025-11-04T18:49:57.903771] [DEBUG] [VALIDATOR ] EVN: All expected trading days are present. [2025-11-04T18:49:57.903849] [DEBUG] [VALIDATOR ] EVN: Primary data validation passed. [2025-11-04T18:49:57.904210] [INFO ] [MAIN ] EVN: Fetched primary data is valid. [2025-11-04T18:49:57.904250] [DEBUG] [LOCAL ] EVN: Saving primary data. [2025-11-04T18:49:57.904794] [DEBUG] [LOCAL ] EVN: Trimmed 16632 records to 10810 (keeping 200 days of data) [2025-11-04T18:49:57.969150] [DEBUG] [LOCAL ] EVN: Primary data saved (10810 records). [2025-11-04T18:49:57.969293] [INFO ] [MAIN ] EVN: Full history saved. [2025-11-04T18:49:57.969376] [INFO ] [MAIN ] Update Progress: 789/2367 symbols completed. EVN took 28.8s. Estimated remaining: 3h 39m 49s. Rate: 7.2 symbols/min [2025-11-04T18:49:58.135548] [INFO ] [MAIN ] EVTC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:49:58.160679] [DEBUG] [LOCAL ] EVTC: Loaded 26832 existing records from primary storage. [2025-11-04T18:49:58.160930] [INFO ] [MAIN ] EVTC: Loaded 26832 existing records. [2025-11-04T18:49:58.335191] [WARN ] [MAIN ] EVTC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:49:58.335413] [INFO ] [MAIN ] EVTC: Wiping existing primary data before full refresh. [2025-11-04T18:49:58.335458] [INFO ] [LOCAL ] EVTC: Starting deletion of primary data. [2025-11-04T18:49:58.335628] [DEBUG] [LOCAL ] EVTC: Deleted primary data file. [2025-11-04T18:49:58.335712] [INFO ] [MAIN ] EVTC: Starting full primary refresh. [2025-11-04T18:49:58.576780] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-06. [2025-11-04T18:49:58.691144] [INFO ] [REMOTE ] EVTC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:49:59.132441] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-11-04T18:49:59.367046] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-01. [2025-11-04T18:49:59.557081] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T18:50:00.120633] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T18:50:00.167008] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-02. [2025-11-04T18:50:00.967074] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-08. [2025-11-04T18:50:01.236057] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T18:50:01.767005] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-07. [2025-11-04T18:50:01.915775] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T18:50:02.567115] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-09. [2025-11-04T18:50:02.940093] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T18:50:03.084423] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-11-04T18:50:03.367031] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-03. [2025-11-04T18:50:04.167016] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-10. [2025-11-04T18:50:04.297061] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T18:50:04.826322] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-11-04T18:50:04.967053] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-04. [2025-11-04T18:50:05.767028] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-11. [2025-11-04T18:50:05.868452] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-11-04T18:50:06.442918] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T18:50:06.443954] [INFO ] [REMOTE ] EVT: Full fetch complete (24575 records). [2025-11-04T18:50:06.444024] [DEBUG] [VALIDATOR ] EVT: Validating primary data. [2025-11-04T18:50:06.572171] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-08. [2025-11-04T18:50:06.814388] [DEBUG] [VALIDATOR ] EVT: All expected trading days are present. [2025-11-04T18:50:06.814462] [DEBUG] [VALIDATOR ] EVT: Primary data validation passed. [2025-11-04T18:50:06.814915] [INFO ] [MAIN ] EVT: Fetched primary data is valid. [2025-11-04T18:50:06.814975] [DEBUG] [LOCAL ] EVT: Saving primary data. [2025-11-04T18:50:06.815582] [DEBUG] [LOCAL ] EVT: Trimmed 24575 records to 16294 (keeping 200 days of data) [2025-11-04T18:50:06.911037] [DEBUG] [LOCAL ] EVT: Primary data saved (16294 records). [2025-11-04T18:50:06.911162] [INFO ] [MAIN ] EVT: Full history saved. [2025-11-04T18:50:06.911239] [INFO ] [MAIN ] Update Progress: 790/2367 symbols completed. EVT took 31.5s. Estimated remaining: 3h 39m 42s. Rate: 7.2 symbols/min [2025-11-04T18:50:07.084358] [INFO ] [MAIN ] EVTL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:50:07.121671] [DEBUG] [LOCAL ] EVTL: Loaded 43040 existing records from primary storage. [2025-11-04T18:50:07.121958] [INFO ] [MAIN ] EVTL: Loaded 43040 existing records. [2025-11-04T18:50:07.296839] [WARN ] [MAIN ] EVTL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:50:07.297060] [INFO ] [MAIN ] EVTL: Wiping existing primary data before full refresh. [2025-11-04T18:50:07.297107] [INFO ] [LOCAL ] EVTL: Starting deletion of primary data. [2025-11-04T18:50:07.297278] [DEBUG] [LOCAL ] EVTL: Deleted primary data file. [2025-11-04T18:50:07.297315] [INFO ] [MAIN ] EVTL: Starting full primary refresh. [2025-11-04T18:50:07.375716] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-11. [2025-11-04T18:50:07.668507] [INFO ] [REMOTE ] EVTL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:50:07.684007] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T18:50:08.167050] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-01. [2025-11-04T18:50:08.590456] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-11 (1.21 seconds). [2025-11-04T18:50:08.592451] [INFO ] [REMOTE ] EVH: Full fetch complete (71877 records). [2025-11-04T18:50:08.592572] [DEBUG] [VALIDATOR ] EVH: Validating primary data. [2025-11-04T18:50:08.973437] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-05. [2025-11-04T18:50:08.975122] [DEBUG] [VALIDATOR ] EVH: All expected trading days are present. [2025-11-04T18:50:08.975206] [DEBUG] [VALIDATOR ] EVH: Primary data validation passed. [2025-11-04T18:50:08.976108] [INFO ] [MAIN ] EVH: Fetched primary data is valid. [2025-11-04T18:50:08.976277] [DEBUG] [LOCAL ] EVH: Saving primary data. [2025-11-04T18:50:08.977714] [DEBUG] [LOCAL ] EVH: Trimmed 71877 records to 49470 (keeping 200 days of data) [2025-11-04T18:50:09.258124] [DEBUG] [LOCAL ] EVH: Primary data saved (49470 records). [2025-11-04T18:50:09.258356] [INFO ] [MAIN ] EVH: Full history saved. [2025-11-04T18:50:09.258476] [INFO ] [MAIN ] Update Progress: 791/2367 symbols completed. EVH took 52.1s. Estimated remaining: 3h 39m 22s. Rate: 7.2 symbols/min [2025-11-04T18:50:09.380881] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T18:50:09.455367] [INFO ] [MAIN ] EVTR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:50:09.481415] [DEBUG] [LOCAL ] EVTR: Loaded 25987 existing records from primary storage. [2025-11-04T18:50:09.481737] [INFO ] [MAIN ] EVTR: Loaded 25987 existing records. [2025-11-04T18:50:09.659713] [WARN ] [MAIN ] EVTR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:50:09.660053] [INFO ] [MAIN ] EVTR: Wiping existing primary data before full refresh. [2025-11-04T18:50:09.660103] [INFO ] [LOCAL ] EVTR: Starting deletion of primary data. [2025-11-04T18:50:09.660326] [DEBUG] [LOCAL ] EVTR: Deleted primary data file. [2025-11-04T18:50:09.660382] [INFO ] [MAIN ] EVTR: Starting full primary refresh. [2025-11-04T18:50:09.771998] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-09. [2025-11-04T18:50:10.034140] [INFO ] [REMOTE ] EVTR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:50:10.037180] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T18:50:10.567535] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-06. [2025-11-04T18:50:10.770633] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T18:50:11.353803] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T18:50:11.366993] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-07. [2025-11-04T18:50:12.167118] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-10. [2025-11-04T18:50:12.294392] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-11-04T18:50:12.967033] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-01. [2025-11-04T18:50:13.133716] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T18:50:13.661005] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T18:50:13.767091] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-11. [2025-11-04T18:50:14.567045] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-02. [2025-11-04T18:50:14.912958] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T18:50:14.915126] [INFO ] [REMOTE ] EVR: Full fetch complete (50713 records). [2025-11-04T18:50:14.915280] [DEBUG] [VALIDATOR ] EVR: Validating primary data. [2025-11-04T18:50:15.326137] [DEBUG] [VALIDATOR ] EVR: All expected trading days are present. [2025-11-04T18:50:15.326210] [DEBUG] [VALIDATOR ] EVR: Primary data validation passed. [2025-11-04T18:50:15.326899] [INFO ] [MAIN ] EVR: Fetched primary data is valid. [2025-11-04T18:50:15.326961] [DEBUG] [LOCAL ] EVR: Saving primary data. [2025-11-04T18:50:15.327977] [DEBUG] [LOCAL ] EVR: Trimmed 50713 records to 32856 (keeping 200 days of data) [2025-11-04T18:50:15.384711] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-02. [2025-11-04T18:50:15.533441] [DEBUG] [LOCAL ] EVR: Primary data saved (32856 records). [2025-11-04T18:50:15.533682] [INFO ] [MAIN ] EVR: Full history saved. [2025-11-04T18:50:15.533795] [INFO ] [MAIN ] Update Progress: 792/2367 symbols completed. EVR took 42.3s. Estimated remaining: 3h 39m 9s. Rate: 7.2 symbols/min [2025-11-04T18:50:15.652022] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T18:50:15.724891] [INFO ] [MAIN ] EW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:50:15.774852] [DEBUG] [LOCAL ] EW: Loaded 53388 existing records from primary storage. [2025-11-04T18:50:15.775200] [INFO ] [MAIN ] EW: Loaded 53388 existing records. [2025-11-04T18:50:15.951463] [WARN ] [MAIN ] EW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:50:15.951731] [INFO ] [MAIN ] EW: Wiping existing primary data before full refresh. [2025-11-04T18:50:15.951792] [INFO ] [LOCAL ] EW: Starting deletion of primary data. [2025-11-04T18:50:15.952014] [DEBUG] [LOCAL ] EW: Deleted primary data file. [2025-11-04T18:50:15.952097] [INFO ] [MAIN ] EW: Starting full primary refresh. [2025-11-04T18:50:16.190483] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-03. [2025-11-04T18:50:16.243546] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-11-04T18:50:16.339556] [INFO ] [REMOTE ] EW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:50:16.967095] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-01. [2025-11-04T18:50:17.177424] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T18:50:17.767048] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-04. [2025-11-04T18:50:18.230603] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T18:50:18.554616] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-11-04T18:50:18.566988] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-05. [2025-11-04T18:50:19.367051] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-08. [2025-11-04T18:50:19.590066] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T18:50:20.132096] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T18:50:20.167079] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-09. [2025-11-04T18:50:20.967079] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-06. [2025-11-04T18:50:21.094993] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-11-04T18:50:21.767034] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-02. [2025-11-04T18:50:22.206575] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T18:50:22.567084] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-10. [2025-11-04T18:50:22.980674] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T18:50:23.367039] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-03. [2025-11-04T18:50:23.491337] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-11-04T18:50:24.167075] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-07. [2025-11-04T18:50:24.603860] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T18:50:24.967075] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-04. [2025-11-04T18:50:25.707064] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-11-04T18:50:25.767016] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-08. [2025-11-04T18:50:26.231978] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T18:50:26.567099] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-11. [2025-11-04T18:50:27.036933] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T18:50:27.367016] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-05. [2025-11-04T18:50:27.428123] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T18:50:27.429522] [INFO ] [REMOTE ] EVTC: Full fetch complete (38763 records). [2025-11-04T18:50:27.429608] [DEBUG] [VALIDATOR ] EVTC: Validating primary data. [2025-11-04T18:50:27.799924] [DEBUG] [VALIDATOR ] EVTC: All expected trading days are present. [2025-11-04T18:50:27.800004] [DEBUG] [VALIDATOR ] EVTC: Primary data validation passed. [2025-11-04T18:50:27.800565] [INFO ] [MAIN ] EVTC: Fetched primary data is valid. [2025-11-04T18:50:27.800605] [DEBUG] [LOCAL ] EVTC: Saving primary data. [2025-11-04T18:50:27.801494] [DEBUG] [LOCAL ] EVTC: Trimmed 38763 records to 26849 (keeping 200 days of data) [2025-11-04T18:50:27.965235] [DEBUG] [LOCAL ] EVTC: Primary data saved (26849 records). [2025-11-04T18:50:27.965412] [INFO ] [MAIN ] EVTC: Full history saved. [2025-11-04T18:50:27.965545] [INFO ] [MAIN ] Update Progress: 793/2367 symbols completed. EVTC took 30.0s. Estimated remaining: 3h 39m 9s. Rate: 7.2 symbols/min [2025-11-04T18:50:28.132816] [INFO ] [MAIN ] EXG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:50:28.162352] [DEBUG] [LOCAL ] EXG: Loaded 32730 existing records from primary storage. [2025-11-04T18:50:28.162622] [INFO ] [MAIN ] EXG: Loaded 32730 existing records. [2025-11-04T18:50:28.173801] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-03. [2025-11-04T18:50:28.338179] [WARN ] [MAIN ] EXG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:50:28.338366] [INFO ] [MAIN ] EXG: Wiping existing primary data before full refresh. [2025-11-04T18:50:28.338414] [INFO ] [LOCAL ] EXG: Starting deletion of primary data. [2025-11-04T18:50:28.338580] [DEBUG] [LOCAL ] EXG: Deleted primary data file. [2025-11-04T18:50:28.338618] [INFO ] [MAIN ] EXG: Starting full primary refresh. [2025-11-04T18:50:28.691723] [INFO ] [REMOTE ] EXG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:50:28.721474] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-11-04T18:50:28.814434] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T18:50:28.967011] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-01. [2025-11-04T18:50:29.767090] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-04. [2025-11-04T18:50:30.004086] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-11-04T18:50:30.300152] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T18:50:30.567039] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-02. [2025-11-04T18:50:31.321797] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T18:50:31.367044] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-03. [2025-11-04T18:50:32.167079] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-09. [2025-11-04T18:50:32.447151] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T18:50:32.967069] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-04. [2025-11-04T18:50:33.528474] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T18:50:33.767803] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-06. [2025-11-04T18:50:33.986301] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T18:50:34.567061] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-10. [2025-11-04T18:50:34.976979] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T18:50:35.367039] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-07. [2025-11-04T18:50:36.176487] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-05. [2025-11-04T18:50:36.182046] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-11-04T18:50:36.666836] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T18:50:36.871133] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-11-04T18:50:36.967054] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-06. [2025-11-04T18:50:37.732156] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T18:50:37.767009] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-07. [2025-11-04T18:50:38.487364] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-11-04T18:50:38.567037] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-08. [2025-11-04T18:50:39.345728] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-11-04T18:50:39.367004] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-05. [2025-11-04T18:50:40.167022] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-11. [2025-11-04T18:50:40.319200] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-11-04T18:50:40.967049] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-09. [2025-11-04T18:50:41.389968] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T18:50:41.392211] [INFO ] [REMOTE ] EVTL: Full fetch complete (56445 records). [2025-11-04T18:50:41.392479] [DEBUG] [VALIDATOR ] EVTL: Validating primary data. [2025-11-04T18:50:41.777501] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-08. [2025-11-04T18:50:41.828865] [DEBUG] [VALIDATOR ] EVTL: All expected trading days are present. [2025-11-04T18:50:41.828949] [DEBUG] [VALIDATOR ] EVTL: Primary data validation passed. [2025-11-04T18:50:41.829764] [INFO ] [MAIN ] EVTL: Fetched primary data is valid. [2025-11-04T18:50:41.829830] [DEBUG] [LOCAL ] EVTL: Saving primary data. [2025-11-04T18:50:41.831386] [DEBUG] [LOCAL ] EVTL: Trimmed 56445 records to 43306 (keeping 200 days of data) [2025-11-04T18:50:42.065618] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T18:50:42.129826] [DEBUG] [LOCAL ] EVTL: Primary data saved (43306 records). [2025-11-04T18:50:42.130025] [INFO ] [MAIN ] EVTL: Full history saved. [2025-11-04T18:50:42.130144] [INFO ] [MAIN ] Update Progress: 794/2367 symbols completed. EVTL took 35.2s. Estimated remaining: 3h 39m 12s. Rate: 7.2 symbols/min [2025-11-04T18:50:42.313823] [INFO ] [MAIN ] EXK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:50:42.370951] [DEBUG] [LOCAL ] EXK: Loaded 53188 existing records from primary storage. [2025-11-04T18:50:42.371426] [INFO ] [MAIN ] EXK: Loaded 53188 existing records. [2025-11-04T18:50:42.571796] [WARN ] [MAIN ] EXK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:50:42.571926] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-06. [2025-11-04T18:50:42.573328] [INFO ] [MAIN ] EXK: Wiping existing primary data before full refresh. [2025-11-04T18:50:42.573392] [INFO ] [LOCAL ] EXK: Starting deletion of primary data. [2025-11-04T18:50:42.573625] [DEBUG] [LOCAL ] EXK: Deleted primary data file. [2025-11-04T18:50:42.573734] [INFO ] [MAIN ] EXK: Starting full primary refresh. [2025-11-04T18:50:42.991781] [INFO ] [REMOTE ] EXK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:50:43.218547] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-11-04T18:50:43.367139] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-10. [2025-11-04T18:50:43.431352] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T18:50:44.167075] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-07. [2025-11-04T18:50:44.392681] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-11-04T18:50:44.967081] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-01. [2025-11-04T18:50:45.175393] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T18:50:45.767115] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-11. [2025-11-04T18:50:46.210782] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-11-04T18:50:46.567019] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-02. [2025-11-04T18:50:46.580069] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T18:50:46.581464] [INFO ] [REMOTE ] EVTR: Full fetch complete (35122 records). [2025-11-04T18:50:46.581534] [DEBUG] [VALIDATOR ] EVTR: Validating primary data. [2025-11-04T18:50:46.955690] [DEBUG] [VALIDATOR ] EVTR: All expected trading days are present. [2025-11-04T18:50:46.955807] [DEBUG] [VALIDATOR ] EVTR: Primary data validation passed. [2025-11-04T18:50:46.956356] [INFO ] [MAIN ] EVTR: Fetched primary data is valid. [2025-11-04T18:50:46.956397] [DEBUG] [LOCAL ] EVTR: Saving primary data. [2025-11-04T18:50:46.957246] [DEBUG] [LOCAL ] EVTR: Trimmed 35122 records to 26068 (keeping 200 days of data) [2025-11-04T18:50:47.108522] [DEBUG] [LOCAL ] EVTR: Primary data saved (26068 records). [2025-11-04T18:50:47.108662] [INFO ] [MAIN ] EVTR: Full history saved. [2025-11-04T18:50:47.108756] [INFO ] [MAIN ] Update Progress: 795/2367 symbols completed. EVTR took 37.9s. Estimated remaining: 3h 38m 57s. Rate: 7.2 symbols/min [2025-11-04T18:50:47.273622] [INFO ] [MAIN ] EXP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:50:47.301159] [DEBUG] [LOCAL ] EXP: Loaded 28946 existing records from primary storage. [2025-11-04T18:50:47.301431] [INFO ] [MAIN ] EXP: Loaded 28946 existing records. [2025-11-04T18:50:47.373367] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-08. [2025-11-04T18:50:47.476723] [WARN ] [MAIN ] EXP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:50:47.476995] [INFO ] [MAIN ] EXP: Wiping existing primary data before full refresh. [2025-11-04T18:50:47.477041] [INFO ] [LOCAL ] EXP: Starting deletion of primary data. [2025-11-04T18:50:47.477217] [DEBUG] [LOCAL ] EXP: Deleted primary data file. [2025-11-04T18:50:47.477271] [INFO ] [MAIN ] EXP: Starting full primary refresh. [2025-11-04T18:50:47.836622] [INFO ] [REMOTE ] EXP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:50:47.865474] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T18:50:48.167094] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-09. [2025-11-04T18:50:48.358465] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-11-04T18:50:48.967048] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-03. [2025-11-04T18:50:49.481580] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T18:50:49.767112] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-09. [2025-11-04T18:50:50.230751] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-11-04T18:50:50.567050] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-04. [2025-11-04T18:50:50.729811] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T18:50:51.367043] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-01. [2025-11-04T18:50:51.881123] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T18:50:52.167048] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-05. [2025-11-04T18:50:52.241244] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T18:50:52.967023] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-02. [2025-11-04T18:50:53.476631] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T18:50:53.767066] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-10. [2025-11-04T18:50:53.852038] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T18:50:54.559382] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T18:50:54.566994] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-11. [2025-11-04T18:50:55.342140] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T18:50:55.343749] [INFO ] [REMOTE ] EXG: Full fetch complete (49307 records). [2025-11-04T18:50:55.343828] [DEBUG] [VALIDATOR ] EXG: Validating primary data. [2025-11-04T18:50:55.374850] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-10. [2025-11-04T18:50:55.742978] [DEBUG] [VALIDATOR ] EXG: All expected trading days are present. [2025-11-04T18:50:55.743057] [DEBUG] [VALIDATOR ] EXG: Primary data validation passed. [2025-11-04T18:50:55.743772] [INFO ] [MAIN ] EXG: Fetched primary data is valid. [2025-11-04T18:50:55.743837] [DEBUG] [LOCAL ] EXG: Saving primary data. [2025-11-04T18:50:55.744842] [DEBUG] [LOCAL ] EXG: Trimmed 49307 records to 32654 (keeping 200 days of data) [2025-11-04T18:50:55.948635] [DEBUG] [LOCAL ] EXG: Primary data saved (32654 records). [2025-11-04T18:50:55.948810] [INFO ] [MAIN ] EXG: Full history saved. [2025-11-04T18:50:55.948892] [INFO ] [MAIN ] Update Progress: 796/2367 symbols completed. EXG took 28.0s. Estimated remaining: 3h 38m 50s. Rate: 7.2 symbols/min [2025-11-04T18:50:56.130952] [INFO ] [MAIN ] EXPD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:50:56.173127] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-06. [2025-11-04T18:50:56.176517] [DEBUG] [LOCAL ] EXPD: Loaded 47382 existing records from primary storage. [2025-11-04T18:50:56.176996] [INFO ] [MAIN ] EXPD: Loaded 47382 existing records. [2025-11-04T18:50:56.358949] [WARN ] [MAIN ] EXPD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:50:56.359204] [INFO ] [MAIN ] EXPD: Wiping existing primary data before full refresh. [2025-11-04T18:50:56.359251] [INFO ] [LOCAL ] EXPD: Starting deletion of primary data. [2025-11-04T18:50:56.359445] [DEBUG] [LOCAL ] EXPD: Deleted primary data file. [2025-11-04T18:50:56.359502] [INFO ] [MAIN ] EXPD: Starting full primary refresh. [2025-11-04T18:50:56.707014] [INFO ] [REMOTE ] EXPD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:50:56.904010] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-11-04T18:50:56.967007] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-11. [2025-11-04T18:50:57.350847] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T18:50:57.353355] [INFO ] [REMOTE ] EW: Full fetch complete (77506 records). [2025-11-04T18:50:57.353521] [DEBUG] [VALIDATOR ] EW: Validating primary data. [2025-11-04T18:50:57.776414] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-03. [2025-11-04T18:50:57.850505] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-11-04T18:50:57.862284] [DEBUG] [VALIDATOR ] EW: All expected trading days are present. [2025-11-04T18:50:57.862538] [DEBUG] [VALIDATOR ] EW: Primary data validation passed. [2025-11-04T18:50:57.863542] [INFO ] [MAIN ] EW: Fetched primary data is valid. [2025-11-04T18:50:57.864275] [DEBUG] [LOCAL ] EW: Saving primary data. [2025-11-04T18:50:57.865816] [DEBUG] [LOCAL ] EW: Trimmed 77506 records to 53389 (keeping 200 days of data) [2025-11-04T18:50:58.177619] [DEBUG] [LOCAL ] EW: Primary data saved (53389 records). [2025-11-04T18:50:58.177809] [INFO ] [MAIN ] EW: Full history saved. [2025-11-04T18:50:58.177902] [INFO ] [MAIN ] Update Progress: 797/2367 symbols completed. EW took 42.6s. Estimated remaining: 3h 38m 29s. Rate: 7.2 symbols/min [2025-11-04T18:50:58.567121] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-07. [2025-11-04T18:50:58.807236] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T18:50:59.367120] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-04. [2025-11-04T18:51:00.090286] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-11-04T18:51:00.133256] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-11-04T18:51:00.167007] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-08. [2025-11-04T18:51:00.967068] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-01. [2025-11-04T18:51:01.694327] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T18:51:01.767013] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-09. [2025-11-04T18:51:02.198349] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T18:51:02.567107] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-02. [2025-11-04T18:51:03.171499] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T18:51:03.367036] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-10. [2025-11-04T18:51:03.784524] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T18:51:04.167064] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-05. [2025-11-04T18:51:04.967062] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-03. [2025-11-04T18:51:05.056462] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T18:51:05.311718] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T18:51:05.767094] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-06. [2025-11-04T18:51:06.185926] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T18:51:06.567015] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-11. [2025-11-04T18:51:06.636843] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T18:51:07.367040] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-07. [2025-11-04T18:51:07.985726] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-11 (1.42 seconds). [2025-11-04T18:51:07.987754] [INFO ] [REMOTE ] EXK: Full fetch complete (76948 records). [2025-11-04T18:51:07.987853] [DEBUG] [VALIDATOR ] EXK: Validating primary data. [2025-11-04T18:51:08.186014] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-04. [2025-11-04T18:51:08.381251] [DEBUG] [VALIDATOR ] EXK: All expected trading days are present. [2025-11-04T18:51:08.381403] [DEBUG] [VALIDATOR ] EXK: Primary data validation passed. [2025-11-04T18:51:08.382294] [INFO ] [MAIN ] EXK: Fetched primary data is valid. [2025-11-04T18:51:08.382373] [DEBUG] [LOCAL ] EXK: Saving primary data. [2025-11-04T18:51:08.383906] [DEBUG] [LOCAL ] EXK: Trimmed 76948 records to 53197 (keeping 200 days of data) [2025-11-04T18:51:08.508814] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T18:51:08.705908] [DEBUG] [LOCAL ] EXK: Primary data saved (53197 records). [2025-11-04T18:51:08.706060] [INFO ] [MAIN ] EXK: Full history saved. [2025-11-04T18:51:08.706167] [INFO ] [MAIN ] Update Progress: 798/2367 symbols completed. EXK took 26.6s. Estimated remaining: 3h 38m 25s. Rate: 7.2 symbols/min [2025-11-04T18:51:08.967097] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-08. [2025-11-04T18:51:09.546212] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T18:51:09.766996] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-05. [2025-11-04T18:51:09.883456] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-11-04T18:51:10.567092] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-09. [2025-11-04T18:51:11.079953] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T18:51:11.366995] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-06. [2025-11-04T18:51:11.511457] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-11-04T18:51:12.167078] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-10. [2025-11-04T18:51:12.718466] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-11-04T18:51:12.967005] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-07. [2025-11-04T18:51:13.042721] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-11-04T18:51:13.767042] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-11. [2025-11-04T18:51:14.164606] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T18:51:14.507842] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T18:51:14.509409] [INFO ] [REMOTE ] EXP: Full fetch complete (43195 records). [2025-11-04T18:51:14.509488] [DEBUG] [VALIDATOR ] EXP: Validating primary data. [2025-11-04T18:51:14.573223] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-08. [2025-11-04T18:51:14.899814] [DEBUG] [VALIDATOR ] EXP: All expected trading days are present. [2025-11-04T18:51:14.899894] [DEBUG] [VALIDATOR ] EXP: Primary data validation passed. [2025-11-04T18:51:14.900481] [INFO ] [MAIN ] EXP: Fetched primary data is valid. [2025-11-04T18:51:14.900521] [DEBUG] [LOCAL ] EXP: Saving primary data. [2025-11-04T18:51:14.901468] [DEBUG] [LOCAL ] EXP: Trimmed 43195 records to 29050 (keeping 200 days of data) [2025-11-04T18:51:15.076293] [DEBUG] [LOCAL ] EXP: Primary data saved (29050 records). [2025-11-04T18:51:15.076417] [INFO ] [MAIN ] EXP: Full history saved. [2025-11-04T18:51:15.076499] [INFO ] [MAIN ] Update Progress: 799/2367 symbols completed. EXP took 28.0s. Estimated remaining: 3h 38m 13s. Rate: 7.2 symbols/min [2025-11-04T18:51:15.728448] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T18:51:15.738562] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-09. [2025-11-04T18:51:16.933523] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T18:51:16.944003] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-10. [2025-11-04T18:51:18.099325] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T18:51:18.110258] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-11. [2025-11-04T18:51:18.366752] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-11 (0.26 seconds). [2025-11-04T18:51:18.369107] [INFO ] [REMOTE ] EXPD: Full fetch complete (69280 records). [2025-11-04T18:51:18.369229] [DEBUG] [VALIDATOR ] EXPD: Validating primary data. [2025-11-04T18:51:18.759696] [DEBUG] [VALIDATOR ] EXPD: All expected trading days are present. [2025-11-04T18:51:18.759791] [DEBUG] [VALIDATOR ] EXPD: Primary data validation passed. [2025-11-04T18:51:18.760611] [INFO ] [MAIN ] EXPD: Fetched primary data is valid. [2025-11-04T18:51:18.760698] [DEBUG] [LOCAL ] EXPD: Saving primary data. [2025-11-04T18:51:18.762014] [DEBUG] [LOCAL ] EXPD: Trimmed 69280 records to 47420 (keeping 200 days of data) [2025-11-04T18:51:19.057887] [DEBUG] [LOCAL ] EXPD: Primary data saved (47420 records). [2025-11-04T18:51:19.058034] [INFO ] [MAIN ] EXPD: Full history saved. [2025-11-04T18:51:19.058127] [INFO ] [MAIN ] Update Progress: 800/2367 symbols completed. EXPD took 23.1s. Estimated remaining: 3h 37m 56s. Rate: 7.2 symbols/min [2025-11-04T18:51:19.686436] [INFO ] [MAIN ] F: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:51:19.678839] [INFO ] [MAIN ] FBIN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:51:19.774428] [INFO ] [MAIN ] EXR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:51:19.795105] [INFO ] [MAIN ] FAF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:51:19.894097] [DEBUG] [LOCAL ] FBIN: Loaded 50140 existing records from primary storage. [2025-11-04T18:51:19.895333] [INFO ] [MAIN ] FBIN: Loaded 50140 existing records. [2025-11-04T18:51:19.895167] [DEBUG] [LOCAL ] F: Loaded 53820 existing records from primary storage. [2025-11-04T18:51:19.917705] [INFO ] [MAIN ] F: Loaded 53820 existing records. [2025-11-04T18:51:19.933562] [DEBUG] [LOCAL ] FAF: Loaded 41156 existing records from primary storage. [2025-11-04T18:51:20.026836] [DEBUG] [LOCAL ] EXR: Loaded 45344 existing records from primary storage. [2025-11-04T18:51:20.045966] [INFO ] [MAIN ] FAF: Loaded 41156 existing records. [2025-11-04T18:51:20.103220] [INFO ] [MAIN ] EXR: Loaded 45344 existing records. [2025-11-04T18:51:20.438815] [WARN ] [MAIN ] FBIN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:51:20.496374] [INFO ] [MAIN ] FBIN: Wiping existing primary data before full refresh. [2025-11-04T18:51:20.624241] [WARN ] [MAIN ] F: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:51:20.535766] [INFO ] [LOCAL ] FBIN: Starting deletion of primary data. [2025-11-04T18:51:20.701689] [INFO ] [MAIN ] F: Wiping existing primary data before full refresh. [2025-11-04T18:51:20.706009] [DEBUG] [LOCAL ] FBIN: Deleted primary data file. [2025-11-04T18:51:20.727026] [WARN ] [MAIN ] EXR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:51:20.705867] [WARN ] [MAIN ] FAF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:51:20.727286] [INFO ] [MAIN ] EXR: Wiping existing primary data before full refresh. [2025-11-04T18:51:20.727159] [INFO ] [MAIN ] FBIN: Starting full primary refresh. [2025-11-04T18:51:20.738732] [INFO ] [LOCAL ] EXR: Starting deletion of primary data. [2025-11-04T18:51:20.762176] [INFO ] [MAIN ] FAF: Wiping existing primary data before full refresh. [2025-11-04T18:51:20.738511] [INFO ] [LOCAL ] F: Starting deletion of primary data. [2025-11-04T18:51:20.767786] [INFO ] [LOCAL ] FAF: Starting deletion of primary data. [2025-11-04T18:51:20.767844] [DEBUG] [LOCAL ] EXR: Deleted primary data file. [2025-11-04T18:51:20.805601] [INFO ] [MAIN ] EXR: Starting full primary refresh. [2025-11-04T18:51:20.789977] [DEBUG] [LOCAL ] F: Deleted primary data file. [2025-11-04T18:51:20.795266] [DEBUG] [LOCAL ] FAF: Deleted primary data file. [2025-11-04T18:51:20.805901] [INFO ] [MAIN ] F: Starting full primary refresh. [2025-11-04T18:51:20.831295] [INFO ] [MAIN ] FAF: Starting full primary refresh. [2025-11-04T18:51:21.985612] [INFO ] [REMOTE ] FBIN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:51:22.064301] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-01. [2025-11-04T18:51:22.097196] [INFO ] [REMOTE ] EXR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:51:22.150635] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-01. [2025-11-04T18:51:22.238904] [INFO ] [REMOTE ] F: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:51:22.211740] [INFO ] [REMOTE ] FAF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:51:22.239589] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-01. [2025-11-04T18:51:22.239818] [INFO ] [REMOTE ] F: Fetching primary data for 2025-01. [2025-11-04T18:51:23.332745] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T18:51:23.342419] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-02. [2025-11-04T18:51:23.365219] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T18:51:23.372699] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-02. [2025-11-04T18:51:23.401216] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T18:51:23.411263] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-02. [2025-11-04T18:51:24.000450] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-11-04T18:51:24.010954] [INFO ] [REMOTE ] F: Fetching primary data for 2025-02. [2025-11-04T18:51:24.220079] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T18:51:24.227687] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-03. [2025-11-04T18:51:24.467941] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T18:51:24.671951] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T18:51:24.967051] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-03. [2025-11-04T18:51:25.162831] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-11-04T18:51:25.705232] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-11-04T18:51:25.767092] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-04. [2025-11-04T18:51:26.232209] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T18:51:26.567108] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-04. [2025-11-04T18:51:27.033091] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T18:51:27.367047] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-03. [2025-11-04T18:51:27.778437] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T18:51:28.167057] [INFO ] [REMOTE ] F: Fetching primary data for 2025-03. [2025-11-04T18:51:28.616941] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-11-04T18:51:28.967061] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-05. [2025-11-04T18:51:29.767076] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-04. [2025-11-04T18:51:30.048699] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-11-04T18:51:30.088315] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-11-04T18:51:30.567034] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-05. [2025-11-04T18:51:30.987177] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T18:51:31.367087] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-06. [2025-11-04T18:51:31.713244] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T18:51:32.167035] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-06. [2025-11-04T18:51:32.457575] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T18:51:32.967073] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-07. [2025-11-04T18:51:33.421335] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-11-04T18:51:33.767090] [INFO ] [REMOTE ] F: Fetching primary data for 2025-04. [2025-11-04T18:51:34.182862] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-11-04T18:51:34.567040] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-07. [2025-11-04T18:51:35.367065] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-05. [2025-11-04T18:51:35.897734] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T18:51:35.993725] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-11-04T18:51:36.167009] [INFO ] [REMOTE ] F: Fetching primary data for 2025-05. [2025-11-04T18:51:36.627117] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T18:51:36.967067] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-08. [2025-11-04T18:51:37.767071] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-06. [2025-11-04T18:51:38.016877] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-11-04T18:51:38.057581] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-11-04T18:51:38.567037] [INFO ] [REMOTE ] F: Fetching primary data for 2025-06. [2025-11-04T18:51:38.891254] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T18:51:39.367031] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-09. [2025-11-04T18:51:40.167046] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-08. [2025-11-04T18:51:40.314706] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-11-04T18:51:40.397909] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-11-04T18:51:40.967096] [INFO ] [REMOTE ] F: Fetching primary data for 2025-07. [2025-11-04T18:51:41.423401] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T18:51:41.767043] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-09. [2025-11-04T18:51:42.567072] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-10. [2025-11-04T18:51:42.854062] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-11-04T18:51:43.188029] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-07 (2.22 seconds). [2025-11-04T18:51:43.367023] [INFO ] [REMOTE ] F: Fetching primary data for 2025-08. [2025-11-04T18:51:43.764721] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T18:51:44.167055] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-10. [2025-11-04T18:51:44.967014] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-11. [2025-11-04T18:51:45.185663] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-11-04T18:51:45.349944] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T18:51:45.767052] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-07. [2025-11-04T18:51:45.946745] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-11 (0.98 seconds). [2025-11-04T18:51:45.948391] [INFO ] [REMOTE ] FAF: Full fetch complete (58508 records). [2025-11-04T18:51:45.948465] [DEBUG] [VALIDATOR ] FAF: Validating primary data. [2025-11-04T18:51:46.329292] [DEBUG] [VALIDATOR ] FAF: All expected trading days are present. [2025-11-04T18:51:46.329386] [DEBUG] [VALIDATOR ] FAF: Primary data validation passed. [2025-11-04T18:51:46.330115] [INFO ] [MAIN ] FAF: Fetched primary data is valid. [2025-11-04T18:51:46.330175] [DEBUG] [LOCAL ] FAF: Saving primary data. [2025-11-04T18:51:46.331353] [DEBUG] [LOCAL ] FAF: Trimmed 58508 records to 41120 (keeping 200 days of data) [2025-11-04T18:51:46.569092] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-11. [2025-11-04T18:51:46.572499] [DEBUG] [LOCAL ] FAF: Primary data saved (41120 records). [2025-11-04T18:51:46.573873] [INFO ] [MAIN ] FAF: Full history saved. [2025-11-04T18:51:46.573963] [INFO ] [MAIN ] Update Progress: 801/2367 symbols completed. FAF took 27.5s. Estimated remaining: 3h 38m 25s. Rate: 7.2 symbols/min [2025-11-04T18:51:46.743743] [INFO ] [MAIN ] FBK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:51:46.769439] [DEBUG] [LOCAL ] FBK: Loaded 26864 existing records from primary storage. [2025-11-04T18:51:46.769745] [INFO ] [MAIN ] FBK: Loaded 26864 existing records. [2025-11-04T18:51:46.956877] [WARN ] [MAIN ] FBK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:51:46.957178] [INFO ] [MAIN ] FBK: Wiping existing primary data before full refresh. [2025-11-04T18:51:46.957307] [INFO ] [LOCAL ] FBK: Starting deletion of primary data. [2025-11-04T18:51:46.957576] [DEBUG] [LOCAL ] FBK: Deleted primary data file. [2025-11-04T18:51:46.957614] [INFO ] [MAIN ] FBK: Starting full primary refresh. [2025-11-04T18:51:47.350485] [INFO ] [REMOTE ] FBK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:51:47.355200] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-11-04T18:51:47.366996] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-08. [2025-11-04T18:51:47.621406] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T18:51:47.623364] [INFO ] [REMOTE ] FBIN: Full fetch complete (72826 records). [2025-11-04T18:51:47.623460] [DEBUG] [VALIDATOR ] FBIN: Validating primary data. [2025-11-04T18:51:47.999759] [DEBUG] [VALIDATOR ] FBIN: All expected trading days are present. [2025-11-04T18:51:47.999838] [DEBUG] [VALIDATOR ] FBIN: Primary data validation passed. [2025-11-04T18:51:48.000667] [INFO ] [MAIN ] FBIN: Fetched primary data is valid. [2025-11-04T18:51:48.000733] [DEBUG] [LOCAL ] FBIN: Saving primary data. [2025-11-04T18:51:48.002115] [DEBUG] [LOCAL ] FBIN: Trimmed 72826 records to 50170 (keeping 200 days of data) [2025-11-04T18:51:48.172684] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-01. [2025-11-04T18:51:48.305977] [DEBUG] [LOCAL ] FBIN: Primary data saved (50170 records). [2025-11-04T18:51:48.306133] [INFO ] [MAIN ] FBIN: Full history saved. [2025-11-04T18:51:48.306231] [INFO ] [MAIN ] Update Progress: 802/2367 symbols completed. FBIN took 29.2s. Estimated remaining: 3h 38m 4s. Rate: 7.2 symbols/min [2025-11-04T18:51:48.475401] [INFO ] [MAIN ] FBP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:51:48.514876] [DEBUG] [LOCAL ] FBP: Loaded 45304 existing records from primary storage. [2025-11-04T18:51:48.515427] [INFO ] [MAIN ] FBP: Loaded 45304 existing records. [2025-11-04T18:51:48.693321] [WARN ] [MAIN ] FBP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:51:48.693681] [INFO ] [MAIN ] FBP: Wiping existing primary data before full refresh. [2025-11-04T18:51:48.693831] [INFO ] [LOCAL ] FBP: Starting deletion of primary data. [2025-11-04T18:51:48.694100] [DEBUG] [LOCAL ] FBP: Deleted primary data file. [2025-11-04T18:51:48.694161] [INFO ] [MAIN ] FBP: Starting full primary refresh. [2025-11-04T18:51:48.836036] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-11-04T18:51:48.972330] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-09. [2025-11-04T18:51:49.034850] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T18:51:49.117786] [INFO ] [REMOTE ] FBP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:51:49.767088] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-02. [2025-11-04T18:51:50.178089] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T18:51:50.462434] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T18:51:50.567010] [INFO ] [REMOTE ] F: Fetching primary data for 2025-09. [2025-11-04T18:51:51.367104] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-03. [2025-11-04T18:51:52.140044] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-11-04T18:51:52.167069] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-10. [2025-11-04T18:51:52.383811] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-11-04T18:51:52.967145] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-04. [2025-11-04T18:51:53.728884] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-11-04T18:51:53.767119] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-11. [2025-11-04T18:51:53.988724] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T18:51:54.567064] [INFO ] [REMOTE ] F: Fetching primary data for 2025-10. [2025-11-04T18:51:54.890131] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T18:51:54.892037] [INFO ] [REMOTE ] EXR: Full fetch complete (66120 records). [2025-11-04T18:51:54.892127] [DEBUG] [VALIDATOR ] EXR: Validating primary data. [2025-11-04T18:51:55.277258] [DEBUG] [VALIDATOR ] EXR: All expected trading days are present. [2025-11-04T18:51:55.277341] [DEBUG] [VALIDATOR ] EXR: Primary data validation passed. [2025-11-04T18:51:55.278143] [INFO ] [MAIN ] EXR: Fetched primary data is valid. [2025-11-04T18:51:55.278206] [DEBUG] [LOCAL ] EXR: Saving primary data. [2025-11-04T18:51:55.279482] [DEBUG] [LOCAL ] EXR: Trimmed 66120 records to 45396 (keeping 200 days of data) [2025-11-04T18:51:55.367565] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-05. [2025-11-04T18:51:55.560312] [DEBUG] [LOCAL ] EXR: Primary data saved (45396 records). [2025-11-04T18:51:55.560459] [INFO ] [MAIN ] EXR: Full history saved. [2025-11-04T18:51:55.560559] [INFO ] [MAIN ] Update Progress: 803/2367 symbols completed. EXR took 36.5s. Estimated remaining: 3h 37m 53s. Rate: 7.2 symbols/min [2025-11-04T18:51:55.727385] [INFO ] [MAIN ] FBRT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:51:55.759939] [DEBUG] [LOCAL ] FBRT: Loaded 34757 existing records from primary storage. [2025-11-04T18:51:55.760219] [INFO ] [MAIN ] FBRT: Loaded 34757 existing records. [2025-11-04T18:51:55.932557] [WARN ] [MAIN ] FBRT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:51:55.932803] [INFO ] [MAIN ] FBRT: Wiping existing primary data before full refresh. [2025-11-04T18:51:55.932850] [INFO ] [LOCAL ] FBRT: Starting deletion of primary data. [2025-11-04T18:51:55.933028] [DEBUG] [LOCAL ] FBRT: Deleted primary data file. [2025-11-04T18:51:55.933073] [INFO ] [MAIN ] FBRT: Starting full primary refresh. [2025-11-04T18:51:56.171437] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-01. [2025-11-04T18:51:56.295027] [INFO ] [REMOTE ] FBRT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:51:56.315848] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-11-04T18:51:56.965463] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-10 (2.40 seconds). [2025-11-04T18:51:56.977458] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-06. [2025-11-04T18:51:57.448055] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T18:51:57.612584] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-11-04T18:51:57.767107] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-07. [2025-11-04T18:51:58.567105] [INFO ] [REMOTE ] F: Fetching primary data for 2025-11. [2025-11-04T18:51:58.787079] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T18:51:59.367032] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-08. [2025-11-04T18:52:00.167069] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-01. [2025-11-04T18:52:00.320395] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T18:52:00.736356] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-11 (2.17 seconds). [2025-11-04T18:52:00.738445] [INFO ] [REMOTE ] F: Full fetch complete (78000 records). [2025-11-04T18:52:00.738535] [DEBUG] [VALIDATOR ] F: Validating primary data. [2025-11-04T18:52:00.972139] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-09. [2025-11-04T18:52:01.151244] [DEBUG] [VALIDATOR ] F: All expected trading days are present. [2025-11-04T18:52:01.151360] [DEBUG] [VALIDATOR ] F: Primary data validation passed. [2025-11-04T18:52:01.152319] [INFO ] [MAIN ] F: Fetched primary data is valid. [2025-11-04T18:52:01.152399] [DEBUG] [LOCAL ] F: Saving primary data. [2025-11-04T18:52:01.153852] [DEBUG] [LOCAL ] F: Trimmed 78000 records to 53820 (keeping 200 days of data) [2025-11-04T18:52:01.293189] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T18:52:01.510136] [DEBUG] [LOCAL ] F: Primary data saved (53820 records). [2025-11-04T18:52:01.510274] [INFO ] [MAIN ] F: Full history saved. [2025-11-04T18:52:01.510355] [INFO ] [MAIN ] Update Progress: 804/2367 symbols completed. F took 42.4s. Estimated remaining: 3h 37m 40s. Rate: 7.2 symbols/min [2025-11-04T18:52:01.677760] [INFO ] [MAIN ] FC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:52:01.691846] [DEBUG] [LOCAL ] FC: Loaded 13828 existing records from primary storage. [2025-11-04T18:52:01.692041] [INFO ] [MAIN ] FC: Loaded 13828 existing records. [2025-11-04T18:52:01.772161] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-02. [2025-11-04T18:52:01.872266] [WARN ] [MAIN ] FC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:52:01.872499] [INFO ] [MAIN ] FC: Wiping existing primary data before full refresh. [2025-11-04T18:52:01.872549] [INFO ] [LOCAL ] FC: Starting deletion of primary data. [2025-11-04T18:52:01.872718] [DEBUG] [LOCAL ] FC: Deleted primary data file. [2025-11-04T18:52:01.872784] [INFO ] [MAIN ] FC: Starting full primary refresh. [2025-11-04T18:52:02.148942] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:52:02.249005] [INFO ] [REMOTE ] FC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:52:02.567040] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-01. [2025-11-04T18:52:02.840944] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T18:52:03.139218] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-11-04T18:52:03.367051] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-02. [2025-11-04T18:52:04.167021] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-03. [2025-11-04T18:52:04.214288] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T18:52:04.967068] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-10. [2025-11-04T18:52:05.228477] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-11-04T18:52:05.767047] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-03. [2025-11-04T18:52:05.870040] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-11-04T18:52:06.567014] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-11. [2025-11-04T18:52:06.658859] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-11-04T18:52:07.343706] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T18:52:07.344880] [INFO ] [REMOTE ] FBK: Full fetch complete (35692 records). [2025-11-04T18:52:07.344952] [DEBUG] [VALIDATOR ] FBK: Validating primary data. [2025-11-04T18:52:07.375275] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-02. [2025-11-04T18:52:07.741884] [DEBUG] [VALIDATOR ] FBK: All expected trading days are present. [2025-11-04T18:52:07.741957] [DEBUG] [VALIDATOR ] FBK: Primary data validation passed. [2025-11-04T18:52:07.742499] [INFO ] [MAIN ] FBK: Fetched primary data is valid. [2025-11-04T18:52:07.742559] [DEBUG] [LOCAL ] FBK: Saving primary data. [2025-11-04T18:52:07.743343] [DEBUG] [LOCAL ] FBK: Trimmed 35692 records to 26795 (keeping 200 days of data) [2025-11-04T18:52:07.897608] [DEBUG] [LOCAL ] FBK: Primary data saved (26795 records). [2025-11-04T18:52:07.897758] [INFO ] [MAIN ] FBK: Full history saved. [2025-11-04T18:52:07.897840] [INFO ] [MAIN ] Update Progress: 805/2367 symbols completed. FBK took 21.3s. Estimated remaining: 3h 37m 28s. Rate: 7.2 symbols/min [2025-11-04T18:52:08.074380] [INFO ] [MAIN ] FCF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:52:08.110037] [DEBUG] [LOCAL ] FCF: Loaded 38443 existing records from primary storage. [2025-11-04T18:52:08.110300] [INFO ] [MAIN ] FCF: Loaded 38443 existing records. [2025-11-04T18:52:08.172155] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-04. [2025-11-04T18:52:08.291087] [WARN ] [MAIN ] FCF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:52:08.291321] [INFO ] [MAIN ] FCF: Wiping existing primary data before full refresh. [2025-11-04T18:52:08.291394] [INFO ] [LOCAL ] FCF: Starting deletion of primary data. [2025-11-04T18:52:08.299269] [DEBUG] [LOCAL ] FCF: Deleted primary data file. [2025-11-04T18:52:08.299858] [INFO ] [MAIN ] FCF: Starting full primary refresh. [2025-11-04T18:52:08.300155] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-11-04T18:52:08.674212] [INFO ] [REMOTE ] FCF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:52:08.967037] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-01. [2025-11-04T18:52:09.497546] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T18:52:09.767037] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-03. [2025-11-04T18:52:10.012678] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-11-04T18:52:10.465755] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T18:52:10.567038] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-02. [2025-11-04T18:52:11.367016] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-04. [2025-11-04T18:52:11.556862] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T18:52:12.167045] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-04. [2025-11-04T18:52:12.274196] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-11-04T18:52:12.933066] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-11-04T18:52:12.967001] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-05. [2025-11-04T18:52:13.767063] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-05. [2025-11-04T18:52:14.141056] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T18:52:14.318012] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T18:52:14.567041] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-05. [2025-11-04T18:52:15.367055] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-06. [2025-11-04T18:52:15.490471] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T18:52:15.783003] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-11-04T18:52:16.167066] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-06. [2025-11-04T18:52:16.967019] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-06. [2025-11-04T18:52:17.031955] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T18:52:17.767080] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-07. [2025-11-04T18:52:18.100562] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T18:52:18.464238] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T18:52:18.567009] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-08. [2025-11-04T18:52:19.226248] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-11-04T18:52:19.367044] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-03. [2025-11-04T18:52:20.167069] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-09. [2025-11-04T18:52:20.641458] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T18:52:20.809721] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-11-04T18:52:20.967066] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-04. [2025-11-04T18:52:21.767038] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-07. [2025-11-04T18:52:22.259904] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T18:52:22.567072] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-05. [2025-11-04T18:52:22.808417] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T18:52:23.367045] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-10. [2025-11-04T18:52:23.711138] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T18:52:23.842701] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-11-04T18:52:24.167040] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-08. [2025-11-04T18:52:24.967049] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-06. [2025-11-04T18:52:25.194538] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-11-04T18:52:25.767040] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-11. [2025-11-04T18:52:26.095371] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T18:52:26.381963] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T18:52:26.382937] [INFO ] [REMOTE ] FC: Full fetch complete (21552 records). [2025-11-04T18:52:26.383006] [DEBUG] [VALIDATOR ] FC: Validating primary data. [2025-11-04T18:52:26.576770] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-09. [2025-11-04T18:52:26.788064] [DEBUG] [VALIDATOR ] FC: All expected trading days are present. [2025-11-04T18:52:26.788168] [DEBUG] [VALIDATOR ] FC: Primary data validation passed. [2025-11-04T18:52:26.788947] [INFO ] [MAIN ] FC: Fetched primary data is valid. [2025-11-04T18:52:26.789014] [DEBUG] [LOCAL ] FC: Saving primary data. [2025-11-04T18:52:26.789723] [DEBUG] [LOCAL ] FC: Trimmed 21552 records to 13786 (keeping 200 days of data) [2025-11-04T18:52:26.882212] [DEBUG] [LOCAL ] FC: Primary data saved (13786 records). [2025-11-04T18:52:26.882355] [INFO ] [MAIN ] FC: Full history saved. [2025-11-04T18:52:26.882437] [INFO ] [MAIN ] Update Progress: 806/2367 symbols completed. FC took 25.4s. Estimated remaining: 3h 37m 40s. Rate: 7.2 symbols/min [2025-11-04T18:52:27.054141] [INFO ] [MAIN ] FCN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:52:27.079957] [DEBUG] [LOCAL ] FCN: Loaded 25939 existing records from primary storage. [2025-11-04T18:52:27.080203] [INFO ] [MAIN ] FCN: Loaded 25939 existing records. [2025-11-04T18:52:27.258227] [WARN ] [MAIN ] FCN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:52:27.258472] [INFO ] [MAIN ] FCN: Wiping existing primary data before full refresh. [2025-11-04T18:52:27.258519] [INFO ] [LOCAL ] FCN: Starting deletion of primary data. [2025-11-04T18:52:27.258773] [DEBUG] [LOCAL ] FCN: Deleted primary data file. [2025-11-04T18:52:27.258829] [INFO ] [MAIN ] FCN: Starting full primary refresh. [2025-11-04T18:52:27.368421] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-07. [2025-11-04T18:52:27.635761] [INFO ] [REMOTE ] FCN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:52:27.656294] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T18:52:28.167044] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-10. [2025-11-04T18:52:28.550999] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T18:52:28.967050] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-07. [2025-11-04T18:52:29.262915] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T18:52:29.767043] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-01. [2025-11-04T18:52:30.264510] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T18:52:30.508353] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-11-04T18:52:30.567002] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-02. [2025-11-04T18:52:31.367018] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-11. [2025-11-04T18:52:31.382154] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-11-04T18:52:32.167042] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-08. [2025-11-04T18:52:32.332168] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T18:52:32.333751] [INFO ] [REMOTE ] FBRT: Full fetch complete (49951 records). [2025-11-04T18:52:32.333826] [DEBUG] [VALIDATOR ] FBRT: Validating primary data. [2025-11-04T18:52:32.711762] [DEBUG] [VALIDATOR ] FBRT: All expected trading days are present. [2025-11-04T18:52:32.711840] [DEBUG] [VALIDATOR ] FBRT: Primary data validation passed. [2025-11-04T18:52:32.712550] [INFO ] [MAIN ] FBRT: Fetched primary data is valid. [2025-11-04T18:52:32.712599] [DEBUG] [LOCAL ] FBRT: Saving primary data. [2025-11-04T18:52:32.713803] [DEBUG] [LOCAL ] FBRT: Trimmed 49951 records to 34747 (keeping 200 days of data) [2025-11-04T18:52:32.918667] [DEBUG] [LOCAL ] FBRT: Primary data saved (34747 records). [2025-11-04T18:52:32.918805] [INFO ] [MAIN ] FBRT: Full history saved. [2025-11-04T18:52:32.918880] [INFO ] [MAIN ] Update Progress: 807/2367 symbols completed. FBRT took 37.4s. Estimated remaining: 3h 37m 28s. Rate: 7.2 symbols/min [2025-11-04T18:52:32.977967] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-03. [2025-11-04T18:52:33.099257] [INFO ] [MAIN ] FCPT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:52:33.134151] [DEBUG] [LOCAL ] FCPT: Loaded 36505 existing records from primary storage. [2025-11-04T18:52:33.134524] [INFO ] [MAIN ] FCPT: Loaded 36505 existing records. [2025-11-04T18:52:33.317828] [WARN ] [MAIN ] FCPT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:52:33.318176] [INFO ] [MAIN ] FCPT: Wiping existing primary data before full refresh. [2025-11-04T18:52:33.318340] [INFO ] [LOCAL ] FCPT: Starting deletion of primary data. [2025-11-04T18:52:33.318622] [DEBUG] [LOCAL ] FCPT: Deleted primary data file. [2025-11-04T18:52:33.318812] [INFO ] [MAIN ] FCPT: Starting full primary refresh. [2025-11-04T18:52:33.456569] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T18:52:33.743287] [INFO ] [REMOTE ] FCPT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:52:33.767002] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-01. [2025-11-04T18:52:33.872393] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-11-04T18:52:34.567076] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-04. [2025-11-04T18:52:34.589280] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-11-04T18:52:35.367106] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-02. [2025-11-04T18:52:35.720870] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T18:52:36.167026] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-05. [2025-11-04T18:52:36.294323] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-11-04T18:52:36.967000] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-08. [2025-11-04T18:52:37.065415] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T18:52:37.767009] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-09. [2025-11-04T18:52:37.996983] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-11-04T18:52:38.567087] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-03. [2025-11-04T18:52:39.004897] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-11-04T18:52:39.367099] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-06. [2025-11-04T18:52:39.697519] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-11-04T18:52:40.069629] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T18:52:40.167070] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-09. [2025-11-04T18:52:40.967047] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-07. [2025-11-04T18:52:41.216326] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T18:52:41.767007] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-10. [2025-11-04T18:52:41.895949] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-11-04T18:52:42.567053] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-04. [2025-11-04T18:52:42.839725] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T18:52:43.367028] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-11. [2025-11-04T18:52:43.696415] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T18:52:44.167089] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-05. [2025-11-04T18:52:44.339105] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T18:52:44.340811] [INFO ] [REMOTE ] FCF: Full fetch complete (57862 records). [2025-11-04T18:52:44.340888] [DEBUG] [VALIDATOR ] FCF: Validating primary data. [2025-11-04T18:52:44.721510] [DEBUG] [VALIDATOR ] FCF: All expected trading days are present. [2025-11-04T18:52:44.721581] [DEBUG] [VALIDATOR ] FCF: Primary data validation passed. [2025-11-04T18:52:44.722329] [INFO ] [MAIN ] FCF: Fetched primary data is valid. [2025-11-04T18:52:44.722398] [DEBUG] [LOCAL ] FCF: Saving primary data. [2025-11-04T18:52:44.723588] [DEBUG] [LOCAL ] FCF: Trimmed 57862 records to 38359 (keeping 200 days of data) [2025-11-04T18:52:44.946219] [DEBUG] [LOCAL ] FCF: Primary data saved (38359 records). [2025-11-04T18:52:44.946335] [INFO ] [MAIN ] FCF: Full history saved. [2025-11-04T18:52:44.946406] [INFO ] [MAIN ] Update Progress: 808/2367 symbols completed. FCF took 37.0s. Estimated remaining: 3h 37m 26s. Rate: 7.2 symbols/min [2025-11-04T18:52:44.972335] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-10. [2025-11-04T18:52:45.123966] [INFO ] [MAIN ] FCRX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:52:45.124360] [DEBUG] [LOCAL ] FCRX: No existing data file found in primary storage. [2025-11-04T18:52:45.124484] [WARN ] [MAIN ] FCRX: No existing data found, performing full refresh instead. [2025-11-04T18:52:45.124769] [INFO ] [MAIN ] FCRX: Wiping existing primary data before full refresh. [2025-11-04T18:52:45.124842] [INFO ] [LOCAL ] FCRX: Starting deletion of primary data. [2025-11-04T18:52:45.124956] [DEBUG] [LOCAL ] FCRX: No primary data file found to delete. [2025-11-04T18:52:45.125015] [INFO ] [MAIN ] FCRX: Starting full primary refresh. [2025-11-04T18:52:45.275133] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T18:52:45.515214] [INFO ] [REMOTE ] FCRX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:52:45.767032] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-01. [2025-11-04T18:52:46.111152] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-11-04T18:52:46.141063] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-11-04T18:52:46.567067] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-11. [2025-11-04T18:52:47.367035] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-08. [2025-11-04T18:52:47.545851] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-11 (0.98 seconds). [2025-11-04T18:52:47.547835] [INFO ] [REMOTE ] FBP: Full fetch complete (66845 records). [2025-11-04T18:52:47.547949] [DEBUG] [VALIDATOR ] FBP: Validating primary data. [2025-11-04T18:52:47.947908] [DEBUG] [VALIDATOR ] FBP: All expected trading days are present. [2025-11-04T18:52:47.948036] [DEBUG] [VALIDATOR ] FBP: Primary data validation passed. [2025-11-04T18:52:47.948871] [INFO ] [MAIN ] FBP: Fetched primary data is valid. [2025-11-04T18:52:47.948932] [DEBUG] [LOCAL ] FBP: Saving primary data. [2025-11-04T18:52:47.950176] [DEBUG] [LOCAL ] FBP: Trimmed 66845 records to 45277 (keeping 200 days of data) [2025-11-04T18:52:48.085391] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-11-04T18:52:48.176974] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-09. [2025-11-04T18:52:48.241484] [DEBUG] [LOCAL ] FBP: Primary data saved (45277 records). [2025-11-04T18:52:48.241622] [INFO ] [MAIN ] FBP: Full history saved. [2025-11-04T18:52:48.241742] [INFO ] [MAIN ] Update Progress: 809/2367 symbols completed. FBP took 59.9s. Estimated remaining: 3h 37m 8s. Rate: 7.2 symbols/min [2025-11-04T18:52:48.410957] [INFO ] [MAIN ] FCT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:52:48.429003] [DEBUG] [LOCAL ] FCT: Loaded 14546 existing records from primary storage. [2025-11-04T18:52:48.429260] [INFO ] [MAIN ] FCT: Loaded 14546 existing records. [2025-11-04T18:52:48.607750] [WARN ] [MAIN ] FCT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:52:48.607987] [INFO ] [MAIN ] FCT: Wiping existing primary data before full refresh. [2025-11-04T18:52:48.608035] [INFO ] [LOCAL ] FCT: Starting deletion of primary data. [2025-11-04T18:52:48.608186] [DEBUG] [LOCAL ] FCT: Deleted primary data file. [2025-11-04T18:52:48.608225] [INFO ] [MAIN ] FCT: Starting full primary refresh. [2025-11-04T18:52:48.968906] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-02. [2025-11-04T18:52:48.971726] [INFO ] [REMOTE ] FCT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:52:48.987250] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-11-04T18:52:49.264073] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-02 (0.30 seconds). [2025-11-04T18:52:49.767042] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-01. [2025-11-04T18:52:50.475306] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T18:52:50.567013] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-02. [2025-11-04T18:52:51.106050] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-11-04T18:52:51.367055] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-03. [2025-11-04T18:52:51.602834] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-03 (0.24 seconds). [2025-11-04T18:52:52.167080] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-04. [2025-11-04T18:52:52.498731] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-11-04T18:52:52.967088] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-10. [2025-11-04T18:52:53.767136] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-05. [2025-11-04T18:52:53.927610] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-11-04T18:52:54.071059] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-11-04T18:52:54.567124] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-06. [2025-11-04T18:52:54.807160] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-06 (0.24 seconds). [2025-11-04T18:52:55.367049] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-07. [2025-11-04T18:52:55.587450] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-07 (0.22 seconds). [2025-11-04T18:52:56.167051] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-08. [2025-11-04T18:52:56.423292] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-08 (0.26 seconds). [2025-11-04T18:52:56.967050] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-11. [2025-11-04T18:52:57.737597] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T18:52:57.738926] [INFO ] [REMOTE ] FCN: Full fetch complete (37181 records). [2025-11-04T18:52:57.739003] [DEBUG] [VALIDATOR ] FCN: Validating primary data. [2025-11-04T18:52:57.772119] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-06. [2025-11-04T18:52:58.135976] [DEBUG] [VALIDATOR ] FCN: All expected trading days are present. [2025-11-04T18:52:58.136062] [DEBUG] [VALIDATOR ] FCN: Primary data validation passed. [2025-11-04T18:52:58.136728] [INFO ] [MAIN ] FCN: Fetched primary data is valid. [2025-11-04T18:52:58.136801] [DEBUG] [LOCAL ] FCN: Saving primary data. [2025-11-04T18:52:58.137917] [DEBUG] [LOCAL ] FCN: Trimmed 37181 records to 25895 (keeping 200 days of data) [2025-11-04T18:52:58.299347] [DEBUG] [LOCAL ] FCN: Primary data saved (25895 records). [2025-11-04T18:52:58.299486] [INFO ] [MAIN ] FCN: Full history saved. [2025-11-04T18:52:58.299569] [INFO ] [MAIN ] Update Progress: 810/2367 symbols completed. FCN took 31.4s. Estimated remaining: 3h 37m 3s. Rate: 7.2 symbols/min [2025-11-04T18:52:58.477267] [INFO ] [MAIN ] FCX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:52:58.527220] [DEBUG] [LOCAL ] FCX: Loaded 53702 existing records from primary storage. [2025-11-04T18:52:58.527574] [INFO ] [MAIN ] FCX: Loaded 53702 existing records. [2025-11-04T18:52:58.572232] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-09. [2025-11-04T18:52:58.742922] [WARN ] [MAIN ] FCX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:52:58.743305] [INFO ] [MAIN ] FCX: Wiping existing primary data before full refresh. [2025-11-04T18:52:58.743399] [INFO ] [LOCAL ] FCX: Starting deletion of primary data. [2025-11-04T18:52:58.743725] [DEBUG] [LOCAL ] FCX: Deleted primary data file. [2025-11-04T18:52:58.743794] [INFO ] [MAIN ] FCX: Starting full primary refresh. [2025-11-04T18:52:58.867296] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-09 (0.30 seconds). [2025-11-04T18:52:58.937361] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T18:52:59.166997] [INFO ] [REMOTE ] FCX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:52:59.367042] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-01. [2025-11-04T18:53:00.167051] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-03. [2025-11-04T18:53:00.899036] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-11-04T18:53:00.967025] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-07. [2025-11-04T18:53:01.020489] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-11-04T18:53:01.767062] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-02. [2025-11-04T18:53:02.000053] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T18:53:02.567134] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-08. [2025-11-04T18:53:03.268548] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-11-04T18:53:03.367091] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-04. [2025-11-04T18:53:03.558764] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-11-04T18:53:04.109074] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-11-04T18:53:04.167053] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-03. [2025-11-04T18:53:04.967101] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-09. [2025-11-04T18:53:05.774852] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-05. [2025-11-04T18:53:05.780332] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-11-04T18:53:06.081665] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T18:53:06.366039] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T18:53:06.567040] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-04. [2025-11-04T18:53:07.367048] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-10. [2025-11-04T18:53:07.670149] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-10 (0.30 seconds). [2025-11-04T18:53:08.167020] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-11. [2025-11-04T18:53:08.248384] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-11-04T18:53:08.691093] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-11 (0.52 seconds). [2025-11-04T18:53:08.691841] [INFO ] [REMOTE ] FCRX: Full fetch complete (1590 records). [2025-11-04T18:53:08.691914] [DEBUG] [VALIDATOR ] FCRX: Validating primary data. [2025-11-04T18:53:08.870038] [WARN ] [VALIDATOR ] FCRX: Validation failed - insufficient historical data. Have 188 unique days, need 200 days [2025-11-04T18:53:08.870242] [FATAL] [MAIN ] FCRX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:53:08.870311] [WARN ] [MAIN ] FCRX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:53:08.870360] [INFO ] [LOCAL ] FCRX: Starting deletion of primary data. [2025-11-04T18:53:08.870413] [DEBUG] [LOCAL ] FCRX: No primary data file found to delete. [2025-11-04T18:53:08.870447] [INFO ] [LOCAL ] FCRX: Starting deletion of derivative data. [2025-11-04T18:53:08.870483] [DEBUG] [LOCAL ] FCRX: No derivative data file found to delete. [2025-11-04T18:53:08.870534] [INFO ] [MAIN ] Update Progress: 811/2367 symbols completed. FCRX took 23.9s. Estimated remaining: 3h 36m 59s. Rate: 7.2 symbols/min [2025-11-04T18:53:08.972182] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-05. [2025-11-04T18:53:09.042030] [INFO ] [MAIN ] FDP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:53:09.078869] [DEBUG] [LOCAL ] FDP: Loaded 29616 existing records from primary storage. [2025-11-04T18:53:09.079213] [INFO ] [MAIN ] FDP: Loaded 29616 existing records. [2025-11-04T18:53:09.257522] [WARN ] [MAIN ] FDP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:53:09.257787] [INFO ] [MAIN ] FDP: Wiping existing primary data before full refresh. [2025-11-04T18:53:09.257836] [INFO ] [LOCAL ] FDP: Starting deletion of primary data. [2025-11-04T18:53:09.258038] [DEBUG] [LOCAL ] FDP: Deleted primary data file. [2025-11-04T18:53:09.258093] [INFO ] [MAIN ] FDP: Starting full primary refresh. [2025-11-04T18:53:09.614514] [INFO ] [REMOTE ] FDP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:53:09.767064] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-01. [2025-11-04T18:53:10.564239] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-11-04T18:53:10.574620] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-06. [2025-11-04T18:53:10.648718] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T18:53:11.235636] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-11-04T18:53:11.367022] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-02. [2025-11-04T18:53:12.167050] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-10. [2025-11-04T18:53:12.286117] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-11-04T18:53:12.967045] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-07. [2025-11-04T18:53:13.295882] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-11-04T18:53:13.594857] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-11-04T18:53:13.767080] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-08. [2025-11-04T18:53:14.220143] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-11-04T18:53:14.567083] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-09. [2025-11-04T18:53:15.178949] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-11-04T18:53:15.367094] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-11. [2025-11-04T18:53:16.167092] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-10. [2025-11-04T18:53:16.448062] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-11 (1.08 seconds). [2025-11-04T18:53:16.449693] [INFO ] [REMOTE ] FCPT: Full fetch complete (53938 records). [2025-11-04T18:53:16.449775] [DEBUG] [VALIDATOR ] FCPT: Validating primary data. [2025-11-04T18:53:16.826584] [DEBUG] [VALIDATOR ] FCPT: All expected trading days are present. [2025-11-04T18:53:16.826691] [DEBUG] [VALIDATOR ] FCPT: Primary data validation passed. [2025-11-04T18:53:16.827431] [INFO ] [MAIN ] FCPT: Fetched primary data is valid. [2025-11-04T18:53:16.827476] [DEBUG] [LOCAL ] FCPT: Saving primary data. [2025-11-04T18:53:16.828522] [DEBUG] [LOCAL ] FCPT: Trimmed 53938 records to 36556 (keeping 200 days of data) [2025-11-04T18:53:16.993296] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-06. [2025-11-04T18:53:17.069979] [DEBUG] [LOCAL ] FCPT: Primary data saved (36556 records). [2025-11-04T18:53:17.072132] [INFO ] [MAIN ] FCPT: Full history saved. [2025-11-04T18:53:17.072280] [INFO ] [MAIN ] Update Progress: 812/2367 symbols completed. FCPT took 44.2s. Estimated remaining: 3h 36m 50s. Rate: 7.2 symbols/min [2025-11-04T18:53:17.072012] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-11-04T18:53:17.245212] [INFO ] [MAIN ] FDS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:53:17.273961] [DEBUG] [LOCAL ] FDS: Loaded 28933 existing records from primary storage. [2025-11-04T18:53:17.274217] [INFO ] [MAIN ] FDS: Loaded 28933 existing records. [2025-11-04T18:53:17.446903] [WARN ] [MAIN ] FDS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:53:17.447168] [INFO ] [MAIN ] FDS: Wiping existing primary data before full refresh. [2025-11-04T18:53:17.447214] [INFO ] [LOCAL ] FDS: Starting deletion of primary data. [2025-11-04T18:53:17.447817] [DEBUG] [LOCAL ] FDS: Deleted primary data file. [2025-11-04T18:53:17.447878] [INFO ] [MAIN ] FDS: Starting full primary refresh. [2025-11-04T18:53:17.777636] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-11. [2025-11-04T18:53:17.800236] [INFO ] [REMOTE ] FDS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:53:18.399314] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T18:53:18.400332] [INFO ] [REMOTE ] FCT: Full fetch complete (20807 records). [2025-11-04T18:53:18.400401] [DEBUG] [VALIDATOR ] FCT: Validating primary data. [2025-11-04T18:53:18.575632] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-01. [2025-11-04T18:53:18.823883] [DEBUG] [VALIDATOR ] FCT: All expected trading days are present. [2025-11-04T18:53:18.824015] [DEBUG] [VALIDATOR ] FCT: Primary data validation passed. [2025-11-04T18:53:18.824534] [INFO ] [MAIN ] FCT: Fetched primary data is valid. [2025-11-04T18:53:18.824736] [DEBUG] [LOCAL ] FCT: Saving primary data. [2025-11-04T18:53:18.825737] [DEBUG] [LOCAL ] FCT: Trimmed 20807 records to 14569 (keeping 200 days of data) [2025-11-04T18:53:18.932005] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-11-04T18:53:18.964123] [DEBUG] [LOCAL ] FCT: Primary data saved (14569 records). [2025-11-04T18:53:18.964430] [INFO ] [MAIN ] FCT: Full history saved. [2025-11-04T18:53:18.964971] [INFO ] [MAIN ] Update Progress: 813/2367 symbols completed. FCT took 30.7s. Estimated remaining: 3h 36m 30s. Rate: 7.2 symbols/min [2025-11-04T18:53:19.167553] [INFO ] [MAIN ] FDX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:53:19.216521] [DEBUG] [LOCAL ] FDX: Loaded 50932 existing records from primary storage. [2025-11-04T18:53:19.216886] [INFO ] [MAIN ] FDX: Loaded 50932 existing records. [2025-11-04T18:53:19.378622] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-03. [2025-11-04T18:53:19.420829] [WARN ] [MAIN ] FDX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:53:19.421085] [INFO ] [MAIN ] FDX: Wiping existing primary data before full refresh. [2025-11-04T18:53:19.421178] [INFO ] [LOCAL ] FDX: Starting deletion of primary data. [2025-11-04T18:53:19.421406] [DEBUG] [LOCAL ] FDX: Deleted primary data file. [2025-11-04T18:53:19.421466] [INFO ] [MAIN ] FDX: Starting full primary refresh. [2025-11-04T18:53:19.795639] [INFO ] [REMOTE ] FDX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:53:19.810897] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-11-04T18:53:20.167069] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-01. [2025-11-04T18:53:20.411103] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T18:53:20.967075] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-02. [2025-11-04T18:53:21.602675] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T18:53:21.730122] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T18:53:21.767026] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-02. [2025-11-04T18:53:22.567042] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-04. [2025-11-04T18:53:23.245079] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-11-04T18:53:23.367024] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-03. [2025-11-04T18:53:23.781823] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T18:53:24.167049] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-07. [2025-11-04T18:53:24.441088] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T18:53:24.967063] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-03. [2025-11-04T18:53:25.767090] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-04. [2025-11-04T18:53:25.913012] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-11-04T18:53:26.444110] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-11-04T18:53:26.475215] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-11-04T18:53:26.567052] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-08. [2025-11-04T18:53:27.367064] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-05. [2025-11-04T18:53:28.074554] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T18:53:28.167070] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-05. [2025-11-04T18:53:28.276588] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-11-04T18:53:28.967065] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-09. [2025-11-04T18:53:29.170017] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-11-04T18:53:29.767087] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-04. [2025-11-04T18:53:30.567079] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-06. [2025-11-04T18:53:30.821312] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-11-04T18:53:31.123057] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T18:53:31.367033] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-10. [2025-11-04T18:53:31.420943] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T18:53:32.167082] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-06. [2025-11-04T18:53:32.967070] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-07. [2025-11-04T18:53:32.988697] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-11-04T18:53:33.620265] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-10 (2.25 seconds). [2025-11-04T18:53:33.767018] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-07. [2025-11-04T18:53:33.868082] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T18:53:34.567060] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-05. [2025-11-04T18:53:34.613259] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T18:53:35.367063] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-08. [2025-11-04T18:53:36.167053] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-11. [2025-11-04T18:53:36.205524] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-11-04T18:53:36.317979] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T18:53:36.521205] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-11 (0.35 seconds). [2025-11-04T18:53:36.523815] [INFO ] [REMOTE ] FCX: Full fetch complete (77879 records). [2025-11-04T18:53:36.523934] [DEBUG] [VALIDATOR ] FCX: Validating primary data. [2025-11-04T18:53:36.938517] [DEBUG] [VALIDATOR ] FCX: All expected trading days are present. [2025-11-04T18:53:36.938628] [DEBUG] [VALIDATOR ] FCX: Primary data validation passed. [2025-11-04T18:53:36.940008] [INFO ] [MAIN ] FCX: Fetched primary data is valid. [2025-11-04T18:53:36.940075] [DEBUG] [LOCAL ] FCX: Saving primary data. [2025-11-04T18:53:36.942272] [DEBUG] [LOCAL ] FCX: Trimmed 77879 records to 53702 (keeping 200 days of data) [2025-11-04T18:53:36.998259] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-09. [2025-11-04T18:53:37.292626] [DEBUG] [LOCAL ] FCX: Primary data saved (53702 records). [2025-11-04T18:53:37.292787] [INFO ] [MAIN ] FCX: Full history saved. [2025-11-04T18:53:37.292868] [INFO ] [MAIN ] Update Progress: 814/2367 symbols completed. FCX took 39.0s. Estimated remaining: 3h 36m 40s. Rate: 7.2 symbols/min [2025-11-04T18:53:37.460548] [INFO ] [MAIN ] FE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:53:37.507935] [DEBUG] [LOCAL ] FE: Loaded 53437 existing records from primary storage. [2025-11-04T18:53:37.508253] [INFO ] [MAIN ] FE: Loaded 53437 existing records. [2025-11-04T18:53:37.682383] [WARN ] [MAIN ] FE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:53:37.682616] [INFO ] [MAIN ] FE: Wiping existing primary data before full refresh. [2025-11-04T18:53:37.682697] [INFO ] [LOCAL ] FE: Starting deletion of primary data. [2025-11-04T18:53:37.682934] [DEBUG] [LOCAL ] FE: Deleted primary data file. [2025-11-04T18:53:37.682990] [INFO ] [MAIN ] FE: Starting full primary refresh. [2025-11-04T18:53:37.772319] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-06. [2025-11-04T18:53:38.093539] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T18:53:38.126925] [INFO ] [REMOTE ] FE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:53:38.567070] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-01. [2025-11-04T18:53:39.185786] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-11-04T18:53:39.367089] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-07. [2025-11-04T18:53:39.997383] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T18:53:40.167069] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-10. [2025-11-04T18:53:40.901120] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T18:53:40.967003] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-08. [2025-11-04T18:53:41.138158] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T18:53:41.767009] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-11. [2025-11-04T18:53:41.833271] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-11-04T18:53:42.561817] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-11 (0.79 seconds). [2025-11-04T18:53:42.563281] [INFO ] [REMOTE ] FDP: Full fetch complete (43159 records). [2025-11-04T18:53:42.563375] [DEBUG] [VALIDATOR ] FDP: Validating primary data. [2025-11-04T18:53:42.575160] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-08. [2025-11-04T18:53:42.954113] [DEBUG] [VALIDATOR ] FDP: All expected trading days are present. [2025-11-04T18:53:42.954203] [DEBUG] [VALIDATOR ] FDP: Primary data validation passed. [2025-11-04T18:53:42.954960] [INFO ] [MAIN ] FDP: Fetched primary data is valid. [2025-11-04T18:53:42.955034] [DEBUG] [LOCAL ] FDP: Saving primary data. [2025-11-04T18:53:42.956135] [DEBUG] [LOCAL ] FDP: Trimmed 43159 records to 29574 (keeping 200 days of data) [2025-11-04T18:53:43.135689] [DEBUG] [LOCAL ] FDP: Primary data saved (29574 records). [2025-11-04T18:53:43.135840] [INFO ] [MAIN ] FDP: Full history saved. [2025-11-04T18:53:43.135925] [INFO ] [MAIN ] Update Progress: 815/2367 symbols completed. FDP took 34.3s. Estimated remaining: 3h 36m 27s. Rate: 7.2 symbols/min [2025-11-04T18:53:43.310381] [INFO ] [MAIN ] FENG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:53:43.310568] [DEBUG] [LOCAL ] FENG: No existing data file found in primary storage. [2025-11-04T18:53:43.310605] [WARN ] [MAIN ] FENG: No existing data found, performing full refresh instead. [2025-11-04T18:53:43.310739] [INFO ] [MAIN ] FENG: Wiping existing primary data before full refresh. [2025-11-04T18:53:43.310781] [INFO ] [LOCAL ] FENG: Starting deletion of primary data. [2025-11-04T18:53:43.310853] [DEBUG] [LOCAL ] FENG: No primary data file found to delete. [2025-11-04T18:53:43.310923] [INFO ] [MAIN ] FENG: Starting full primary refresh. [2025-11-04T18:53:43.376784] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-02. [2025-11-04T18:53:43.688997] [INFO ] [REMOTE ] FENG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:53:44.167071] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-09. [2025-11-04T18:53:44.231439] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-11-04T18:53:44.585857] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T18:53:44.967060] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-01. [2025-11-04T18:53:45.216307] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T18:53:45.226434] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-01 (0.26 seconds). [2025-11-04T18:53:45.767104] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-09. [2025-11-04T18:53:46.567073] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-03. [2025-11-04T18:53:47.380630] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-10. [2025-11-04T18:53:47.411820] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-11-04T18:53:47.966146] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T18:53:48.167139] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-02. [2025-11-04T18:53:48.497744] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-11-04T18:53:48.816080] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T18:53:48.967131] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-10. [2025-11-04T18:53:49.767051] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-11. [2025-11-04T18:53:50.412831] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T18:53:50.567062] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-04. [2025-11-04T18:53:50.912265] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T18:53:50.914278] [INFO ] [REMOTE ] FDS: Full fetch complete (40656 records). [2025-11-04T18:53:50.914407] [DEBUG] [VALIDATOR ] FDS: Validating primary data. [2025-11-04T18:53:51.283186] [DEBUG] [VALIDATOR ] FDS: All expected trading days are present. [2025-11-04T18:53:51.283260] [DEBUG] [VALIDATOR ] FDS: Primary data validation passed. [2025-11-04T18:53:51.283870] [INFO ] [MAIN ] FDS: Fetched primary data is valid. [2025-11-04T18:53:51.283931] [DEBUG] [LOCAL ] FDS: Saving primary data. [2025-11-04T18:53:51.284749] [DEBUG] [LOCAL ] FDS: Trimmed 40656 records to 29170 (keeping 200 days of data) [2025-11-04T18:53:51.375461] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-11. [2025-11-04T18:53:51.467525] [DEBUG] [LOCAL ] FDS: Primary data saved (29170 records). [2025-11-04T18:53:51.467686] [INFO ] [MAIN ] FDS: Full history saved. [2025-11-04T18:53:51.467786] [INFO ] [MAIN ] Update Progress: 816/2367 symbols completed. FDS took 34.4s. Estimated remaining: 3h 36m 19s. Rate: 7.2 symbols/min [2025-11-04T18:53:51.636924] [INFO ] [MAIN ] FERG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:53:51.682163] [DEBUG] [LOCAL ] FERG: Loaded 48384 existing records from primary storage. [2025-11-04T18:53:51.682456] [INFO ] [MAIN ] FERG: Loaded 48384 existing records. [2025-11-04T18:53:51.868210] [WARN ] [MAIN ] FERG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:53:51.868442] [INFO ] [MAIN ] FERG: Wiping existing primary data before full refresh. [2025-11-04T18:53:51.868623] [INFO ] [LOCAL ] FERG: Starting deletion of primary data. [2025-11-04T18:53:51.868988] [DEBUG] [LOCAL ] FERG: Deleted primary data file. [2025-11-04T18:53:51.869086] [INFO ] [MAIN ] FERG: Starting full primary refresh. [2025-11-04T18:53:52.171207] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-03. [2025-11-04T18:53:52.224088] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-11-04T18:53:52.297929] [INFO ] [REMOTE ] FERG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:53:52.486987] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-11-04T18:53:52.536505] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T18:53:52.538396] [INFO ] [REMOTE ] FDX: Full fetch complete (74051 records). [2025-11-04T18:53:52.538478] [DEBUG] [VALIDATOR ] FDX: Validating primary data. [2025-11-04T18:53:52.974887] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-05. [2025-11-04T18:53:52.975717] [DEBUG] [VALIDATOR ] FDX: All expected trading days are present. [2025-11-04T18:53:52.975874] [DEBUG] [VALIDATOR ] FDX: Primary data validation passed. [2025-11-04T18:53:52.978042] [INFO ] [MAIN ] FDX: Fetched primary data is valid. [2025-11-04T18:53:52.978226] [DEBUG] [LOCAL ] FDX: Saving primary data. [2025-11-04T18:53:52.979960] [DEBUG] [LOCAL ] FDX: Trimmed 74051 records to 50922 (keeping 200 days of data) [2025-11-04T18:53:53.317088] [DEBUG] [LOCAL ] FDX: Primary data saved (50922 records). [2025-11-04T18:53:53.317212] [INFO ] [MAIN ] FDX: Full history saved. [2025-11-04T18:53:53.317333] [INFO ] [MAIN ] Update Progress: 817/2367 symbols completed. FDX took 34.4s. Estimated remaining: 3h 35m 58s. Rate: 7.2 symbols/min [2025-11-04T18:53:53.481206] [INFO ] [MAIN ] FET: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:53:53.495383] [DEBUG] [LOCAL ] FET: Loaded 13172 existing records from primary storage. [2025-11-04T18:53:53.495590] [INFO ] [MAIN ] FET: Loaded 13172 existing records. [2025-11-04T18:53:53.667006] [WARN ] [MAIN ] FET: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:53:53.667236] [INFO ] [MAIN ] FET: Wiping existing primary data before full refresh. [2025-11-04T18:53:53.667282] [INFO ] [LOCAL ] FET: Starting deletion of primary data. [2025-11-04T18:53:53.667426] [DEBUG] [LOCAL ] FET: Deleted primary data file. [2025-11-04T18:53:53.667462] [INFO ] [MAIN ] FET: Starting full primary refresh. [2025-11-04T18:53:53.767755] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-01. [2025-11-04T18:53:54.025988] [INFO ] [REMOTE ] FET: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:53:54.314837] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T18:53:54.567047] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-04. [2025-11-04T18:53:54.962150] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-11-04T18:53:55.009292] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-11-04T18:53:55.367110] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-05. [2025-11-04T18:53:55.611295] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-11-04T18:53:56.167119] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-02. [2025-11-04T18:53:56.967087] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-06. [2025-11-04T18:53:57.336117] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-11-04T18:53:57.362953] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T18:53:57.767070] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-07. [2025-11-04T18:53:58.037114] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-07 (0.27 seconds). [2025-11-04T18:53:58.567055] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-01. [2025-11-04T18:53:59.109783] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-11-04T18:53:59.367081] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-06. [2025-11-04T18:54:00.167078] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-02. [2025-11-04T18:54:00.730150] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T18:54:00.805321] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T18:54:00.967131] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-07. [2025-11-04T18:54:01.767084] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-03. [2025-11-04T18:54:02.327463] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T18:54:02.567078] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-08. [2025-11-04T18:54:03.367041] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-03. [2025-11-04T18:54:03.411992] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-11-04T18:54:03.852100] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T18:54:04.057830] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-11-04T18:54:04.167010] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-08. [2025-11-04T18:54:04.414959] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-08 (0.25 seconds). [2025-11-04T18:54:04.967042] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-09. [2025-11-04T18:54:05.767094] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-09. [2025-11-04T18:54:06.115631] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-11-04T18:54:06.302030] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T18:54:06.567049] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-04. [2025-11-04T18:54:07.367078] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-10. [2025-11-04T18:54:07.978729] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-11-04T18:54:08.167084] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-04. [2025-11-04T18:54:08.692168] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-11-04T18:54:08.713694] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-11-04T18:54:08.967041] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-05. [2025-11-04T18:54:09.487041] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-11-04T18:54:09.767105] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-05. [2025-11-04T18:54:10.567007] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-06. [2025-11-04T18:54:11.122205] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-11-04T18:54:11.183056] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-11-04T18:54:11.367029] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-07. [2025-11-04T18:54:11.939780] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-11-04T18:54:12.167087] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-06. [2025-11-04T18:54:12.967090] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-11. [2025-11-04T18:54:13.378776] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T18:54:13.380936] [INFO ] [REMOTE ] FE: Full fetch complete (77509 records). [2025-11-04T18:54:13.381035] [DEBUG] [VALIDATOR ] FE: Validating primary data. [2025-11-04T18:54:13.681084] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-11-04T18:54:13.772129] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-08. [2025-11-04T18:54:13.818365] [DEBUG] [VALIDATOR ] FE: All expected trading days are present. [2025-11-04T18:54:13.818437] [DEBUG] [VALIDATOR ] FE: Primary data validation passed. [2025-11-04T18:54:13.819336] [INFO ] [MAIN ] FE: Fetched primary data is valid. [2025-11-04T18:54:13.819398] [DEBUG] [LOCAL ] FE: Saving primary data. [2025-11-04T18:54:13.820802] [DEBUG] [LOCAL ] FE: Trimmed 77509 records to 53435 (keeping 200 days of data) [2025-11-04T18:54:14.129005] [DEBUG] [LOCAL ] FE: Primary data saved (53435 records). [2025-11-04T18:54:14.129129] [INFO ] [MAIN ] FE: Full history saved. [2025-11-04T18:54:14.129204] [INFO ] [MAIN ] Update Progress: 818/2367 symbols completed. FE took 36.8s. Estimated remaining: 3h 36m 13s. Rate: 7.2 symbols/min [2025-11-04T18:54:14.293506] [INFO ] [MAIN ] FF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:54:14.313904] [DEBUG] [LOCAL ] FF: Loaded 22640 existing records from primary storage. [2025-11-04T18:54:14.314134] [INFO ] [MAIN ] FF: Loaded 22640 existing records. [2025-11-04T18:54:14.494156] [WARN ] [MAIN ] FF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:54:14.494388] [INFO ] [MAIN ] FF: Wiping existing primary data before full refresh. [2025-11-04T18:54:14.494469] [INFO ] [LOCAL ] FF: Starting deletion of primary data. [2025-11-04T18:54:14.494675] [DEBUG] [LOCAL ] FF: Deleted primary data file. [2025-11-04T18:54:14.494733] [INFO ] [MAIN ] FF: Starting full primary refresh. [2025-11-04T18:54:14.568667] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-07. [2025-11-04T18:54:14.605330] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-11-04T18:54:14.859782] [INFO ] [REMOTE ] FF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:54:15.367027] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-10. [2025-11-04T18:54:15.671435] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-10 (0.30 seconds). [2025-11-04T18:54:15.861175] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-11-04T18:54:16.167020] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-09. [2025-11-04T18:54:16.807957] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-11-04T18:54:16.967006] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-08. [2025-11-04T18:54:17.767051] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-01. [2025-11-04T18:54:18.064941] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-11-04T18:54:18.567040] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-11. [2025-11-04T18:54:18.626966] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T18:54:18.890001] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-11 (0.32 seconds). [2025-11-04T18:54:18.890630] [INFO ] [REMOTE ] FENG: Full fetch complete (2244 records). [2025-11-04T18:54:18.890724] [DEBUG] [VALIDATOR ] FENG: Validating primary data. [2025-11-04T18:54:19.071954] [WARN ] [VALIDATOR ] FENG: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-11-04T18:54:19.072159] [FATAL] [MAIN ] FENG: Fetched primary data is invalid. Refusing to save it. [2025-11-04T18:54:19.072229] [WARN ] [MAIN ] FENG: Deleting existing primary and derivative data due to validation failure. [2025-11-04T18:54:19.072298] [INFO ] [LOCAL ] FENG: Starting deletion of primary data. [2025-11-04T18:54:19.072364] [DEBUG] [LOCAL ] FENG: No primary data file found to delete. [2025-11-04T18:54:19.072401] [INFO ] [LOCAL ] FENG: Starting deletion of derivative data. [2025-11-04T18:54:19.072441] [DEBUG] [LOCAL ] FENG: No derivative data file found to delete. [2025-11-04T18:54:19.072495] [INFO ] [MAIN ] Update Progress: 819/2367 symbols completed. FENG took 35.9s. Estimated remaining: 3h 35m 58s. Rate: 7.2 symbols/min [2025-11-04T18:54:19.246972] [INFO ] [MAIN ] FFA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:54:19.254224] [DEBUG] [LOCAL ] FFA: Loaded 5192 existing records from primary storage. [2025-11-04T18:54:19.254404] [INFO ] [MAIN ] FFA: Loaded 5192 existing records. [2025-11-04T18:54:19.372152] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-09. [2025-11-04T18:54:19.432758] [WARN ] [MAIN ] FFA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:54:19.432960] [INFO ] [MAIN ] FFA: Wiping existing primary data before full refresh. [2025-11-04T18:54:19.433007] [INFO ] [LOCAL ] FFA: Starting deletion of primary data. [2025-11-04T18:54:19.433162] [DEBUG] [LOCAL ] FFA: Deleted primary data file. [2025-11-04T18:54:19.433217] [INFO ] [MAIN ] FFA: Starting full primary refresh. [2025-11-04T18:54:19.786001] [INFO ] [REMOTE ] FFA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:54:20.167035] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-02. [2025-11-04T18:54:20.692809] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-11-04T18:54:20.756369] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-11-04T18:54:20.967060] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-10. [2025-11-04T18:54:21.767052] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-03. [2025-11-04T18:54:22.396526] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T18:54:22.567037] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-10. [2025-11-04T18:54:22.808375] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T18:54:23.212526] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-11-04T18:54:23.367053] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-11. [2025-11-04T18:54:24.087283] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T18:54:24.089098] [INFO ] [REMOTE ] FET: Full fetch complete (19010 records). [2025-11-04T18:54:24.089201] [DEBUG] [VALIDATOR ] FET: Validating primary data. [2025-11-04T18:54:24.172521] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-04. [2025-11-04T18:54:24.512155] [DEBUG] [VALIDATOR ] FET: All expected trading days are present. [2025-11-04T18:54:24.512231] [DEBUG] [VALIDATOR ] FET: Primary data validation passed. [2025-11-04T18:54:24.512729] [INFO ] [MAIN ] FET: Fetched primary data is valid. [2025-11-04T18:54:24.512791] [DEBUG] [LOCAL ] FET: Saving primary data. [2025-11-04T18:54:24.513345] [DEBUG] [LOCAL ] FET: Trimmed 19010 records to 13256 (keeping 200 days of data) [2025-11-04T18:54:24.591044] [DEBUG] [LOCAL ] FET: Primary data saved (13256 records). [2025-11-04T18:54:24.591196] [INFO ] [MAIN ] FET: Full history saved. [2025-11-04T18:54:24.591285] [INFO ] [MAIN ] Update Progress: 820/2367 symbols completed. FET took 31.3s. Estimated remaining: 3h 35m 44s. Rate: 7.2 symbols/min [2025-11-04T18:54:24.762513] [INFO ] [MAIN ] FFC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:54:24.780028] [DEBUG] [LOCAL ] FFC: Loaded 16163 existing records from primary storage. [2025-11-04T18:54:24.780253] [INFO ] [MAIN ] FFC: Loaded 16163 existing records. [2025-11-04T18:54:24.957032] [WARN ] [MAIN ] FFC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:54:24.957306] [INFO ] [MAIN ] FFC: Wiping existing primary data before full refresh. [2025-11-04T18:54:24.957354] [INFO ] [LOCAL ] FFC: Starting deletion of primary data. [2025-11-04T18:54:24.957539] [DEBUG] [LOCAL ] FFC: Deleted primary data file. [2025-11-04T18:54:24.957608] [INFO ] [MAIN ] FFC: Starting full primary refresh. [2025-11-04T18:54:24.972185] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-11. [2025-11-04T18:54:25.344815] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-11-04T18:54:25.349368] [INFO ] [REMOTE ] FFC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:54:25.767049] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-01. [2025-11-04T18:54:26.158102] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T18:54:26.159919] [INFO ] [REMOTE ] FERG: Full fetch complete (70996 records). [2025-11-04T18:54:26.160025] [DEBUG] [VALIDATOR ] FERG: Validating primary data. [2025-11-04T18:54:26.560680] [DEBUG] [VALIDATOR ] FERG: All expected trading days are present. [2025-11-04T18:54:26.561353] [DEBUG] [VALIDATOR ] FERG: Primary data validation passed. [2025-11-04T18:54:26.562210] [INFO ] [MAIN ] FERG: Fetched primary data is valid. [2025-11-04T18:54:26.562288] [DEBUG] [LOCAL ] FERG: Saving primary data. [2025-11-04T18:54:26.561282] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-11-04T18:54:26.568724] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-02. [2025-11-04T18:54:26.570772] [DEBUG] [LOCAL ] FERG: Trimmed 70996 records to 48343 (keeping 200 days of data) [2025-11-04T18:54:26.886064] [DEBUG] [LOCAL ] FERG: Primary data saved (48343 records). [2025-11-04T18:54:26.886197] [INFO ] [MAIN ] FERG: Full history saved. [2025-11-04T18:54:26.886276] [INFO ] [MAIN ] Update Progress: 821/2367 symbols completed. FERG took 35.4s. Estimated remaining: 3h 35m 25s. Rate: 7.2 symbols/min [2025-11-04T18:54:27.055865] [INFO ] [MAIN ] FFWM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:54:27.090678] [DEBUG] [LOCAL ] FFWM: Loaded 35520 existing records from primary storage. [2025-11-04T18:54:27.091012] [INFO ] [MAIN ] FFWM: Loaded 35520 existing records. [2025-11-04T18:54:27.275205] [WARN ] [MAIN ] FFWM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:54:27.275545] [INFO ] [MAIN ] FFWM: Wiping existing primary data before full refresh. [2025-11-04T18:54:27.275771] [INFO ] [LOCAL ] FFWM: Starting deletion of primary data. [2025-11-04T18:54:27.275972] [DEBUG] [LOCAL ] FFWM: Deleted primary data file. [2025-11-04T18:54:27.276014] [INFO ] [MAIN ] FFWM: Starting full primary refresh. [2025-11-04T18:54:27.369034] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-05. [2025-11-04T18:54:27.397019] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T18:54:27.655360] [INFO ] [REMOTE ] FFWM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:54:28.172613] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-01. [2025-11-04T18:54:28.174696] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T18:54:28.699574] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T18:54:28.967072] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-03. [2025-11-04T18:54:29.571990] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-11-04T18:54:29.767040] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-04. [2025-11-04T18:54:30.508466] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-11-04T18:54:30.567003] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-05. [2025-11-04T18:54:31.166935] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-11-04T18:54:31.367054] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-06. [2025-11-04T18:54:32.099497] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-11-04T18:54:32.167006] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-07. [2025-11-04T18:54:32.967016] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-01. [2025-11-04T18:54:33.006900] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-11-04T18:54:33.767040] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-08. [2025-11-04T18:54:33.845293] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T18:54:34.580142] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-02. [2025-11-04T18:54:34.587875] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-11-04T18:54:35.367060] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-02. [2025-11-04T18:54:35.484285] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T18:54:35.675758] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-11-04T18:54:36.167068] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-03. [2025-11-04T18:54:36.605592] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-11-04T18:54:36.967082] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-04. [2025-11-04T18:54:37.496829] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T18:54:37.767021] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-03. [2025-11-04T18:54:38.567014] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-09. [2025-11-04T18:54:38.704197] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-11-04T18:54:39.303727] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-11-04T18:54:39.367020] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-05. [2025-11-04T18:54:39.824879] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-11-04T18:54:40.167047] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-06. [2025-11-04T18:54:40.617370] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-11-04T18:54:40.967060] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-04. [2025-11-04T18:54:41.767071] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-10. [2025-11-04T18:54:42.064942] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T18:54:42.478903] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-11-04T18:54:42.567061] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-11. [2025-11-04T18:54:43.232468] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T18:54:43.233599] [INFO ] [REMOTE ] FF: Full fetch complete (35919 records). [2025-11-04T18:54:43.233728] [DEBUG] [VALIDATOR ] FF: Validating primary data. [2025-11-04T18:54:43.374207] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-07. [2025-11-04T18:54:43.645738] [DEBUG] [VALIDATOR ] FF: All expected trading days are present. [2025-11-04T18:54:43.645825] [DEBUG] [VALIDATOR ] FF: Primary data validation passed. [2025-11-04T18:54:43.646424] [INFO ] [MAIN ] FF: Fetched primary data is valid. [2025-11-04T18:54:43.646464] [DEBUG] [LOCAL ] FF: Saving primary data. [2025-11-04T18:54:43.647275] [DEBUG] [LOCAL ] FF: Trimmed 35919 records to 22576 (keeping 200 days of data) [2025-11-04T18:54:43.781767] [DEBUG] [LOCAL ] FF: Primary data saved (22576 records). [2025-11-04T18:54:43.782458] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-11-04T18:54:43.783804] [INFO ] [MAIN ] FF: Full history saved. [2025-11-04T18:54:43.784271] [INFO ] [MAIN ] Update Progress: 822/2367 symbols completed. FF took 29.7s. Estimated remaining: 3h 35m 32s. Rate: 7.2 symbols/min [2025-11-04T18:54:43.955627] [INFO ] [MAIN ] FG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:54:43.977590] [DEBUG] [LOCAL ] FG: Loaded 20194 existing records from primary storage. [2025-11-04T18:54:43.977838] [INFO ] [MAIN ] FG: Loaded 20194 existing records. [2025-11-04T18:54:44.158330] [WARN ] [MAIN ] FG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:54:44.158567] [INFO ] [MAIN ] FG: Wiping existing primary data before full refresh. [2025-11-04T18:54:44.158627] [INFO ] [LOCAL ] FG: Starting deletion of primary data. [2025-11-04T18:54:44.158828] [DEBUG] [LOCAL ] FG: Deleted primary data file. [2025-11-04T18:54:44.158872] [INFO ] [MAIN ] FG: Starting full primary refresh. [2025-11-04T18:54:44.172191] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-08. [2025-11-04T18:54:44.528795] [INFO ] [REMOTE ] FG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:54:44.570299] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-11-04T18:54:44.967045] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-09. [2025-11-04T18:54:45.403356] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-11-04T18:54:45.767036] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-10. [2025-11-04T18:54:46.206930] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-11-04T18:54:46.567039] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-06. [2025-11-04T18:54:47.234891] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-11-04T18:54:47.367039] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-07. [2025-11-04T18:54:48.064797] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T18:54:48.167045] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-08. [2025-11-04T18:54:48.845367] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-11-04T18:54:48.967093] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-05. [2025-11-04T18:54:49.767065] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-01. [2025-11-04T18:54:49.974121] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-11-04T18:54:50.315825] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-11-04T18:54:50.567048] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-11. [2025-11-04T18:54:50.990099] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T18:54:50.990854] [INFO ] [REMOTE ] FFA: Full fetch complete (8645 records). [2025-11-04T18:54:50.990920] [DEBUG] [VALIDATOR ] FFA: Validating primary data. [2025-11-04T18:54:51.342238] [DEBUG] [VALIDATOR ] FFA: All expected trading days are present. [2025-11-04T18:54:51.342311] [DEBUG] [VALIDATOR ] FFA: Primary data validation passed. [2025-11-04T18:54:51.342520] [INFO ] [MAIN ] FFA: Fetched primary data is valid. [2025-11-04T18:54:51.342558] [DEBUG] [LOCAL ] FFA: Saving primary data. [2025-11-04T18:54:51.342964] [DEBUG] [LOCAL ] FFA: Trimmed 8645 records to 5166 (keeping 200 days of data) [2025-11-04T18:54:51.368489] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-09. [2025-11-04T18:54:51.373490] [DEBUG] [LOCAL ] FFA: Primary data saved (5166 records). [2025-11-04T18:54:51.374313] [INFO ] [MAIN ] FFA: Full history saved. [2025-11-04T18:54:51.374401] [INFO ] [MAIN ] Update Progress: 823/2367 symbols completed. FFA took 32.3s. Estimated remaining: 3h 35m 22s. Rate: 7.2 symbols/min [2025-11-04T18:54:51.549160] [INFO ] [MAIN ] FGN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:54:51.554671] [DEBUG] [LOCAL ] FGN: Loaded 3440 existing records from primary storage. [2025-11-04T18:54:51.554839] [INFO ] [MAIN ] FGN: Loaded 3440 existing records. [2025-11-04T18:54:51.734263] [WARN ] [MAIN ] FGN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:54:51.734487] [INFO ] [MAIN ] FGN: Wiping existing primary data before full refresh. [2025-11-04T18:54:51.734534] [INFO ] [LOCAL ] FGN: Starting deletion of primary data. [2025-11-04T18:54:51.734709] [DEBUG] [LOCAL ] FGN: Deleted primary data file. [2025-11-04T18:54:51.734780] [INFO ] [MAIN ] FGN: Starting full primary refresh. [2025-11-04T18:54:52.097944] [INFO ] [REMOTE ] FGN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:54:52.108234] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-11-04T18:54:52.167015] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-01. [2025-11-04T18:54:52.536028] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-01 (0.37 seconds). [2025-11-04T18:54:52.967066] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-06. [2025-11-04T18:54:53.767011] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-10. [2025-11-04T18:54:53.901754] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T18:54:54.554749] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T18:54:54.567000] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-11. [2025-11-04T18:54:55.109494] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T18:54:55.110503] [INFO ] [REMOTE ] FFC: Full fetch complete (24435 records). [2025-11-04T18:54:55.110579] [DEBUG] [VALIDATOR ] FFC: Validating primary data. [2025-11-04T18:54:55.372261] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-07. [2025-11-04T18:54:55.512850] [DEBUG] [VALIDATOR ] FFC: All expected trading days are present. [2025-11-04T18:54:55.512966] [DEBUG] [VALIDATOR ] FFC: Primary data validation passed. [2025-11-04T18:54:55.513480] [INFO ] [MAIN ] FFC: Fetched primary data is valid. [2025-11-04T18:54:55.513521] [DEBUG] [LOCAL ] FFC: Saving primary data. [2025-11-04T18:54:55.514179] [DEBUG] [LOCAL ] FFC: Trimmed 24435 records to 16176 (keeping 200 days of data) [2025-11-04T18:54:55.610998] [DEBUG] [LOCAL ] FFC: Primary data saved (16176 records). [2025-11-04T18:54:55.611136] [INFO ] [MAIN ] FFC: Full history saved. [2025-11-04T18:54:55.611215] [INFO ] [MAIN ] Update Progress: 824/2367 symbols completed. FFC took 31.0s. Estimated remaining: 3h 35m 6s. Rate: 7.2 symbols/min [2025-11-04T18:54:55.791038] [INFO ] [MAIN ] FGSN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:54:55.791224] [DEBUG] [LOCAL ] FGSN: No existing data file found in primary storage. [2025-11-04T18:54:55.791262] [WARN ] [MAIN ] FGSN: No existing data found, performing full refresh instead. [2025-11-04T18:54:55.791377] [INFO ] [MAIN ] FGSN: Wiping existing primary data before full refresh. [2025-11-04T18:54:55.791418] [INFO ] [LOCAL ] FGSN: Starting deletion of primary data. [2025-11-04T18:54:55.791458] [DEBUG] [LOCAL ] FGSN: No primary data file found to delete. [2025-11-04T18:54:55.791491] [INFO ] [MAIN ] FGSN: Starting full primary refresh. [2025-11-04T18:54:56.169275] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-02. [2025-11-04T18:54:56.169166] [INFO ] [REMOTE ] FGSN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:54:56.417801] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-11-04T18:54:56.714346] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T18:54:56.967062] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-02. [2025-11-04T18:54:57.262445] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-02 (0.30 seconds). [2025-11-04T18:54:57.767092] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-03. [2025-11-04T18:54:58.130523] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-11-04T18:54:58.567095] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-01. [2025-11-04T18:54:59.030548] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-11-04T18:54:59.367059] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-08. [2025-11-04T18:54:59.426610] [DEBUG] [REMOTE ] FFWM: Error fetching primary data for 2025-08: 'API call returned an empty response.' [2025-11-04T18:54:59.426851] [WARN ] [REMOTE ] FFWM: No data returned for month 2025-08 [2025-11-04T18:54:59.426933] [FATAL] [REMOTE ] FFWM: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:54:59.427025] [FATAL] [MAIN ] FFWM: Failed to fetch primary data. [2025-11-04T18:54:59.427089] [WARN ] [MAIN ] FFWM: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:54:59.427156] [INFO ] [LOCAL ] FFWM: Starting deletion of primary data. [2025-11-04T18:54:59.427215] [DEBUG] [LOCAL ] FFWM: No primary data file found to delete. [2025-11-04T18:54:59.427253] [INFO ] [LOCAL ] FFWM: Starting deletion of derivative data. [2025-11-04T18:54:59.427453] [DEBUG] [LOCAL ] FFWM: Deleted derivative data file. [2025-11-04T18:54:59.427533] [INFO ] [MAIN ] Update Progress: 825/2367 symbols completed. FFWM took 32.5s. Estimated remaining: 3h 34m 49s. Rate: 7.2 symbols/min [2025-11-04T18:54:59.594674] [INFO ] [MAIN ] FHI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:54:59.633728] [DEBUG] [LOCAL ] FHI: Loaded 39191 existing records from primary storage. [2025-11-04T18:54:59.634029] [INFO ] [MAIN ] FHI: Loaded 39191 existing records. [2025-11-04T18:54:59.809244] [WARN ] [MAIN ] FHI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:54:59.809489] [INFO ] [MAIN ] FHI: Wiping existing primary data before full refresh. [2025-11-04T18:54:59.809552] [INFO ] [LOCAL ] FHI: Starting deletion of primary data. [2025-11-04T18:54:59.809752] [DEBUG] [LOCAL ] FHI: Deleted primary data file. [2025-11-04T18:54:59.809814] [INFO ] [MAIN ] FHI: Starting full primary refresh. [2025-11-04T18:55:00.159610] [INFO ] [REMOTE ] FHI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:55:00.167031] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-01. [2025-11-04T18:55:00.967036] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-04. [2025-11-04T18:55:01.223025] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T18:55:01.293046] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-11-04T18:55:01.767074] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-03. [2025-11-04T18:55:02.565234] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-11-04T18:55:02.570677] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-05. [2025-11-04T18:55:02.918410] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-11-04T18:55:03.367034] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-06. [2025-11-04T18:55:03.809976] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-11-04T18:55:04.167039] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-02. [2025-11-04T18:55:04.967045] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-04. [2025-11-04T18:55:05.311394] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T18:55:05.767014] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-03. [2025-11-04T18:55:05.886451] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T18:55:06.567032] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-05. [2025-11-04T18:55:06.903840] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T18:55:07.343021] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-11-04T18:55:07.367008] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-06. [2025-11-04T18:55:07.967265] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-11-04T18:55:08.167038] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-02. [2025-11-04T18:55:08.552812] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-11-04T18:55:08.967130] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-03. [2025-11-04T18:55:09.403095] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-11-04T18:55:09.767055] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-07. [2025-11-04T18:55:10.503766] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T18:55:10.567061] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-08. [2025-11-04T18:55:11.266699] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T18:55:11.367087] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-04. [2025-11-04T18:55:11.742469] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T18:55:12.167062] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-04. [2025-11-04T18:55:12.967052] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-07. [2025-11-04T18:55:13.377209] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-11-04T18:55:13.380285] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T18:55:13.767076] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-09. [2025-11-04T18:55:14.558817] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T18:55:14.566987] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-10. [2025-11-04T18:55:15.219131] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-11-04T18:55:15.367005] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-05. [2025-11-04T18:55:16.167054] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-08. [2025-11-04T18:55:16.494777] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-11-04T18:55:16.577776] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T18:55:16.967017] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-09. [2025-11-04T18:55:17.319142] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-11-04T18:55:17.767038] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-06. [2025-11-04T18:55:18.567053] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-05. [2025-11-04T18:55:18.885790] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T18:55:18.930815] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-11-04T18:55:19.367035] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-07. [2025-11-04T18:55:20.167104] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-10. [2025-11-04T18:55:20.487916] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-10 (0.32 seconds). [2025-11-04T18:55:20.517208] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-11-04T18:55:20.967052] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-08. [2025-11-04T18:55:21.767044] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-11. [2025-11-04T18:55:21.987857] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-11-04T18:55:22.521818] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-11 (0.75 seconds). [2025-11-04T18:55:22.522957] [INFO ] [REMOTE ] FG: Full fetch complete (29183 records). [2025-11-04T18:55:22.523026] [DEBUG] [VALIDATOR ] FG: Validating primary data. [2025-11-04T18:55:22.575626] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-11. [2025-11-04T18:55:22.890871] [DEBUG] [VALIDATOR ] FG: All expected trading days are present. [2025-11-04T18:55:22.890944] [DEBUG] [VALIDATOR ] FG: Primary data validation passed. [2025-11-04T18:55:22.891481] [INFO ] [MAIN ] FG: Fetched primary data is valid. [2025-11-04T18:55:22.891519] [DEBUG] [LOCAL ] FG: Saving primary data. [2025-11-04T18:55:22.892211] [DEBUG] [LOCAL ] FG: Trimmed 29183 records to 20178 (keeping 200 days of data) [2025-11-04T18:55:23.012303] [DEBUG] [LOCAL ] FG: Primary data saved (20178 records). [2025-11-04T18:55:23.012399] [INFO ] [MAIN ] FG: Full history saved. [2025-11-04T18:55:23.012531] [INFO ] [MAIN ] Update Progress: 826/2367 symbols completed. FG took 39.2s. Estimated remaining: 3h 35m 10s. Rate: 7.2 symbols/min [2025-11-04T18:55:23.115741] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T18:55:23.127328] [INFO ] [REMOTE ] FGN: Full fetch complete (5374 records). [2025-11-04T18:55:23.127432] [DEBUG] [VALIDATOR ] FGN: Validating primary data. [2025-11-04T18:55:23.290470] [INFO ] [MAIN ] FHN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:55:23.368295] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-06. [2025-11-04T18:55:23.578581] [DEBUG] [VALIDATOR ] FGN: All expected trading days are present. [2025-11-04T18:55:23.578741] [DEBUG] [VALIDATOR ] FGN: Primary data validation passed. [2025-11-04T18:55:23.578931] [INFO ] [MAIN ] FGN: Fetched primary data is valid. [2025-11-04T18:55:23.579007] [DEBUG] [LOCAL ] FGN: Saving primary data. [2025-11-04T18:55:23.593782] [DEBUG] [LOCAL ] FGN: Trimmed 5374 records to 3447 (keeping 200 days of data) [2025-11-04T18:55:23.595817] [DEBUG] [LOCAL ] FHN: Loaded 53395 existing records from primary storage. [2025-11-04T18:55:23.604479] [INFO ] [MAIN ] FHN: Loaded 53395 existing records. [2025-11-04T18:55:23.719065] [DEBUG] [LOCAL ] FGN: Primary data saved (3447 records). [2025-11-04T18:55:23.724823] [INFO ] [MAIN ] FGN: Full history saved. [2025-11-04T18:55:23.730042] [INFO ] [MAIN ] Update Progress: 827/2367 symbols completed. FGN took 32.4s. Estimated remaining: 3h 34m 47s. Rate: 7.2 symbols/min [2025-11-04T18:55:23.840851] [WARN ] [MAIN ] FHN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:55:23.874266] [INFO ] [MAIN ] FHN: Wiping existing primary data before full refresh. [2025-11-04T18:55:23.902682] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-11-04T18:55:23.902743] [INFO ] [LOCAL ] FHN: Starting deletion of primary data. [2025-11-04T18:55:23.960540] [DEBUG] [LOCAL ] FHN: Deleted primary data file. [2025-11-04T18:55:23.982256] [INFO ] [MAIN ] FHN: Starting full primary refresh. [2025-11-04T18:55:24.015678] [INFO ] [MAIN ] FI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:55:24.178326] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-09. [2025-11-04T18:55:24.379340] [INFO ] [REMOTE ] FHN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:55:24.381303] [DEBUG] [LOCAL ] FI: Loaded 53592 existing records from primary storage. [2025-11-04T18:55:24.381531] [INFO ] [MAIN ] FI: Loaded 53592 existing records. [2025-11-04T18:55:24.559756] [WARN ] [MAIN ] FI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:55:24.559994] [INFO ] [MAIN ] FI: Wiping existing primary data before full refresh. [2025-11-04T18:55:24.560042] [INFO ] [LOCAL ] FI: Starting deletion of primary data. [2025-11-04T18:55:24.560413] [DEBUG] [LOCAL ] FI: Deleted primary data file. [2025-11-04T18:55:24.560450] [INFO ] [MAIN ] FI: Starting full primary refresh. [2025-11-04T18:55:24.910724] [INFO ] [REMOTE ] FI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:55:24.967002] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-01. [2025-11-04T18:55:25.165849] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T18:55:25.767034] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-07. [2025-11-04T18:55:26.264309] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-11-04T18:55:26.266035] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T18:55:26.567049] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-10. [2025-11-04T18:55:27.367079] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-02. [2025-11-04T18:55:27.600345] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-11-04T18:55:28.167070] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-11. [2025-11-04T18:55:28.722597] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T18:55:28.967035] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-01. [2025-11-04T18:55:29.239721] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T18:55:29.241385] [INFO ] [REMOTE ] FHI: Full fetch complete (57434 records). [2025-11-04T18:55:29.241467] [DEBUG] [VALIDATOR ] FHI: Validating primary data. [2025-11-04T18:55:29.641022] [DEBUG] [VALIDATOR ] FHI: All expected trading days are present. [2025-11-04T18:55:29.641118] [DEBUG] [VALIDATOR ] FHI: Primary data validation passed. [2025-11-04T18:55:29.641869] [INFO ] [MAIN ] FHI: Fetched primary data is valid. [2025-11-04T18:55:29.641995] [DEBUG] [LOCAL ] FHI: Saving primary data. [2025-11-04T18:55:29.643242] [DEBUG] [LOCAL ] FHI: Trimmed 57434 records to 39171 (keeping 200 days of data) [2025-11-04T18:55:29.811083] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-03. [2025-11-04T18:55:29.877220] [DEBUG] [LOCAL ] FHI: Primary data saved (39171 records). [2025-11-04T18:55:29.877337] [INFO ] [MAIN ] FHI: Full history saved. [2025-11-04T18:55:29.877414] [INFO ] [MAIN ] Update Progress: 828/2367 symbols completed. FHI took 30.4s. Estimated remaining: 3h 34m 34s. Rate: 7.2 symbols/min [2025-11-04T18:55:30.043549] [INFO ] [MAIN ] FICO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:55:30.066612] [DEBUG] [LOCAL ] FICO: Loaded 22233 existing records from primary storage. [2025-11-04T18:55:30.066915] [INFO ] [MAIN ] FICO: Loaded 22233 existing records. [2025-11-04T18:55:30.246156] [WARN ] [MAIN ] FICO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:55:30.246348] [INFO ] [MAIN ] FICO: Wiping existing primary data before full refresh. [2025-11-04T18:55:30.246395] [INFO ] [LOCAL ] FICO: Starting deletion of primary data. [2025-11-04T18:55:30.246743] [DEBUG] [LOCAL ] FICO: Deleted primary data file. [2025-11-04T18:55:30.246801] [INFO ] [MAIN ] FICO: Starting full primary refresh. [2025-11-04T18:55:30.576059] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-08. [2025-11-04T18:55:30.622571] [INFO ] [REMOTE ] FICO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:55:30.653561] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-11-04T18:55:30.980854] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-11-04T18:55:31.173702] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T18:55:31.367025] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-04. [2025-11-04T18:55:32.167049] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-09. [2025-11-04T18:55:32.561336] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-11-04T18:55:32.648787] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T18:55:32.967044] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-02. [2025-11-04T18:55:33.767090] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-10. [2025-11-04T18:55:34.109510] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-10 (0.34 seconds). [2025-11-04T18:55:34.156812] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T18:55:34.567091] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-05. [2025-11-04T18:55:35.367031] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-01. [2025-11-04T18:55:35.807931] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-11-04T18:55:36.167041] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-11. [2025-11-04T18:55:36.244342] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T18:55:36.967021] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-06. [2025-11-04T18:55:37.275223] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T18:55:37.276000] [INFO ] [REMOTE ] FGSN: Full fetch complete (8313 records). [2025-11-04T18:55:37.276072] [DEBUG] [VALIDATOR ] FGSN: Validating primary data. [2025-11-04T18:55:37.650202] [DEBUG] [VALIDATOR ] FGSN: All expected trading days are present. [2025-11-04T18:55:37.650282] [DEBUG] [VALIDATOR ] FGSN: Primary data validation passed. [2025-11-04T18:55:37.650487] [INFO ] [MAIN ] FGSN: Fetched primary data is valid. [2025-11-04T18:55:37.650527] [DEBUG] [LOCAL ] FGSN: Saving primary data. [2025-11-04T18:55:37.650997] [DEBUG] [LOCAL ] FGSN: Trimmed 8313 records to 4430 (keeping 200 days of data) [2025-11-04T18:55:37.677076] [DEBUG] [LOCAL ] FGSN: Primary data saved (4430 records). [2025-11-04T18:55:37.677137] [INFO ] [MAIN ] FGSN: Full history saved. [2025-11-04T18:55:37.677196] [INFO ] [MAIN ] Update Progress: 829/2367 symbols completed. FGSN took 42.1s. Estimated remaining: 3h 34m 25s. Rate: 7.2 symbols/min [2025-11-04T18:55:37.773699] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-03. [2025-11-04T18:55:37.847724] [INFO ] [MAIN ] FIG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:55:37.847867] [DEBUG] [LOCAL ] FIG: No existing data file found in primary storage. [2025-11-04T18:55:37.847903] [WARN ] [MAIN ] FIG: No existing data found, performing full refresh instead. [2025-11-04T18:55:37.848007] [INFO ] [MAIN ] FIG: Wiping existing primary data before full refresh. [2025-11-04T18:55:37.848047] [INFO ] [LOCAL ] FIG: Starting deletion of primary data. [2025-11-04T18:55:37.848085] [DEBUG] [LOCAL ] FIG: No primary data file found to delete. [2025-11-04T18:55:37.848116] [INFO ] [MAIN ] FIG: Starting full primary refresh. [2025-11-04T18:55:38.208677] [INFO ] [REMOTE ] FIG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:55:38.243312] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T18:55:38.567075] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-02. [2025-11-04T18:55:39.241599] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-11-04T18:55:39.367009] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-04. [2025-11-04T18:55:39.402934] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-11-04T18:55:40.167072] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-07. [2025-11-04T18:55:40.930778] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T18:55:40.967023] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-05. [2025-11-04T18:55:41.503427] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T18:55:41.767065] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-03. [2025-11-04T18:55:42.486063] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-11-04T18:55:42.506182] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T18:55:42.566998] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-04. [2025-11-04T18:55:43.292269] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T18:55:43.367001] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-05. [2025-11-04T18:55:44.167029] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-06. [2025-11-04T18:55:44.253192] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T18:55:44.967074] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-01. [2025-11-04T18:55:45.151104] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-01 (0.18 seconds). [2025-11-04T18:55:45.625304] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T18:55:45.767062] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-08. [2025-11-04T18:55:46.567041] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-07. [2025-11-04T18:55:47.016601] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T18:55:47.367097] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-06. [2025-11-04T18:55:48.010095] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-11-04T18:55:48.150602] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T18:55:48.167010] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-07. [2025-11-04T18:55:48.967052] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-02. [2025-11-04T18:55:49.165785] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-02 (0.20 seconds). [2025-11-04T18:55:49.220433] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-11-04T18:55:49.767049] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-09. [2025-11-04T18:55:50.567045] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-08. [2025-11-04T18:55:51.056946] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T18:55:51.367044] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-08. [2025-11-04T18:55:52.179091] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-10. [2025-11-04T18:55:52.203143] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-11-04T18:55:52.284725] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-11-04T18:55:52.967073] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-03. [2025-11-04T18:55:53.157719] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-03 (0.19 seconds). [2025-11-04T18:55:53.605327] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T18:55:53.767094] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-09. [2025-11-04T18:55:54.567102] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-09. [2025-11-04T18:55:55.119752] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T18:55:55.327781] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-11-04T18:55:55.367022] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-04. [2025-11-04T18:55:55.594562] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-04 (0.23 seconds). [2025-11-04T18:55:56.167037] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-11. [2025-11-04T18:55:56.967059] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-10. [2025-11-04T18:55:57.298118] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T18:55:57.300149] [INFO ] [REMOTE ] FHN: Full fetch complete (77483 records). [2025-11-04T18:55:57.300312] [DEBUG] [VALIDATOR ] FHN: Validating primary data. [2025-11-04T18:55:57.715482] [DEBUG] [VALIDATOR ] FHN: All expected trading days are present. [2025-11-04T18:55:57.715557] [DEBUG] [VALIDATOR ] FHN: Primary data validation passed. [2025-11-04T18:55:57.716443] [INFO ] [MAIN ] FHN: Fetched primary data is valid. [2025-11-04T18:55:57.716504] [DEBUG] [LOCAL ] FHN: Saving primary data. [2025-11-04T18:55:57.717961] [DEBUG] [LOCAL ] FHN: Trimmed 77483 records to 53395 (keeping 200 days of data) [2025-11-04T18:55:57.771301] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-05. [2025-11-04T18:55:58.029428] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-05 (0.26 seconds). [2025-11-04T18:55:58.053399] [DEBUG] [LOCAL ] FHN: Primary data saved (53395 records). [2025-11-04T18:55:58.053501] [INFO ] [MAIN ] FHN: Full history saved. [2025-11-04T18:55:58.053589] [INFO ] [MAIN ] Update Progress: 830/2367 symbols completed. FHN took 35.0s. Estimated remaining: 3h 34m 39s. Rate: 7.2 symbols/min [2025-11-04T18:55:58.230458] [INFO ] [MAIN ] FIGS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:55:58.272283] [DEBUG] [LOCAL ] FIGS: Loaded 48278 existing records from primary storage. [2025-11-04T18:55:58.272635] [INFO ] [MAIN ] FIGS: Loaded 48278 existing records. [2025-11-04T18:55:58.455250] [WARN ] [MAIN ] FIGS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:55:58.455487] [INFO ] [MAIN ] FIGS: Wiping existing primary data before full refresh. [2025-11-04T18:55:58.455539] [INFO ] [LOCAL ] FIGS: Starting deletion of primary data. [2025-11-04T18:55:58.455791] [DEBUG] [LOCAL ] FIGS: Deleted primary data file. [2025-11-04T18:55:58.455848] [INFO ] [MAIN ] FIGS: Starting full primary refresh. [2025-11-04T18:55:58.572133] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-06. [2025-11-04T18:55:58.795558] [DEBUG] [REMOTE ] FIG: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:55:58.837178] [INFO ] [REMOTE ] FIGS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:55:58.837500] [WARN ] [REMOTE ] FIG: No data returned for month 2025-06 [2025-11-04T18:55:58.838057] [FATAL] [REMOTE ] FIG: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:55:58.838186] [FATAL] [MAIN ] FIG: Failed to fetch primary data. [2025-11-04T18:55:58.838435] [WARN ] [MAIN ] FIG: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:55:58.838602] [INFO ] [LOCAL ] FIG: Starting deletion of primary data. [2025-11-04T18:55:58.838708] [DEBUG] [LOCAL ] FIG: No primary data file found to delete. [2025-11-04T18:55:58.838758] [INFO ] [LOCAL ] FIG: Starting deletion of derivative data. [2025-11-04T18:55:58.838807] [DEBUG] [LOCAL ] FIG: No derivative data file found to delete. [2025-11-04T18:55:58.838852] [INFO ] [MAIN ] Update Progress: 831/2367 symbols completed. FIG took 21.2s. Estimated remaining: 3h 34m 16s. Rate: 7.2 symbols/min [2025-11-04T18:55:59.020246] [INFO ] [MAIN ] FIHL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:55:59.055042] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-11-04T18:55:59.097367] [DEBUG] [LOCAL ] FIHL: Loaded 38381 existing records from primary storage. [2025-11-04T18:55:59.097678] [INFO ] [MAIN ] FIHL: Loaded 38381 existing records. [2025-11-04T18:55:59.283162] [WARN ] [MAIN ] FIHL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:55:59.283395] [INFO ] [MAIN ] FIHL: Wiping existing primary data before full refresh. [2025-11-04T18:55:59.283472] [INFO ] [LOCAL ] FIHL: Starting deletion of primary data. [2025-11-04T18:55:59.283719] [DEBUG] [LOCAL ] FIHL: Deleted primary data file. [2025-11-04T18:55:59.283785] [INFO ] [MAIN ] FIHL: Starting full primary refresh. [2025-11-04T18:55:59.372282] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-11. [2025-11-04T18:55:59.643710] [INFO ] [REMOTE ] FIHL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:00.167138] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-01. [2025-11-04T18:56:00.967072] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-01. [2025-11-04T18:56:00.996866] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-11 (1.62 seconds). [2025-11-04T18:56:00.999511] [INFO ] [REMOTE ] FI: Full fetch complete (77516 records). [2025-11-04T18:56:00.999634] [DEBUG] [VALIDATOR ] FI: Validating primary data. [2025-11-04T18:56:01.403587] [DEBUG] [VALIDATOR ] FI: All expected trading days are present. [2025-11-04T18:56:01.403749] [DEBUG] [VALIDATOR ] FI: Primary data validation passed. [2025-11-04T18:56:01.404757] [INFO ] [MAIN ] FI: Fetched primary data is valid. [2025-11-04T18:56:01.404867] [DEBUG] [LOCAL ] FI: Saving primary data. [2025-11-04T18:56:01.406453] [DEBUG] [LOCAL ] FI: Trimmed 77516 records to 53595 (keeping 200 days of data) [2025-11-04T18:56:01.648305] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-11-04T18:56:01.774680] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-02. [2025-11-04T18:56:01.779412] [DEBUG] [LOCAL ] FI: Primary data saved (53595 records). [2025-11-04T18:56:01.779558] [INFO ] [MAIN ] FI: Full history saved. [2025-11-04T18:56:01.779702] [INFO ] [MAIN ] Update Progress: 832/2367 symbols completed. FI took 38.0s. Estimated remaining: 3h 33m 58s. Rate: 7.2 symbols/min [2025-11-04T18:56:01.955920] [INFO ] [MAIN ] FINS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:56:01.977544] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T18:56:01.991626] [DEBUG] [LOCAL ] FINS: Loaded 8009 existing records from primary storage. [2025-11-04T18:56:01.991900] [INFO ] [MAIN ] FINS: Loaded 8009 existing records. [2025-11-04T18:56:02.167036] [WARN ] [MAIN ] FINS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:56:02.167249] [INFO ] [MAIN ] FINS: Wiping existing primary data before full refresh. [2025-11-04T18:56:02.167296] [INFO ] [LOCAL ] FINS: Starting deletion of primary data. [2025-11-04T18:56:02.167435] [DEBUG] [LOCAL ] FINS: Deleted primary data file. [2025-11-04T18:56:02.167472] [INFO ] [MAIN ] FINS: Starting full primary refresh. [2025-11-04T18:56:02.511582] [INFO ] [REMOTE ] FINS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:02.567000] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-01. [2025-11-04T18:56:02.896063] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-11-04T18:56:03.162991] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-11-04T18:56:03.367046] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-03. [2025-11-04T18:56:04.167011] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-02. [2025-11-04T18:56:04.531228] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-11-04T18:56:04.593747] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T18:56:04.967063] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-03. [2025-11-04T18:56:05.356750] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-11-04T18:56:05.767060] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-02. [2025-11-04T18:56:06.567006] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-10. [2025-11-04T18:56:06.728220] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T18:56:07.367082] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-04. [2025-11-04T18:56:07.715591] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-11-04T18:56:07.747214] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T18:56:08.167064] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-05. [2025-11-04T18:56:08.704317] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-11-04T18:56:08.967043] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-04. [2025-11-04T18:56:09.767046] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-11. [2025-11-04T18:56:10.355060] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T18:56:10.567081] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-06. [2025-11-04T18:56:10.675902] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T18:56:10.677619] [INFO ] [REMOTE ] FICO: Full fetch complete (30820 records). [2025-11-04T18:56:10.677718] [DEBUG] [VALIDATOR ] FICO: Validating primary data. [2025-11-04T18:56:11.039731] [DEBUG] [VALIDATOR ] FICO: All expected trading days are present. [2025-11-04T18:56:11.039896] [DEBUG] [VALIDATOR ] FICO: Primary data validation passed. [2025-11-04T18:56:11.040473] [INFO ] [MAIN ] FICO: Fetched primary data is valid. [2025-11-04T18:56:11.040547] [DEBUG] [LOCAL ] FICO: Saving primary data. [2025-11-04T18:56:11.047394] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-11-04T18:56:11.050590] [DEBUG] [LOCAL ] FICO: Trimmed 30820 records to 22525 (keeping 200 days of data) [2025-11-04T18:56:11.185035] [DEBUG] [LOCAL ] FICO: Primary data saved (22525 records). [2025-11-04T18:56:11.185159] [INFO ] [MAIN ] FICO: Full history saved. [2025-11-04T18:56:11.185234] [INFO ] [MAIN ] Update Progress: 833/2367 symbols completed. FICO took 41.3s. Estimated remaining: 3h 33m 52s. Rate: 7.2 symbols/min [2025-11-04T18:56:11.355124] [INFO ] [MAIN ] FINV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:56:11.376421] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-05. [2025-11-04T18:56:11.395402] [DEBUG] [LOCAL ] FINV: Loaded 45242 existing records from primary storage. [2025-11-04T18:56:11.395712] [INFO ] [MAIN ] FINV: Loaded 45242 existing records. [2025-11-04T18:56:11.569180] [WARN ] [MAIN ] FINV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:56:11.569386] [INFO ] [MAIN ] FINV: Wiping existing primary data before full refresh. [2025-11-04T18:56:11.569432] [INFO ] [LOCAL ] FINV: Starting deletion of primary data. [2025-11-04T18:56:11.569608] [DEBUG] [LOCAL ] FINV: Deleted primary data file. [2025-11-04T18:56:11.569706] [INFO ] [MAIN ] FINV: Starting full primary refresh. [2025-11-04T18:56:11.940050] [INFO ] [REMOTE ] FINV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:12.167050] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-07. [2025-11-04T18:56:12.468150] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-11-04T18:56:12.666356] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-11-04T18:56:12.967034] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-08. [2025-11-04T18:56:13.335517] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-11-04T18:56:13.767056] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-03. [2025-11-04T18:56:14.567140] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-06. [2025-11-04T18:56:15.060380] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-11-04T18:56:15.367079] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-01. [2025-11-04T18:56:15.777794] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T18:56:16.167071] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-09. [2025-11-04T18:56:16.511109] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T18:56:16.722367] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T18:56:16.967096] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-10. [2025-11-04T18:56:17.422569] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-11-04T18:56:17.767089] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-07. [2025-11-04T18:56:18.567037] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-02. [2025-11-04T18:56:18.930218] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-11-04T18:56:19.367009] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-08. [2025-11-04T18:56:19.572763] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-11-04T18:56:20.167056] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-11. [2025-11-04T18:56:20.565840] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T18:56:20.712408] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T18:56:20.713482] [INFO ] [REMOTE ] FINS: Full fetch complete (11486 records). [2025-11-04T18:56:20.713605] [DEBUG] [VALIDATOR ] FINS: Validating primary data. [2025-11-04T18:56:20.972124] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-03. [2025-11-04T18:56:21.082680] [DEBUG] [VALIDATOR ] FINS: All expected trading days are present. [2025-11-04T18:56:21.082767] [DEBUG] [VALIDATOR ] FINS: Primary data validation passed. [2025-11-04T18:56:21.083046] [INFO ] [MAIN ] FINS: Fetched primary data is valid. [2025-11-04T18:56:21.083084] [DEBUG] [LOCAL ] FINS: Saving primary data. [2025-11-04T18:56:21.083558] [DEBUG] [LOCAL ] FINS: Trimmed 11486 records to 8161 (keeping 200 days of data) [2025-11-04T18:56:21.138693] [DEBUG] [LOCAL ] FINS: Primary data saved (8161 records). [2025-11-04T18:56:21.138884] [INFO ] [MAIN ] FINS: Full history saved. [2025-11-04T18:56:21.139015] [INFO ] [MAIN ] Update Progress: 834/2367 symbols completed. FINS took 19.4s. Estimated remaining: 3h 33m 46s. Rate: 7.2 symbols/min [2025-11-04T18:56:21.362440] [INFO ] [MAIN ] FIS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:56:21.410475] [DEBUG] [LOCAL ] FIS: Loaded 53324 existing records from primary storage. [2025-11-04T18:56:21.410834] [INFO ] [MAIN ] FIS: Loaded 53324 existing records. [2025-11-04T18:56:21.600363] [WARN ] [MAIN ] FIS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:56:21.600587] [INFO ] [MAIN ] FIS: Wiping existing primary data before full refresh. [2025-11-04T18:56:21.600635] [INFO ] [LOCAL ] FIS: Starting deletion of primary data. [2025-11-04T18:56:21.600900] [DEBUG] [LOCAL ] FIS: Deleted primary data file. [2025-11-04T18:56:21.600957] [INFO ] [MAIN ] FIS: Starting full primary refresh. [2025-11-04T18:56:21.773393] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-09. [2025-11-04T18:56:21.976896] [INFO ] [REMOTE ] FIS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:22.256535] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T18:56:22.567074] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-01. [2025-11-04T18:56:23.051509] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T18:56:23.367008] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-10. [2025-11-04T18:56:23.856379] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T18:56:24.167042] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-02. [2025-11-04T18:56:24.564133] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T18:56:24.967043] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-04. [2025-11-04T18:56:25.352464] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-11-04T18:56:25.767044] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-03. [2025-11-04T18:56:26.140277] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-11-04T18:56:26.567033] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-05. [2025-11-04T18:56:27.042420] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T18:56:27.367049] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-04. [2025-11-04T18:56:27.579083] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-11-04T18:56:28.167044] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-04. [2025-11-04T18:56:28.680999] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T18:56:28.967039] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-11. [2025-11-04T18:56:29.219574] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-11-04T18:56:29.767054] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-06. [2025-11-04T18:56:29.952532] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T18:56:29.954388] [INFO ] [REMOTE ] FIGS: Full fetch complete (70595 records). [2025-11-04T18:56:29.954474] [DEBUG] [VALIDATOR ] FIGS: Validating primary data. [2025-11-04T18:56:30.335057] [DEBUG] [VALIDATOR ] FIGS: All expected trading days are present. [2025-11-04T18:56:30.335145] [DEBUG] [VALIDATOR ] FIGS: Primary data validation passed. [2025-11-04T18:56:30.336021] [INFO ] [MAIN ] FIGS: Fetched primary data is valid. [2025-11-04T18:56:30.336089] [DEBUG] [LOCAL ] FIGS: Saving primary data. [2025-11-04T18:56:30.337590] [DEBUG] [LOCAL ] FIGS: Trimmed 70595 records to 48324 (keeping 200 days of data) [2025-11-04T18:56:30.573709] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-05. [2025-11-04T18:56:30.603591] [DEBUG] [LOCAL ] FIGS: Primary data saved (48324 records). [2025-11-04T18:56:30.604016] [INFO ] [MAIN ] FIGS: Full history saved. [2025-11-04T18:56:30.604120] [INFO ] [MAIN ] Update Progress: 835/2367 symbols completed. FIGS took 32.6s. Estimated remaining: 3h 33m 40s. Rate: 7.2 symbols/min [2025-11-04T18:56:30.812064] [INFO ] [MAIN ] FIX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:56:30.833443] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T18:56:30.865203] [DEBUG] [LOCAL ] FIX: Loaded 28199 existing records from primary storage. [2025-11-04T18:56:30.865443] [INFO ] [MAIN ] FIX: Loaded 28199 existing records. [2025-11-04T18:56:31.048223] [WARN ] [MAIN ] FIX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:56:31.048432] [INFO ] [MAIN ] FIX: Wiping existing primary data before full refresh. [2025-11-04T18:56:31.048481] [INFO ] [LOCAL ] FIX: Starting deletion of primary data. [2025-11-04T18:56:31.048702] [DEBUG] [LOCAL ] FIX: Deleted primary data file. [2025-11-04T18:56:31.048757] [INFO ] [MAIN ] FIX: Starting full primary refresh. [2025-11-04T18:56:31.372502] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-05. [2025-11-04T18:56:31.405782] [INFO ] [REMOTE ] FIX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:31.851213] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T18:56:32.167070] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-01. [2025-11-04T18:56:32.455755] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-11-04T18:56:32.967046] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-06. [2025-11-04T18:56:33.281471] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-11-04T18:56:33.767070] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-02. [2025-11-04T18:56:34.208716] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T18:56:34.567033] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-07. [2025-11-04T18:56:34.834718] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T18:56:35.367073] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-07. [2025-11-04T18:56:35.946604] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-11-04T18:56:36.167056] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-03. [2025-11-04T18:56:36.383913] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T18:56:36.967039] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-08. [2025-11-04T18:56:37.325591] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-11-04T18:56:37.767044] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-04. [2025-11-04T18:56:38.267615] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T18:56:38.567038] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-08. [2025-11-04T18:56:38.918771] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T18:56:39.367052] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-09. [2025-11-04T18:56:39.647301] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-11-04T18:56:40.167057] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-06. [2025-11-04T18:56:40.655043] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T18:56:40.967048] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-09. [2025-11-04T18:56:41.211579] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T18:56:41.767078] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-10. [2025-11-04T18:56:41.938745] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-11-04T18:56:42.567085] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-05. [2025-11-04T18:56:43.201796] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T18:56:43.367060] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-07. [2025-11-04T18:56:43.430408] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-11-04T18:56:44.167060] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-06. [2025-11-04T18:56:44.630068] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T18:56:44.967029] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-11. [2025-11-04T18:56:45.052619] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T18:56:45.252180] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-11 (0.29 seconds). [2025-11-04T18:56:45.254227] [INFO ] [REMOTE ] FIS: Full fetch complete (77420 records). [2025-11-04T18:56:45.254338] [DEBUG] [VALIDATOR ] FIS: Validating primary data. [2025-11-04T18:56:45.637199] [DEBUG] [VALIDATOR ] FIS: All expected trading days are present. [2025-11-04T18:56:45.637280] [DEBUG] [VALIDATOR ] FIS: Primary data validation passed. [2025-11-04T18:56:45.638232] [INFO ] [MAIN ] FIS: Fetched primary data is valid. [2025-11-04T18:56:45.638299] [DEBUG] [LOCAL ] FIS: Saving primary data. [2025-11-04T18:56:45.639750] [DEBUG] [LOCAL ] FIS: Trimmed 77420 records to 53324 (keeping 200 days of data) [2025-11-04T18:56:45.817983] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-08. [2025-11-04T18:56:45.973811] [DEBUG] [LOCAL ] FIS: Primary data saved (53324 records). [2025-11-04T18:56:45.973961] [INFO ] [MAIN ] FIS: Full history saved. [2025-11-04T18:56:45.974045] [INFO ] [MAIN ] Update Progress: 836/2367 symbols completed. FIS took 24.8s. Estimated remaining: 3h 33m 44s. Rate: 7.2 symbols/min [2025-11-04T18:56:46.152953] [INFO ] [MAIN ] FIXT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:56:46.153143] [DEBUG] [LOCAL ] FIXT: No existing data file found in primary storage. [2025-11-04T18:56:46.153181] [WARN ] [MAIN ] FIXT: No existing data found, performing full refresh instead. [2025-11-04T18:56:46.153303] [INFO ] [MAIN ] FIXT: Wiping existing primary data before full refresh. [2025-11-04T18:56:46.153357] [INFO ] [LOCAL ] FIXT: Starting deletion of primary data. [2025-11-04T18:56:46.153401] [DEBUG] [LOCAL ] FIXT: No primary data file found to delete. [2025-11-04T18:56:46.153434] [INFO ] [MAIN ] FIXT: Starting full primary refresh. [2025-11-04T18:56:46.558053] [INFO ] [REMOTE ] FIXT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:46.567006] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-01. [2025-11-04T18:56:46.729592] [DEBUG] [REMOTE ] FIXT: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T18:56:46.729975] [WARN ] [REMOTE ] FIXT: No data returned for month 2025-01 [2025-11-04T18:56:46.730064] [FATAL] [REMOTE ] FIXT: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T18:56:46.730109] [FATAL] [MAIN ] FIXT: Failed to fetch primary data. [2025-11-04T18:56:46.730169] [WARN ] [MAIN ] FIXT: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T18:56:46.730240] [INFO ] [LOCAL ] FIXT: Starting deletion of primary data. [2025-11-04T18:56:46.730302] [DEBUG] [LOCAL ] FIXT: No primary data file found to delete. [2025-11-04T18:56:46.730337] [INFO ] [LOCAL ] FIXT: Starting deletion of derivative data. [2025-11-04T18:56:46.730376] [DEBUG] [LOCAL ] FIXT: No derivative data file found to delete. [2025-11-04T18:56:46.730421] [INFO ] [MAIN ] Update Progress: 837/2367 symbols completed. FIXT took 0.8s. Estimated remaining: 3h 33m 22s. Rate: 7.2 symbols/min [2025-11-04T18:56:46.990203] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T18:56:47.367089] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-10. [2025-11-04T18:56:48.167116] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-09. [2025-11-04T18:56:48.570306] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T18:56:48.967056] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-11. [2025-11-04T18:56:49.417882] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T18:56:49.767014] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-10. [2025-11-04T18:56:49.798378] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T18:56:49.800016] [INFO ] [REMOTE ] FIHL: Full fetch complete (56522 records). [2025-11-04T18:56:49.800096] [DEBUG] [VALIDATOR ] FIHL: Validating primary data. [2025-11-04T18:56:50.183600] [DEBUG] [VALIDATOR ] FIHL: All expected trading days are present. [2025-11-04T18:56:50.183704] [DEBUG] [VALIDATOR ] FIHL: Primary data validation passed. [2025-11-04T18:56:50.184427] [INFO ] [MAIN ] FIHL: Fetched primary data is valid. [2025-11-04T18:56:50.184470] [DEBUG] [LOCAL ] FIHL: Saving primary data. [2025-11-04T18:56:50.185569] [DEBUG] [LOCAL ] FIHL: Trimmed 56522 records to 38332 (keeping 200 days of data) [2025-11-04T18:56:50.408583] [DEBUG] [LOCAL ] FIHL: Primary data saved (38332 records). [2025-11-04T18:56:50.408869] [INFO ] [MAIN ] FIHL: Full history saved. [2025-11-04T18:56:50.408955] [INFO ] [MAIN ] Update Progress: 838/2367 symbols completed. FIHL took 51.6s. Estimated remaining: 3h 33m 5s. Rate: 7.2 symbols/min [2025-11-04T18:56:50.567015] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-07. [2025-11-04T18:56:50.899114] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-11-04T18:56:51.367013] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-11. [2025-11-04T18:56:51.417010] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T18:56:52.167042] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-08. [2025-11-04T18:56:52.387530] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T18:56:52.389721] [INFO ] [REMOTE ] FINV: Full fetch complete (65450 records). [2025-11-04T18:56:52.389823] [DEBUG] [VALIDATOR ] FINV: Validating primary data. [2025-11-04T18:56:52.783351] [DEBUG] [VALIDATOR ] FINV: All expected trading days are present. [2025-11-04T18:56:52.783447] [DEBUG] [VALIDATOR ] FINV: Primary data validation passed. [2025-11-04T18:56:52.784242] [INFO ] [MAIN ] FINV: Fetched primary data is valid. [2025-11-04T18:56:52.784306] [DEBUG] [LOCAL ] FINV: Saving primary data. [2025-11-04T18:56:52.785575] [DEBUG] [LOCAL ] FINV: Trimmed 65450 records to 45196 (keeping 200 days of data) [2025-11-04T18:56:53.042916] [DEBUG] [LOCAL ] FINV: Primary data saved (45196 records). [2025-11-04T18:56:53.043164] [INFO ] [MAIN ] FINV: Full history saved. [2025-11-04T18:56:53.043333] [INFO ] [MAIN ] Update Progress: 839/2367 symbols completed. FINV took 41.9s. Estimated remaining: 3h 32m 46s. Rate: 7.2 symbols/min [2025-11-04T18:56:53.059456] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T18:56:53.065512] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-09. [2025-11-04T18:56:53.902440] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-11-04T18:56:53.908745] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-10. [2025-11-04T18:56:54.979525] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T18:56:54.991018] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-11. [2025-11-04T18:56:55.987172] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T18:56:55.988810] [INFO ] [REMOTE ] FIX: Full fetch complete (45738 records). [2025-11-04T18:56:55.988884] [DEBUG] [VALIDATOR ] FIX: Validating primary data. [2025-11-04T18:56:56.373860] [DEBUG] [VALIDATOR ] FIX: All expected trading days are present. [2025-11-04T18:56:56.373952] [DEBUG] [VALIDATOR ] FIX: Primary data validation passed. [2025-11-04T18:56:56.374584] [INFO ] [MAIN ] FIX: Fetched primary data is valid. [2025-11-04T18:56:56.374625] [DEBUG] [LOCAL ] FIX: Saving primary data. [2025-11-04T18:56:56.375750] [DEBUG] [LOCAL ] FIX: Trimmed 45738 records to 28318 (keeping 200 days of data) [2025-11-04T18:56:56.548288] [DEBUG] [LOCAL ] FIX: Primary data saved (28318 records). [2025-11-04T18:56:56.548427] [INFO ] [MAIN ] FIX: Full history saved. [2025-11-04T18:56:56.548553] [INFO ] [MAIN ] Update Progress: 840/2367 symbols completed. FIX took 25.9s. Estimated remaining: 3h 32m 29s. Rate: 7.2 symbols/min [2025-11-04T18:56:57.155349] [INFO ] [MAIN ] FLC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:56:57.258083] [INFO ] [MAIN ] FLG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:56:57.336774] [INFO ] [MAIN ] FLNG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:56:57.371178] [INFO ] [MAIN ] FLO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:56:57.374036] [DEBUG] [LOCAL ] FLC: Loaded 4998 existing records from primary storage. [2025-11-04T18:56:57.374989] [INFO ] [MAIN ] FLC: Loaded 4998 existing records. [2025-11-04T18:56:57.576265] [WARN ] [MAIN ] FLC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:56:57.600164] [INFO ] [MAIN ] FLC: Wiping existing primary data before full refresh. [2025-11-04T18:56:57.600425] [INFO ] [LOCAL ] FLC: Starting deletion of primary data. [2025-11-04T18:56:57.601626] [DEBUG] [LOCAL ] FLC: Deleted primary data file. [2025-11-04T18:56:57.601519] [DEBUG] [LOCAL ] FLNG: Loaded 32841 existing records from primary storage. [2025-11-04T18:56:57.601894] [INFO ] [MAIN ] FLC: Starting full primary refresh. [2025-11-04T18:56:57.614000] [INFO ] [MAIN ] FLNG: Loaded 32841 existing records. [2025-11-04T18:56:57.841131] [DEBUG] [LOCAL ] FLG: Loaded 53298 existing records from primary storage. [2025-11-04T18:56:57.889165] [DEBUG] [LOCAL ] FLO: Loaded 52171 existing records from primary storage. [2025-11-04T18:56:57.935930] [INFO ] [MAIN ] FLG: Loaded 53298 existing records. [2025-11-04T18:56:57.936073] [INFO ] [MAIN ] FLO: Loaded 52171 existing records. [2025-11-04T18:56:58.235837] [WARN ] [MAIN ] FLNG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:56:58.317145] [INFO ] [MAIN ] FLNG: Wiping existing primary data before full refresh. [2025-11-04T18:56:58.360492] [INFO ] [LOCAL ] FLNG: Starting deletion of primary data. [2025-11-04T18:56:58.418469] [DEBUG] [LOCAL ] FLNG: Deleted primary data file. [2025-11-04T18:56:58.418600] [INFO ] [MAIN ] FLNG: Starting full primary refresh. [2025-11-04T18:56:58.578502] [WARN ] [MAIN ] FLG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:56:58.681637] [INFO ] [MAIN ] FLG: Wiping existing primary data before full refresh. [2025-11-04T18:56:58.695989] [WARN ] [MAIN ] FLO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:56:58.729406] [INFO ] [REMOTE ] FLC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:58.734921] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-01. [2025-11-04T18:56:58.729498] [INFO ] [LOCAL ] FLG: Starting deletion of primary data. [2025-11-04T18:56:58.741728] [DEBUG] [LOCAL ] FLG: Deleted primary data file. [2025-11-04T18:56:58.741825] [INFO ] [MAIN ] FLG: Starting full primary refresh. [2025-11-04T18:56:58.752697] [INFO ] [MAIN ] FLO: Wiping existing primary data before full refresh. [2025-11-04T18:56:58.774086] [INFO ] [LOCAL ] FLO: Starting deletion of primary data. [2025-11-04T18:56:58.818683] [DEBUG] [LOCAL ] FLO: Deleted primary data file. [2025-11-04T18:56:58.823961] [INFO ] [MAIN ] FLO: Starting full primary refresh. [2025-11-04T18:56:59.269381] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T18:56:59.367718] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-02. [2025-11-04T18:56:59.622840] [INFO ] [REMOTE ] FLNG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:59.698480] [INFO ] [REMOTE ] FLG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:59.715337] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-01. [2025-11-04T18:56:59.715860] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-01. [2025-11-04T18:56:59.818130] [INFO ] [REMOTE ] FLO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:56:59.818473] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-01. [2025-11-04T18:56:59.881677] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T18:56:59.883839] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-03. [2025-11-04T18:57:00.350638] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-11-04T18:57:00.352600] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-04. [2025-11-04T18:57:00.793320] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T18:57:00.891158] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-11-04T18:57:00.967058] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-02. [2025-11-04T18:57:01.045624] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T18:57:01.154515] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T18:57:01.767104] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-02. [2025-11-04T18:57:02.149378] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T18:57:02.567084] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-02. [2025-11-04T18:57:02.928013] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T18:57:03.367055] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-03. [2025-11-04T18:57:03.782879] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T18:57:04.167032] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-05. [2025-11-04T18:57:04.414601] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-11-04T18:57:04.548281] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-11-04T18:57:04.967054] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-06. [2025-11-04T18:57:05.377094] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-11-04T18:57:05.767049] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-04. [2025-11-04T18:57:06.567048] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-03. [2025-11-04T18:57:06.701632] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-11-04T18:57:07.367133] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-05. [2025-11-04T18:57:07.837419] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T18:57:08.167058] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-03. [2025-11-04T18:57:08.370276] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-11-04T18:57:08.967049] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-07. [2025-11-04T18:57:09.382330] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T18:57:09.412829] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-11-04T18:57:09.767037] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-08. [2025-11-04T18:57:10.224973] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-11-04T18:57:10.567052] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-06. [2025-11-04T18:57:11.367088] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-04. [2025-11-04T18:57:11.593379] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T18:57:12.167096] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-07. [2025-11-04T18:57:12.799133] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T18:57:12.967060] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-09. [2025-11-04T18:57:13.165023] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T18:57:13.358435] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-11-04T18:57:13.767104] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-08. [2025-11-04T18:57:14.567115] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-10. [2025-11-04T18:57:14.840234] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T18:57:14.890864] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-10 (0.32 seconds). [2025-11-04T18:57:15.367096] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-05. [2025-11-04T18:57:16.167047] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-09. [2025-11-04T18:57:16.770930] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T18:57:16.967050] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-04. [2025-11-04T18:57:17.128174] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T18:57:17.767016] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-11. [2025-11-04T18:57:18.183130] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T18:57:18.183875] [INFO ] [REMOTE ] FLC: Full fetch complete (7787 records). [2025-11-04T18:57:18.183944] [DEBUG] [VALIDATOR ] FLC: Validating primary data. [2025-11-04T18:57:18.548819] [DEBUG] [VALIDATOR ] FLC: All expected trading days are present. [2025-11-04T18:57:18.548921] [DEBUG] [VALIDATOR ] FLC: Primary data validation passed. [2025-11-04T18:57:18.549151] [INFO ] [MAIN ] FLC: Fetched primary data is valid. [2025-11-04T18:57:18.549217] [DEBUG] [LOCAL ] FLC: Saving primary data. [2025-11-04T18:57:18.549962] [DEBUG] [LOCAL ] FLC: Trimmed 7787 records to 4988 (keeping 200 days of data) [2025-11-04T18:57:18.572099] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-10. [2025-11-04T18:57:18.582937] [DEBUG] [LOCAL ] FLC: Primary data saved (4988 records). [2025-11-04T18:57:18.611812] [INFO ] [MAIN ] FLC: Full history saved. [2025-11-04T18:57:18.612007] [INFO ] [MAIN ] Update Progress: 841/2367 symbols completed. FLC took 22.1s. Estimated remaining: 3h 32m 46s. Rate: 7.2 symbols/min [2025-11-04T18:57:18.682957] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-11-04T18:57:18.822367] [INFO ] [MAIN ] FLOC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:57:18.850077] [DEBUG] [LOCAL ] FLOC: Loaded 27578 existing records from primary storage. [2025-11-04T18:57:18.850376] [INFO ] [MAIN ] FLOC: Loaded 27578 existing records. [2025-11-04T18:57:19.024915] [WARN ] [MAIN ] FLOC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:57:19.025123] [INFO ] [MAIN ] FLOC: Wiping existing primary data before full refresh. [2025-11-04T18:57:19.025170] [INFO ] [LOCAL ] FLOC: Starting deletion of primary data. [2025-11-04T18:57:19.025336] [DEBUG] [LOCAL ] FLOC: Deleted primary data file. [2025-11-04T18:57:19.025373] [INFO ] [MAIN ] FLOC: Starting full primary refresh. [2025-11-04T18:57:19.376603] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-06. [2025-11-04T18:57:19.387947] [INFO ] [REMOTE ] FLOC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:57:19.468273] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-11-04T18:57:20.167046] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-11. [2025-11-04T18:57:20.669282] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-11-04T18:57:20.884552] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T18:57:20.886027] [INFO ] [REMOTE ] FLNG: Full fetch complete (49512 records). [2025-11-04T18:57:20.886101] [DEBUG] [VALIDATOR ] FLNG: Validating primary data. [2025-11-04T18:57:20.972223] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-05. [2025-11-04T18:57:21.293933] [DEBUG] [VALIDATOR ] FLNG: All expected trading days are present. [2025-11-04T18:57:21.294023] [DEBUG] [VALIDATOR ] FLNG: Primary data validation passed. [2025-11-04T18:57:21.294721] [INFO ] [MAIN ] FLNG: Fetched primary data is valid. [2025-11-04T18:57:21.294789] [DEBUG] [LOCAL ] FLNG: Saving primary data. [2025-11-04T18:57:21.295864] [DEBUG] [LOCAL ] FLNG: Trimmed 49512 records to 32837 (keeping 200 days of data) [2025-11-04T18:57:21.490957] [DEBUG] [LOCAL ] FLNG: Primary data saved (32837 records). [2025-11-04T18:57:21.491081] [INFO ] [MAIN ] FLNG: Full history saved. [2025-11-04T18:57:21.491157] [INFO ] [MAIN ] Update Progress: 842/2367 symbols completed. FLNG took 24.9s. Estimated remaining: 3h 32m 27s. Rate: 7.2 symbols/min [2025-11-04T18:57:21.662628] [INFO ] [MAIN ] FLR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:57:21.711524] [DEBUG] [LOCAL ] FLR: Loaded 53238 existing records from primary storage. [2025-11-04T18:57:21.711843] [INFO ] [MAIN ] FLR: Loaded 53238 existing records. [2025-11-04T18:57:21.779880] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-01. [2025-11-04T18:57:21.900440] [WARN ] [MAIN ] FLR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:57:21.900726] [INFO ] [MAIN ] FLR: Wiping existing primary data before full refresh. [2025-11-04T18:57:21.900782] [INFO ] [LOCAL ] FLR: Starting deletion of primary data. [2025-11-04T18:57:21.900999] [DEBUG] [LOCAL ] FLR: Deleted primary data file. [2025-11-04T18:57:21.901056] [INFO ] [MAIN ] FLR: Starting full primary refresh. [2025-11-04T18:57:22.292820] [INFO ] [REMOTE ] FLR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:57:22.339908] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T18:57:22.563255] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-11-04T18:57:22.570208] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-01. [2025-11-04T18:57:23.367048] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-06. [2025-11-04T18:57:23.772144] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T18:57:24.167053] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-02. [2025-11-04T18:57:24.640770] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T18:57:24.972162] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-07. [2025-11-04T18:57:24.984540] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-11-04T18:57:25.767022] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-07. [2025-11-04T18:57:26.350670] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-11-04T18:57:26.567043] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-03. [2025-11-04T18:57:27.007936] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-11-04T18:57:27.367049] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-08. [2025-11-04T18:57:27.499329] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T18:57:28.167033] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-04. [2025-11-04T18:57:28.653862] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T18:57:28.976554] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-09. [2025-11-04T18:57:28.978024] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-11-04T18:57:29.767020] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-08. [2025-11-04T18:57:30.269889] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T18:57:30.567064] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-05. [2025-11-04T18:57:31.168735] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T18:57:31.367040] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-09. [2025-11-04T18:57:31.453049] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T18:57:32.167026] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-02. [2025-11-04T18:57:32.827707] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-11-04T18:57:32.967012] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-10. [2025-11-04T18:57:33.340089] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T18:57:33.767047] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-06. [2025-11-04T18:57:34.341905] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T18:57:34.441698] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-11-04T18:57:34.567060] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-07. [2025-11-04T18:57:35.266214] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T18:57:35.367042] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-11. [2025-11-04T18:57:36.167029] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-08. [2025-11-04T18:57:36.353243] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T18:57:36.355287] [INFO ] [REMOTE ] FLO: Full fetch complete (75490 records). [2025-11-04T18:57:36.355378] [DEBUG] [VALIDATOR ] FLO: Validating primary data. [2025-11-04T18:57:36.744778] [DEBUG] [VALIDATOR ] FLO: All expected trading days are present. [2025-11-04T18:57:36.744856] [DEBUG] [VALIDATOR ] FLO: Primary data validation passed. [2025-11-04T18:57:36.745730] [INFO ] [MAIN ] FLO: Fetched primary data is valid. [2025-11-04T18:57:36.745801] [DEBUG] [LOCAL ] FLO: Saving primary data. [2025-11-04T18:57:36.748157] [DEBUG] [LOCAL ] FLO: Trimmed 75490 records to 52226 (keeping 200 days of data) [2025-11-04T18:57:36.973679] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-03. [2025-11-04T18:57:37.072311] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-11-04T18:57:37.090772] [DEBUG] [LOCAL ] FLO: Primary data saved (52226 records). [2025-11-04T18:57:37.090905] [INFO ] [MAIN ] FLO: Full history saved. [2025-11-04T18:57:37.090983] [INFO ] [MAIN ] Update Progress: 843/2367 symbols completed. FLO took 40.5s. Estimated remaining: 3h 32m 32s. Rate: 7.2 symbols/min [2025-11-04T18:57:37.259258] [INFO ] [MAIN ] FLS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:57:37.300956] [DEBUG] [LOCAL ] FLS: Loaded 50443 existing records from primary storage. [2025-11-04T18:57:37.301253] [INFO ] [MAIN ] FLS: Loaded 50443 existing records. [2025-11-04T18:57:37.478106] [WARN ] [MAIN ] FLS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:57:37.478320] [INFO ] [MAIN ] FLS: Wiping existing primary data before full refresh. [2025-11-04T18:57:37.478368] [INFO ] [LOCAL ] FLS: Starting deletion of primary data. [2025-11-04T18:57:37.478857] [DEBUG] [LOCAL ] FLS: Deleted primary data file. [2025-11-04T18:57:37.478914] [INFO ] [MAIN ] FLS: Starting full primary refresh. [2025-11-04T18:57:37.772253] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-09. [2025-11-04T18:57:37.839833] [INFO ] [REMOTE ] FLS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:57:38.318014] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T18:57:38.567018] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-04. [2025-11-04T18:57:38.619313] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-11-04T18:57:39.367061] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-10. [2025-11-04T18:57:40.046847] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-11-04T18:57:40.167004] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-05. [2025-11-04T18:57:40.898886] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-11-04T18:57:40.967050] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-10. [2025-11-04T18:57:41.579742] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T18:57:41.767013] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-06. [2025-11-04T18:57:41.873519] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-11-04T18:57:42.567043] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-11. [2025-11-04T18:57:43.090503] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T18:57:43.367072] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-07. [2025-11-04T18:57:43.384516] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-11 (0.82 seconds). [2025-11-04T18:57:43.386353] [INFO ] [REMOTE ] FLOC: Full fetch complete (42457 records). [2025-11-04T18:57:43.386540] [DEBUG] [VALIDATOR ] FLOC: Validating primary data. [2025-11-04T18:57:43.782223] [DEBUG] [VALIDATOR ] FLOC: All expected trading days are present. [2025-11-04T18:57:43.782309] [DEBUG] [VALIDATOR ] FLOC: Primary data validation passed. [2025-11-04T18:57:43.783006] [INFO ] [MAIN ] FLOC: Fetched primary data is valid. [2025-11-04T18:57:43.783069] [DEBUG] [LOCAL ] FLOC: Saving primary data. [2025-11-04T18:57:43.784053] [DEBUG] [LOCAL ] FLOC: Trimmed 42457 records to 27578 (keeping 200 days of data) [2025-11-04T18:57:43.941731] [DEBUG] [LOCAL ] FLOC: Primary data saved (27578 records). [2025-11-04T18:57:43.941854] [INFO ] [MAIN ] FLOC: Full history saved. [2025-11-04T18:57:43.941931] [INFO ] [MAIN ] Update Progress: 844/2367 symbols completed. FLOC took 25.3s. Estimated remaining: 3h 32m 21s. Rate: 7.2 symbols/min [2025-11-04T18:57:44.107696] [INFO ] [MAIN ] FLUT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:57:44.153680] [DEBUG] [LOCAL ] FLUT: Loaded 50621 existing records from primary storage. [2025-11-04T18:57:44.154007] [INFO ] [MAIN ] FLUT: Loaded 50621 existing records. [2025-11-04T18:57:44.174421] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-01. [2025-11-04T18:57:44.333638] [WARN ] [MAIN ] FLUT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:57:44.333916] [INFO ] [MAIN ] FLUT: Wiping existing primary data before full refresh. [2025-11-04T18:57:44.333983] [INFO ] [LOCAL ] FLUT: Starting deletion of primary data. [2025-11-04T18:57:44.334236] [DEBUG] [LOCAL ] FLUT: Deleted primary data file. [2025-11-04T18:57:44.334291] [INFO ] [MAIN ] FLUT: Starting full primary refresh. [2025-11-04T18:57:44.684737] [INFO ] [REMOTE ] FLUT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:57:44.728842] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T18:57:44.967030] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-01. [2025-11-04T18:57:45.474724] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T18:57:45.767088] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-02. [2025-11-04T18:57:46.170174] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T18:57:46.567090] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-02. [2025-11-04T18:57:46.825497] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T18:57:47.367072] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-11. [2025-11-04T18:57:47.768174] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T18:57:48.167090] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-03. [2025-11-04T18:57:48.381060] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-11 (1.01 seconds). [2025-11-04T18:57:48.383099] [INFO ] [REMOTE ] FLG: Full fetch complete (77405 records). [2025-11-04T18:57:48.383187] [DEBUG] [VALIDATOR ] FLG: Validating primary data. [2025-11-04T18:57:48.769913] [DEBUG] [VALIDATOR ] FLG: All expected trading days are present. [2025-11-04T18:57:48.769991] [DEBUG] [VALIDATOR ] FLG: Primary data validation passed. [2025-11-04T18:57:48.770872] [INFO ] [MAIN ] FLG: Fetched primary data is valid. [2025-11-04T18:57:48.770933] [DEBUG] [LOCAL ] FLG: Saving primary data. [2025-11-04T18:57:48.772304] [DEBUG] [LOCAL ] FLG: Trimmed 77405 records to 53296 (keeping 200 days of data) [2025-11-04T18:57:48.975247] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-08. [2025-11-04T18:57:49.092881] [DEBUG] [LOCAL ] FLG: Primary data saved (53296 records). [2025-11-04T18:57:49.093023] [INFO ] [MAIN ] FLG: Full history saved. [2025-11-04T18:57:49.093103] [INFO ] [MAIN ] Update Progress: 845/2367 symbols completed. FLG took 52.5s. Estimated remaining: 3h 32m 7s. Rate: 7.2 symbols/min [2025-11-04T18:57:49.273906] [INFO ] [MAIN ] FLXR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:57:49.295064] [DEBUG] [LOCAL ] FLXR: Loaded 22054 existing records from primary storage. [2025-11-04T18:57:49.295352] [INFO ] [MAIN ] FLXR: Loaded 22054 existing records. [2025-11-04T18:57:49.481833] [WARN ] [MAIN ] FLXR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:57:49.482184] [INFO ] [MAIN ] FLXR: Wiping existing primary data before full refresh. [2025-11-04T18:57:49.482341] [INFO ] [LOCAL ] FLXR: Starting deletion of primary data. [2025-11-04T18:57:49.482620] [DEBUG] [LOCAL ] FLXR: Deleted primary data file. [2025-11-04T18:57:49.482800] [INFO ] [MAIN ] FLXR: Starting full primary refresh. [2025-11-04T18:57:49.769388] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-03. [2025-11-04T18:57:49.818014] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-11-04T18:57:49.907165] [INFO ] [REMOTE ] FLXR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:57:50.567119] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-01. [2025-11-04T18:57:50.625177] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-11-04T18:57:51.049746] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T18:57:51.084689] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-11-04T18:57:51.367039] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-02. [2025-11-04T18:57:51.917356] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T18:57:52.167110] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-03. [2025-11-04T18:57:52.794249] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-11-04T18:57:52.967063] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-04. [2025-11-04T18:57:53.767069] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-09. [2025-11-04T18:57:54.296378] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T18:57:54.567044] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-05. [2025-11-04T18:57:55.145080] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T18:57:55.367050] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-04. [2025-11-04T18:57:55.801735] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T18:57:55.997672] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-11-04T18:57:56.167046] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-06. [2025-11-04T18:57:56.967057] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-04. [2025-11-04T18:57:57.333508] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T18:57:57.767079] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-10. [2025-11-04T18:57:58.183623] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T18:57:58.567055] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-05. [2025-11-04T18:57:59.164417] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T18:57:59.367070] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-05. [2025-11-04T18:57:59.584019] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T18:58:00.056057] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-11-04T18:58:00.167060] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-06. [2025-11-04T18:58:00.967042] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-06. [2025-11-04T18:58:01.320242] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T18:58:01.702765] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-11-04T18:58:01.767065] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-07. [2025-11-04T18:58:02.567077] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-07. [2025-11-04T18:58:03.045852] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T18:58:03.263796] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T18:58:03.367055] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-07. [2025-11-04T18:58:04.167044] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-08. [2025-11-04T18:58:04.677806] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T18:58:04.940449] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T18:58:04.967095] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-08. [2025-11-04T18:58:05.767087] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-11. [2025-11-04T18:58:06.345541] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T18:58:06.567112] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-09. [2025-11-04T18:58:06.970771] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T18:58:06.972929] [INFO ] [REMOTE ] FLR: Full fetch complete (76966 records). [2025-11-04T18:58:06.973090] [DEBUG] [VALIDATOR ] FLR: Validating primary data. [2025-11-04T18:58:07.368471] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-09. [2025-11-04T18:58:07.393206] [DEBUG] [VALIDATOR ] FLR: All expected trading days are present. [2025-11-04T18:58:07.393320] [DEBUG] [VALIDATOR ] FLR: Primary data validation passed. [2025-11-04T18:58:07.395571] [INFO ] [MAIN ] FLR: Fetched primary data is valid. [2025-11-04T18:58:07.395806] [DEBUG] [LOCAL ] FLR: Saving primary data. [2025-11-04T18:58:07.422315] [DEBUG] [LOCAL ] FLR: Trimmed 76966 records to 53243 (keeping 200 days of data) [2025-11-04T18:58:07.424231] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-11-04T18:58:07.745050] [DEBUG] [LOCAL ] FLR: Primary data saved (53243 records). [2025-11-04T18:58:07.745232] [INFO ] [MAIN ] FLR: Full history saved. [2025-11-04T18:58:07.745321] [INFO ] [MAIN ] Update Progress: 846/2367 symbols completed. FLR took 46.3s. Estimated remaining: 3h 32m 17s. Rate: 7.2 symbols/min [2025-11-04T18:58:07.913247] [INFO ] [MAIN ] FMC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:58:07.958669] [DEBUG] [LOCAL ] FMC: Loaded 51894 existing records from primary storage. [2025-11-04T18:58:07.959006] [INFO ] [MAIN ] FMC: Loaded 51894 existing records. [2025-11-04T18:58:08.133268] [WARN ] [MAIN ] FMC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:58:08.133473] [INFO ] [MAIN ] FMC: Wiping existing primary data before full refresh. [2025-11-04T18:58:08.133520] [INFO ] [LOCAL ] FMC: Starting deletion of primary data. [2025-11-04T18:58:08.134337] [DEBUG] [LOCAL ] FMC: Deleted primary data file. [2025-11-04T18:58:08.134394] [INFO ] [MAIN ] FMC: Starting full primary refresh. [2025-11-04T18:58:08.176048] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-08. [2025-11-04T18:58:08.487156] [INFO ] [REMOTE ] FMC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:58:08.894771] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-11-04T18:58:08.967041] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-10. [2025-11-04T18:58:09.667376] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T18:58:09.767127] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-01. [2025-11-04T18:58:10.429952] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T18:58:10.567099] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-09. [2025-11-04T18:58:11.078381] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T18:58:11.367056] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-02. [2025-11-04T18:58:11.886545] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-11-04T18:58:12.167069] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-10. [2025-11-04T18:58:12.745679] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-11-04T18:58:12.967095] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-10. [2025-11-04T18:58:13.635854] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T18:58:13.684731] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-11-04T18:58:13.767028] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-11. [2025-11-04T18:58:14.482063] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T18:58:14.483332] [INFO ] [REMOTE ] FLXR: Full fetch complete (28742 records). [2025-11-04T18:58:14.483401] [DEBUG] [VALIDATOR ] FLXR: Validating primary data. [2025-11-04T18:58:14.572301] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-11. [2025-11-04T18:58:14.881243] [DEBUG] [VALIDATOR ] FLXR: All expected trading days are present. [2025-11-04T18:58:14.881384] [DEBUG] [VALIDATOR ] FLXR: Primary data validation passed. [2025-11-04T18:58:14.881910] [INFO ] [MAIN ] FLXR: Fetched primary data is valid. [2025-11-04T18:58:14.882798] [DEBUG] [LOCAL ] FLXR: Saving primary data. [2025-11-04T18:58:14.883498] [DEBUG] [LOCAL ] FLXR: Trimmed 28742 records to 22137 (keeping 200 days of data) [2025-11-04T18:58:15.013606] [DEBUG] [LOCAL ] FLXR: Primary data saved (22137 records). [2025-11-04T18:58:15.013750] [INFO ] [MAIN ] FLXR: Full history saved. [2025-11-04T18:58:15.013848] [INFO ] [MAIN ] Update Progress: 847/2367 symbols completed. FLXR took 25.9s. Estimated remaining: 3h 32m 7s. Rate: 7.2 symbols/min [2025-11-04T18:58:15.179433] [INFO ] [MAIN ] FMN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:58:15.185841] [DEBUG] [LOCAL ] FMN: Loaded 5618 existing records from primary storage. [2025-11-04T18:58:15.186007] [INFO ] [MAIN ] FMN: Loaded 5618 existing records. [2025-11-04T18:58:15.359786] [WARN ] [MAIN ] FMN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:58:15.359993] [INFO ] [MAIN ] FMN: Wiping existing primary data before full refresh. [2025-11-04T18:58:15.360041] [INFO ] [LOCAL ] FMN: Starting deletion of primary data. [2025-11-04T18:58:15.360226] [DEBUG] [LOCAL ] FMN: Deleted primary data file. [2025-11-04T18:58:15.360265] [INFO ] [MAIN ] FMN: Starting full primary refresh. [2025-11-04T18:58:15.372822] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-03. [2025-11-04T18:58:15.716998] [INFO ] [REMOTE ] FMN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:58:15.748832] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T18:58:15.750707] [INFO ] [REMOTE ] FLS: Full fetch complete (72821 records). [2025-11-04T18:58:15.750793] [DEBUG] [VALIDATOR ] FLS: Validating primary data. [2025-11-04T18:58:16.136051] [DEBUG] [VALIDATOR ] FLS: All expected trading days are present. [2025-11-04T18:58:16.136144] [DEBUG] [VALIDATOR ] FLS: Primary data validation passed. [2025-11-04T18:58:16.137807] [INFO ] [MAIN ] FLS: Fetched primary data is valid. [2025-11-04T18:58:16.137873] [DEBUG] [LOCAL ] FLS: Saving primary data. [2025-11-04T18:58:16.139434] [DEBUG] [LOCAL ] FLS: Trimmed 72821 records to 50458 (keeping 200 days of data) [2025-11-04T18:58:16.191742] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-11. [2025-11-04T18:58:16.445002] [DEBUG] [LOCAL ] FLS: Primary data saved (50458 records). [2025-11-04T18:58:16.445148] [INFO ] [MAIN ] FLS: Full history saved. [2025-11-04T18:58:16.445231] [INFO ] [MAIN ] Update Progress: 848/2367 symbols completed. FLS took 39.4s. Estimated remaining: 3h 31m 46s. Rate: 7.2 symbols/min [2025-11-04T18:58:16.616699] [INFO ] [MAIN ] FMS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:58:16.650023] [DEBUG] [LOCAL ] FMS: Loaded 30145 existing records from primary storage. [2025-11-04T18:58:16.650389] [INFO ] [MAIN ] FMS: Loaded 30145 existing records. [2025-11-04T18:58:16.836745] [WARN ] [MAIN ] FMS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:58:16.837191] [INFO ] [MAIN ] FMS: Wiping existing primary data before full refresh. [2025-11-04T18:58:16.837528] [INFO ] [LOCAL ] FMS: Starting deletion of primary data. [2025-11-04T18:58:16.837848] [DEBUG] [LOCAL ] FMS: Deleted primary data file. [2025-11-04T18:58:16.837955] [INFO ] [MAIN ] FMS: Starting full primary refresh. [2025-11-04T18:58:16.967236] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-01. [2025-11-04T18:58:17.159263] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-11-04T18:58:17.248116] [INFO ] [REMOTE ] FMS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:58:17.452221] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-11 (1.26 seconds). [2025-11-04T18:58:17.454553] [INFO ] [REMOTE ] FLUT: Full fetch complete (73499 records). [2025-11-04T18:58:17.454763] [DEBUG] [VALIDATOR ] FLUT: Validating primary data. [2025-11-04T18:58:17.664452] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-11-04T18:58:17.772206] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-01. [2025-11-04T18:58:17.881973] [DEBUG] [VALIDATOR ] FLUT: All expected trading days are present. [2025-11-04T18:58:17.882057] [DEBUG] [VALIDATOR ] FLUT: Primary data validation passed. [2025-11-04T18:58:17.882910] [INFO ] [MAIN ] FLUT: Fetched primary data is valid. [2025-11-04T18:58:17.882994] [DEBUG] [LOCAL ] FLUT: Saving primary data. [2025-11-04T18:58:17.884382] [DEBUG] [LOCAL ] FLUT: Trimmed 73499 records to 50652 (keeping 200 days of data) [2025-11-04T18:58:18.192573] [DEBUG] [LOCAL ] FLUT: Primary data saved (50652 records). [2025-11-04T18:58:18.192736] [INFO ] [MAIN ] FLUT: Full history saved. [2025-11-04T18:58:18.192825] [INFO ] [MAIN ] Update Progress: 849/2367 symbols completed. FLUT took 34.3s. Estimated remaining: 3h 31m 26s. Rate: 7.2 symbols/min [2025-11-04T18:58:18.358833] [INFO ] [MAIN ] FMX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:58:18.394585] [DEBUG] [LOCAL ] FMX: Loaded 39892 existing records from primary storage. [2025-11-04T18:58:18.394913] [INFO ] [MAIN ] FMX: Loaded 39892 existing records. [2025-11-04T18:58:18.576569] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-04. [2025-11-04T18:58:18.582935] [WARN ] [MAIN ] FMX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:58:18.584182] [INFO ] [MAIN ] FMX: Wiping existing primary data before full refresh. [2025-11-04T18:58:18.584355] [INFO ] [LOCAL ] FMX: Starting deletion of primary data. [2025-11-04T18:58:18.590396] [DEBUG] [LOCAL ] FMX: Deleted primary data file. [2025-11-04T18:58:18.599708] [INFO ] [MAIN ] FMX: Starting full primary refresh. [2025-11-04T18:58:18.622961] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-11-04T18:58:18.957435] [INFO ] [REMOTE ] FMX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:58:19.367061] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-01. [2025-11-04T18:58:19.923751] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T18:58:20.172320] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-02. [2025-11-04T18:58:20.183376] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-11-04T18:58:20.962281] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T18:58:20.967245] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-02. [2025-11-04T18:58:21.447965] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T18:58:21.767038] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-05. [2025-11-04T18:58:22.567045] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-02. [2025-11-04T18:58:23.011324] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-11-04T18:58:23.367037] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-06. [2025-11-04T18:58:23.605957] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-11-04T18:58:24.167048] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-03. [2025-11-04T18:58:24.516945] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T18:58:24.967000] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-03. [2025-11-04T18:58:25.012061] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-11-04T18:58:25.767042] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-04. [2025-11-04T18:58:26.071205] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T18:58:26.567030] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-07. [2025-11-04T18:58:26.653933] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-11-04T18:58:27.367044] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-03. [2025-11-04T18:58:27.853953] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-11-04T18:58:27.890167] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-11-04T18:58:28.167041] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-04. [2025-11-04T18:58:28.638083] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-11-04T18:58:28.967058] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-05. [2025-11-04T18:58:29.364564] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-11-04T18:58:29.767076] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-04. [2025-11-04T18:58:30.567084] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-08. [2025-11-04T18:58:30.969371] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T18:58:31.367042] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-05. [2025-11-04T18:58:31.819566] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T18:58:32.167016] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-09. [2025-11-04T18:58:32.214341] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-11-04T18:58:32.967054] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-06. [2025-11-04T18:58:33.425940] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T18:58:33.767054] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-10. [2025-11-04T18:58:33.806945] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T18:58:34.567073] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-07. [2025-11-04T18:58:35.373666] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-06. [2025-11-04T18:58:35.396665] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-11-04T18:58:35.424552] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-11-04T18:58:35.756443] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-11-04T18:58:36.167051] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-11. [2025-11-04T18:58:36.967074] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-08. [2025-11-04T18:58:37.445267] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-11 (1.28 seconds). [2025-11-04T18:58:37.447504] [INFO ] [REMOTE ] FMC: Full fetch complete (75538 records). [2025-11-04T18:58:37.447776] [DEBUG] [VALIDATOR ] FMC: Validating primary data. [2025-11-04T18:58:37.771464] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-07. [2025-11-04T18:58:37.853351] [DEBUG] [VALIDATOR ] FMC: All expected trading days are present. [2025-11-04T18:58:37.853629] [DEBUG] [VALIDATOR ] FMC: Primary data validation passed. [2025-11-04T18:58:37.854688] [INFO ] [MAIN ] FMC: Fetched primary data is valid. [2025-11-04T18:58:37.854928] [DEBUG] [LOCAL ] FMC: Saving primary data. [2025-11-04T18:58:37.873857] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-11-04T18:58:37.879874] [DEBUG] [LOCAL ] FMC: Trimmed 75538 records to 51926 (keeping 200 days of data) [2025-11-04T18:58:38.190838] [DEBUG] [LOCAL ] FMC: Primary data saved (51926 records). [2025-11-04T18:58:38.190974] [INFO ] [MAIN ] FMC: Full history saved. [2025-11-04T18:58:38.191055] [INFO ] [MAIN ] Update Progress: 850/2367 symbols completed. FMC took 30.4s. Estimated remaining: 3h 31m 38s. Rate: 7.2 symbols/min [2025-11-04T18:58:38.302386] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-11-04T18:58:38.372990] [INFO ] [MAIN ] FMY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:58:38.375969] [DEBUG] [LOCAL ] FMY: Loaded 1317 existing records from primary storage. [2025-11-04T18:58:38.376127] [INFO ] [MAIN ] FMY: Loaded 1317 existing records. [2025-11-04T18:58:38.551908] [WARN ] [MAIN ] FMY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:58:38.552146] [INFO ] [MAIN ] FMY: Wiping existing primary data before full refresh. [2025-11-04T18:58:38.552193] [INFO ] [LOCAL ] FMY: Starting deletion of primary data. [2025-11-04T18:58:38.552365] [DEBUG] [LOCAL ] FMY: Deleted primary data file. [2025-11-04T18:58:38.552422] [INFO ] [MAIN ] FMY: Starting full primary refresh. [2025-11-04T18:58:38.572847] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-05. [2025-11-04T18:58:38.935466] [INFO ] [REMOTE ] FMY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:58:39.367027] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-08. [2025-11-04T18:58:39.699586] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T18:58:39.711082] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-11-04T18:58:40.167035] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-06. [2025-11-04T18:58:40.967011] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-09. [2025-11-04T18:58:41.132098] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-11-04T18:58:41.767022] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-01. [2025-11-04T18:58:41.868802] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-11-04T18:58:42.011070] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-01 (0.24 seconds). [2025-11-04T18:58:42.567065] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-09. [2025-11-04T18:58:42.968315] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-11-04T18:58:43.367059] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-10. [2025-11-04T18:58:43.915016] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-11-04T18:58:44.167027] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-11. [2025-11-04T18:58:44.628194] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-11 (0.46 seconds). [2025-11-04T18:58:44.629105] [INFO ] [REMOTE ] FMN: Full fetch complete (9850 records). [2025-11-04T18:58:44.629174] [DEBUG] [VALIDATOR ] FMN: Validating primary data. [2025-11-04T18:58:44.972132] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-02. [2025-11-04T18:58:45.042681] [DEBUG] [VALIDATOR ] FMN: All expected trading days are present. [2025-11-04T18:58:45.042802] [DEBUG] [VALIDATOR ] FMN: Primary data validation passed. [2025-11-04T18:58:45.043113] [INFO ] [MAIN ] FMN: Fetched primary data is valid. [2025-11-04T18:58:45.043188] [DEBUG] [LOCAL ] FMN: Saving primary data. [2025-11-04T18:58:45.043841] [DEBUG] [LOCAL ] FMN: Trimmed 9850 records to 5693 (keeping 200 days of data) [2025-11-04T18:58:45.081972] [DEBUG] [LOCAL ] FMN: Primary data saved (5693 records). [2025-11-04T18:58:45.082116] [INFO ] [MAIN ] FMN: Full history saved. [2025-11-04T18:58:45.082213] [INFO ] [MAIN ] Update Progress: 851/2367 symbols completed. FMN took 30.1s. Estimated remaining: 3h 31m 27s. Rate: 7.2 symbols/min [2025-11-04T18:58:45.260005] [INFO ] [MAIN ] FN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:58:45.292331] [DEBUG] [LOCAL ] FN: Loaded 36973 existing records from primary storage. [2025-11-04T18:58:45.292612] [INFO ] [MAIN ] FN: Loaded 36973 existing records. [2025-11-04T18:58:45.365542] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-11-04T18:58:45.474002] [WARN ] [MAIN ] FN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:58:45.474246] [INFO ] [MAIN ] FN: Wiping existing primary data before full refresh. [2025-11-04T18:58:45.474292] [INFO ] [LOCAL ] FN: Starting deletion of primary data. [2025-11-04T18:58:45.475043] [DEBUG] [LOCAL ] FN: Deleted primary data file. [2025-11-04T18:58:45.475100] [INFO ] [MAIN ] FN: Starting full primary refresh. [2025-11-04T18:58:45.772273] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-10. [2025-11-04T18:58:45.827281] [INFO ] [REMOTE ] FN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:58:46.567010] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-01. [2025-11-04T18:58:46.757765] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-11-04T18:58:47.367019] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-11. [2025-11-04T18:58:47.612945] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-11-04T18:58:48.167061] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-03. [2025-11-04T18:58:48.328563] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-11 (0.96 seconds). [2025-11-04T18:58:48.330334] [INFO ] [REMOTE ] FMS: Full fetch complete (43519 records). [2025-11-04T18:58:48.330433] [DEBUG] [VALIDATOR ] FMS: Validating primary data. [2025-11-04T18:58:48.437085] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-03 (0.27 seconds). [2025-11-04T18:58:48.698481] [DEBUG] [VALIDATOR ] FMS: All expected trading days are present. [2025-11-04T18:58:48.698560] [DEBUG] [VALIDATOR ] FMS: Primary data validation passed. [2025-11-04T18:58:48.699242] [INFO ] [MAIN ] FMS: Fetched primary data is valid. [2025-11-04T18:58:48.699303] [DEBUG] [LOCAL ] FMS: Saving primary data. [2025-11-04T18:58:48.700247] [DEBUG] [LOCAL ] FMS: Trimmed 43519 records to 30211 (keeping 200 days of data) [2025-11-04T18:58:48.874173] [DEBUG] [LOCAL ] FMS: Primary data saved (30211 records). [2025-11-04T18:58:48.874305] [INFO ] [MAIN ] FMS: Full history saved. [2025-11-04T18:58:48.874395] [INFO ] [MAIN ] Update Progress: 852/2367 symbols completed. FMS took 32.4s. Estimated remaining: 3h 31m 10s. Rate: 7.2 symbols/min [2025-11-04T18:58:48.972539] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-04. [2025-11-04T18:58:49.045920] [INFO ] [MAIN ] FNB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:58:49.094575] [DEBUG] [LOCAL ] FNB: Loaded 52594 existing records from primary storage. [2025-11-04T18:58:49.094924] [INFO ] [MAIN ] FNB: Loaded 52594 existing records. [2025-11-04T18:58:49.276591] [WARN ] [MAIN ] FNB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:58:49.276830] [INFO ] [MAIN ] FNB: Wiping existing primary data before full refresh. [2025-11-04T18:58:49.276881] [INFO ] [LOCAL ] FNB: Starting deletion of primary data. [2025-11-04T18:58:49.277910] [DEBUG] [LOCAL ] FNB: Deleted primary data file. [2025-11-04T18:58:49.277972] [INFO ] [MAIN ] FNB: Starting full primary refresh. [2025-11-04T18:58:49.351167] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T18:58:49.635684] [INFO ] [REMOTE ] FNB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:58:49.767055] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-05. [2025-11-04T18:58:49.991953] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-05 (0.22 seconds). [2025-11-04T18:58:50.567050] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-06. [2025-11-04T18:58:50.829457] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-06 (0.26 seconds). [2025-11-04T18:58:51.367026] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-02. [2025-11-04T18:58:52.167027] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-07. [2025-11-04T18:58:52.431771] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T18:58:52.967044] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-03. [2025-11-04T18:58:53.195716] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T18:58:53.767047] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-07. [2025-11-04T18:58:54.142233] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-11-04T18:58:54.177415] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-11-04T18:58:54.567043] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-08. [2025-11-04T18:58:54.954455] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-11-04T18:58:55.367055] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-08. [2025-11-04T18:58:56.167082] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-09. [2025-11-04T18:58:56.417354] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-09 (0.25 seconds). [2025-11-04T18:58:56.510223] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T18:58:56.967069] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-01. [2025-11-04T18:58:57.767056] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-04. [2025-11-04T18:58:58.090910] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T18:58:58.567093] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-02. [2025-11-04T18:58:58.980280] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T18:58:59.367094] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-05. [2025-11-04T18:58:59.670146] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T18:59:00.167032] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-10. [2025-11-04T18:59:00.387639] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T18:59:00.447338] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-10 (0.28 seconds). [2025-11-04T18:59:00.967042] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-06. [2025-11-04T18:59:01.767119] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-09. [2025-11-04T18:59:01.963506] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T18:59:02.567073] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-11. [2025-11-04T18:59:02.911535] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T18:59:02.912667] [INFO ] [REMOTE ] FMY: Full fetch complete (1934 records). [2025-11-04T18:59:02.912760] [DEBUG] [VALIDATOR ] FMY: Validating primary data. [2025-11-04T18:59:02.913627] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T18:59:03.271981] [DEBUG] [VALIDATOR ] FMY: All expected trading days are present. [2025-11-04T18:59:03.272056] [DEBUG] [VALIDATOR ] FMY: Primary data validation passed. [2025-11-04T18:59:03.272136] [INFO ] [MAIN ] FMY: Fetched primary data is valid. [2025-11-04T18:59:03.272173] [DEBUG] [LOCAL ] FMY: Saving primary data. [2025-11-04T18:59:03.272489] [DEBUG] [LOCAL ] FMY: Trimmed 1934 records to 1310 (keeping 200 days of data) [2025-11-04T18:59:03.280666] [DEBUG] [LOCAL ] FMY: Primary data saved (1310 records). [2025-11-04T18:59:03.280737] [INFO ] [MAIN ] FMY: Full history saved. [2025-11-04T18:59:03.280803] [INFO ] [MAIN ] Update Progress: 853/2367 symbols completed. FMY took 25.1s. Estimated remaining: 3h 31m 13s. Rate: 7.2 symbols/min [2025-11-04T18:59:03.372263] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-03. [2025-11-04T18:59:03.449844] [INFO ] [MAIN ] FND: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:59:03.493982] [DEBUG] [LOCAL ] FND: Loaded 52023 existing records from primary storage. [2025-11-04T18:59:03.494287] [INFO ] [MAIN ] FND: Loaded 52023 existing records. [2025-11-04T18:59:03.665942] [WARN ] [MAIN ] FND: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:59:03.666133] [INFO ] [MAIN ] FND: Wiping existing primary data before full refresh. [2025-11-04T18:59:03.666178] [INFO ] [LOCAL ] FND: Starting deletion of primary data. [2025-11-04T18:59:03.667064] [DEBUG] [LOCAL ] FND: Deleted primary data file. [2025-11-04T18:59:03.667121] [INFO ] [MAIN ] FND: Starting full primary refresh. [2025-11-04T18:59:04.007752] [INFO ] [REMOTE ] FND: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:59:04.167003] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-01. [2025-11-04T18:59:04.701092] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T18:59:04.967059] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-04. [2025-11-04T18:59:05.293027] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T18:59:05.767070] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-10. [2025-11-04T18:59:06.217616] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T18:59:06.567025] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-02. [2025-11-04T18:59:06.709340] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T18:59:07.367078] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-05. [2025-11-04T18:59:07.765986] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T18:59:08.167034] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-11. [2025-11-04T18:59:08.589182] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T18:59:08.967020] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-06. [2025-11-04T18:59:09.053487] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T18:59:09.055042] [INFO ] [REMOTE ] FMX: Full fetch complete (57859 records). [2025-11-04T18:59:09.055136] [DEBUG] [VALIDATOR ] FMX: Validating primary data. [2025-11-04T18:59:09.448741] [DEBUG] [VALIDATOR ] FMX: All expected trading days are present. [2025-11-04T18:59:09.448847] [DEBUG] [VALIDATOR ] FMX: Primary data validation passed. [2025-11-04T18:59:09.450039] [INFO ] [MAIN ] FMX: Fetched primary data is valid. [2025-11-04T18:59:09.450143] [DEBUG] [LOCAL ] FMX: Saving primary data. [2025-11-04T18:59:09.451832] [DEBUG] [LOCAL ] FMX: Trimmed 57859 records to 39899 (keeping 200 days of data) [2025-11-04T18:59:09.689825] [DEBUG] [LOCAL ] FMX: Primary data saved (39899 records). [2025-11-04T18:59:09.689950] [INFO ] [MAIN ] FMX: Full history saved. [2025-11-04T18:59:09.690025] [INFO ] [MAIN ] Update Progress: 854/2367 symbols completed. FMX took 51.5s. Estimated remaining: 3h 31m 1s. Rate: 7.2 symbols/min [2025-11-04T18:59:09.772111] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-03. [2025-11-04T18:59:09.872637] [INFO ] [MAIN ] FNF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:59:09.916024] [DEBUG] [LOCAL ] FNF: Loaded 48334 existing records from primary storage. [2025-11-04T18:59:09.916338] [INFO ] [MAIN ] FNF: Loaded 48334 existing records. [2025-11-04T18:59:10.102122] [WARN ] [MAIN ] FNF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:59:10.102375] [INFO ] [MAIN ] FNF: Wiping existing primary data before full refresh. [2025-11-04T18:59:10.102503] [INFO ] [LOCAL ] FNF: Starting deletion of primary data. [2025-11-04T18:59:10.102826] [DEBUG] [LOCAL ] FNF: Deleted primary data file. [2025-11-04T18:59:10.102944] [INFO ] [MAIN ] FNF: Starting full primary refresh. [2025-11-04T18:59:10.286245] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T18:59:10.505676] [INFO ] [REMOTE ] FNF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:59:10.567062] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-01. [2025-11-04T18:59:11.183436] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T18:59:11.367044] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-04. [2025-11-04T18:59:11.575964] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T18:59:12.167041] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-07. [2025-11-04T18:59:12.734862] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-11-04T18:59:12.967057] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-07. [2025-11-04T18:59:13.421604] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T18:59:13.767051] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-08. [2025-11-04T18:59:14.046269] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T18:59:14.567031] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-08. [2025-11-04T18:59:15.039182] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T18:59:15.367059] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-02. [2025-11-04T18:59:15.828681] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T18:59:16.167040] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-05. [2025-11-04T18:59:16.423541] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T18:59:16.967042] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-03. [2025-11-04T18:59:17.531059] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T18:59:17.767038] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-06. [2025-11-04T18:59:18.089904] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T18:59:18.567018] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-04. [2025-11-04T18:59:18.767744] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T18:59:19.367053] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-09. [2025-11-04T18:59:19.665341] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T18:59:20.167040] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-05. [2025-11-04T18:59:20.548406] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T18:59:20.967035] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-10. [2025-11-04T18:59:21.320394] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T18:59:21.767058] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-09. [2025-11-04T18:59:22.155361] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-11-04T18:59:22.567040] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-11. [2025-11-04T18:59:22.817899] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T18:59:23.367083] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-06. [2025-11-04T18:59:23.838320] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-11 (1.27 seconds). [2025-11-04T18:59:23.840482] [INFO ] [REMOTE ] FNB: Full fetch complete (76258 records). [2025-11-04T18:59:23.840615] [DEBUG] [VALIDATOR ] FNB: Validating primary data. [2025-11-04T18:59:24.172167] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-10. [2025-11-04T18:59:24.254726] [DEBUG] [VALIDATOR ] FNB: All expected trading days are present. [2025-11-04T18:59:24.254807] [DEBUG] [VALIDATOR ] FNB: Primary data validation passed. [2025-11-04T18:59:24.255691] [INFO ] [MAIN ] FNB: Fetched primary data is valid. [2025-11-04T18:59:24.255753] [DEBUG] [LOCAL ] FNB: Saving primary data. [2025-11-04T18:59:24.257276] [DEBUG] [LOCAL ] FNB: Trimmed 76258 records to 52593 (keeping 200 days of data) [2025-11-04T18:59:24.601448] [DEBUG] [LOCAL ] FNB: Primary data saved (52593 records). [2025-11-04T18:59:24.601694] [INFO ] [MAIN ] FNB: Full history saved. [2025-11-04T18:59:24.602391] [INFO ] [MAIN ] Update Progress: 855/2367 symbols completed. FNB took 35.7s. Estimated remaining: 3h 31m 4s. Rate: 7.2 symbols/min [2025-11-04T18:59:24.603358] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T18:59:24.787279] [INFO ] [MAIN ] FNV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:59:24.827408] [DEBUG] [LOCAL ] FNV: Loaded 45681 existing records from primary storage. [2025-11-04T18:59:24.827725] [INFO ] [MAIN ] FNV: Loaded 45681 existing records. [2025-11-04T18:59:24.972205] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-07. [2025-11-04T18:59:25.014069] [WARN ] [MAIN ] FNV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:59:25.014491] [INFO ] [MAIN ] FNV: Wiping existing primary data before full refresh. [2025-11-04T18:59:25.014541] [INFO ] [LOCAL ] FNV: Starting deletion of primary data. [2025-11-04T18:59:25.015564] [DEBUG] [LOCAL ] FNV: Deleted primary data file. [2025-11-04T18:59:25.015627] [INFO ] [MAIN ] FNV: Starting full primary refresh. [2025-11-04T18:59:25.384197] [INFO ] [REMOTE ] FNV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:59:25.408280] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-11-04T18:59:25.767044] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-11. [2025-11-04T18:59:26.108868] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T18:59:26.567038] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-07. [2025-11-04T18:59:26.739921] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T18:59:26.741865] [INFO ] [REMOTE ] FN: Full fetch complete (56935 records). [2025-11-04T18:59:26.741994] [DEBUG] [VALIDATOR ] FN: Validating primary data. [2025-11-04T18:59:27.130523] [DEBUG] [VALIDATOR ] FN: All expected trading days are present. [2025-11-04T18:59:27.130596] [DEBUG] [VALIDATOR ] FN: Primary data validation passed. [2025-11-04T18:59:27.131310] [INFO ] [MAIN ] FN: Fetched primary data is valid. [2025-11-04T18:59:27.131371] [DEBUG] [LOCAL ] FN: Saving primary data. [2025-11-04T18:59:27.132447] [DEBUG] [LOCAL ] FN: Trimmed 56935 records to 37070 (keeping 200 days of data) [2025-11-04T18:59:27.345619] [DEBUG] [LOCAL ] FN: Primary data saved (37070 records). [2025-11-04T18:59:27.345767] [INFO ] [MAIN ] FN: Full history saved. [2025-11-04T18:59:27.345846] [INFO ] [MAIN ] Update Progress: 856/2367 symbols completed. FN took 42.3s. Estimated remaining: 3h 30m 46s. Rate: 7.2 symbols/min [2025-11-04T18:59:27.374891] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-08. [2025-11-04T18:59:27.520362] [INFO ] [MAIN ] FOA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:59:27.534682] [DEBUG] [LOCAL ] FOA: Loaded 12614 existing records from primary storage. [2025-11-04T18:59:27.534947] [INFO ] [MAIN ] FOA: Loaded 12614 existing records. [2025-11-04T18:59:27.718302] [WARN ] [MAIN ] FOA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:59:27.718670] [INFO ] [MAIN ] FOA: Wiping existing primary data before full refresh. [2025-11-04T18:59:27.718854] [INFO ] [LOCAL ] FOA: Starting deletion of primary data. [2025-11-04T18:59:27.719143] [DEBUG] [LOCAL ] FOA: Deleted primary data file. [2025-11-04T18:59:27.719216] [INFO ] [MAIN ] FOA: Starting full primary refresh. [2025-11-04T18:59:28.089062] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-11-04T18:59:28.110712] [INFO ] [REMOTE ] FOA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:59:28.167023] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-01. [2025-11-04T18:59:28.559133] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-11-04T18:59:28.951739] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-11-04T18:59:28.967008] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-02. [2025-11-04T18:59:29.627801] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-11-04T18:59:29.767039] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-01. [2025-11-04T18:59:30.567120] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-09. [2025-11-04T18:59:30.970620] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T18:59:31.367049] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-08. [2025-11-04T18:59:31.875476] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T18:59:32.167033] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-10. [2025-11-04T18:59:32.560561] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T18:59:32.967068] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-09. [2025-11-04T18:59:33.522203] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-11-04T18:59:33.767059] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-11. [2025-11-04T18:59:34.289936] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-11-04T18:59:34.567039] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-03. [2025-11-04T18:59:34.798100] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T18:59:34.800336] [INFO ] [REMOTE ] FND: Full fetch complete (75209 records). [2025-11-04T18:59:34.800473] [DEBUG] [VALIDATOR ] FND: Validating primary data. [2025-11-04T18:59:35.185220] [DEBUG] [VALIDATOR ] FND: All expected trading days are present. [2025-11-04T18:59:35.185318] [DEBUG] [VALIDATOR ] FND: Primary data validation passed. [2025-11-04T18:59:35.186256] [INFO ] [MAIN ] FND: Fetched primary data is valid. [2025-11-04T18:59:35.186338] [DEBUG] [LOCAL ] FND: Saving primary data. [2025-11-04T18:59:35.196686] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-11-04T18:59:35.201263] [DEBUG] [LOCAL ] FND: Trimmed 75209 records to 52040 (keeping 200 days of data) [2025-11-04T18:59:35.370241] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-04. [2025-11-04T18:59:35.499202] [DEBUG] [LOCAL ] FND: Primary data saved (52040 records). [2025-11-04T18:59:35.499359] [INFO ] [MAIN ] FND: Full history saved. [2025-11-04T18:59:35.499471] [INFO ] [MAIN ] Update Progress: 857/2367 symbols completed. FND took 32.2s. Estimated remaining: 3h 30m 37s. Rate: 7.2 symbols/min [2025-11-04T18:59:35.668436] [INFO ] [MAIN ] FOF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:59:35.678345] [DEBUG] [LOCAL ] FOF: Loaded 8084 existing records from primary storage. [2025-11-04T18:59:35.678532] [INFO ] [MAIN ] FOF: Loaded 8084 existing records. [2025-11-04T18:59:35.892420] [WARN ] [MAIN ] FOF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:59:35.892749] [INFO ] [MAIN ] FOF: Wiping existing primary data before full refresh. [2025-11-04T18:59:35.892810] [INFO ] [LOCAL ] FOF: Starting deletion of primary data. [2025-11-04T18:59:35.892941] [DEBUG] [LOCAL ] FOF: Deleted primary data file. [2025-11-04T18:59:35.892980] [INFO ] [MAIN ] FOF: Starting full primary refresh. [2025-11-04T18:59:36.099032] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T18:59:36.170922] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-02. [2025-11-04T18:59:36.289070] [INFO ] [REMOTE ] FOF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:59:36.967075] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-05. [2025-11-04T18:59:37.269330] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T18:59:37.511414] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-11-04T18:59:37.767047] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-01. [2025-11-04T18:59:38.220135] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-11-04T18:59:38.567054] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-10. [2025-11-04T18:59:39.367053] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-02. [2025-11-04T18:59:39.745430] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T18:59:39.837596] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-11-04T18:59:40.167118] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-03. [2025-11-04T18:59:40.967104] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-03. [2025-11-04T18:59:41.528465] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-11-04T18:59:41.577936] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T18:59:41.767082] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-04. [2025-11-04T18:59:42.567048] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-11. [2025-11-04T18:59:43.144725] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T18:59:43.367044] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-04. [2025-11-04T18:59:43.529819] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-11 (0.96 seconds). [2025-11-04T18:59:43.531701] [INFO ] [REMOTE ] FNF: Full fetch complete (69909 records). [2025-11-04T18:59:43.531802] [DEBUG] [VALIDATOR ] FNF: Validating primary data. [2025-11-04T18:59:43.934744] [DEBUG] [VALIDATOR ] FNF: All expected trading days are present. [2025-11-04T18:59:43.934877] [DEBUG] [VALIDATOR ] FNF: Primary data validation passed. [2025-11-04T18:59:43.935712] [INFO ] [MAIN ] FNF: Fetched primary data is valid. [2025-11-04T18:59:43.935938] [DEBUG] [LOCAL ] FNF: Saving primary data. [2025-11-04T18:59:43.944183] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-11-04T18:59:43.947629] [DEBUG] [LOCAL ] FNF: Trimmed 69909 records to 48333 (keeping 200 days of data) [2025-11-04T18:59:44.171136] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-06. [2025-11-04T18:59:44.238739] [DEBUG] [LOCAL ] FNF: Primary data saved (48333 records). [2025-11-04T18:59:44.238872] [INFO ] [MAIN ] FNF: Full history saved. [2025-11-04T18:59:44.238966] [INFO ] [MAIN ] Update Progress: 858/2367 symbols completed. FNF took 34.5s. Estimated remaining: 3h 30m 29s. Rate: 7.2 symbols/min [2025-11-04T18:59:44.414399] [INFO ] [MAIN ] FOR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T18:59:44.431511] [DEBUG] [LOCAL ] FOR: Loaded 18787 existing records from primary storage. [2025-11-04T18:59:44.431764] [INFO ] [MAIN ] FOR: Loaded 18787 existing records. [2025-11-04T18:59:44.619838] [WARN ] [MAIN ] FOR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T18:59:44.620032] [INFO ] [MAIN ] FOR: Wiping existing primary data before full refresh. [2025-11-04T18:59:44.620079] [INFO ] [LOCAL ] FOR: Starting deletion of primary data. [2025-11-04T18:59:44.620502] [DEBUG] [LOCAL ] FOR: Deleted primary data file. [2025-11-04T18:59:44.620560] [INFO ] [MAIN ] FOR: Starting full primary refresh. [2025-11-04T18:59:44.757949] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-11-04T18:59:44.977016] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-07. [2025-11-04T18:59:45.009357] [INFO ] [REMOTE ] FOR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T18:59:45.540872] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-11-04T18:59:45.767051] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-01. [2025-11-04T18:59:46.567014] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-05. [2025-11-04T18:59:46.638036] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T18:59:47.367060] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-05. [2025-11-04T18:59:47.849113] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T18:59:47.922305] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-11-04T18:59:48.167034] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-06. [2025-11-04T18:59:48.652693] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-11-04T18:59:48.967125] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-07. [2025-11-04T18:59:49.391218] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-11-04T18:59:49.767063] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-02. [2025-11-04T18:59:50.529485] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T18:59:50.567069] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-03. [2025-11-04T18:59:51.267742] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T18:59:51.366997] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-04. [2025-11-04T18:59:52.149839] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-11-04T18:59:52.167026] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-05. [2025-11-04T18:59:52.972170] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-08. [2025-11-04T18:59:52.974269] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T18:59:53.625518] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-11-04T18:59:53.767023] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-06. [2025-11-04T18:59:54.567112] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-06. [2025-11-04T18:59:54.879592] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T18:59:55.203229] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-11-04T18:59:55.367037] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-09. [2025-11-04T18:59:56.001774] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T18:59:56.167036] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-07. [2025-11-04T18:59:56.796973] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-11-04T18:59:56.967041] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-08. [2025-11-04T18:59:57.537151] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-11-04T18:59:57.767016] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-08. [2025-11-04T18:59:58.278943] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-11-04T18:59:58.567086] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-10. [2025-11-04T18:59:59.160536] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-11-04T18:59:59.367074] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-09. [2025-11-04T18:59:59.959347] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-11-04T19:00:00.167044] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-11. [2025-11-04T19:00:00.733343] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T19:00:00.734343] [INFO ] [REMOTE ] FOA: Full fetch complete (20210 records). [2025-11-04T19:00:00.734418] [DEBUG] [VALIDATOR ] FOA: Validating primary data. [2025-11-04T19:00:00.975051] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-07. [2025-11-04T19:00:01.105461] [DEBUG] [VALIDATOR ] FOA: All expected trading days are present. [2025-11-04T19:00:01.105536] [DEBUG] [VALIDATOR ] FOA: Primary data validation passed. [2025-11-04T19:00:01.106027] [INFO ] [MAIN ] FOA: Fetched primary data is valid. [2025-11-04T19:00:01.106094] [DEBUG] [LOCAL ] FOA: Saving primary data. [2025-11-04T19:00:01.106672] [DEBUG] [LOCAL ] FOA: Trimmed 20210 records to 12597 (keeping 200 days of data) [2025-11-04T19:00:01.182036] [DEBUG] [LOCAL ] FOA: Primary data saved (12597 records). [2025-11-04T19:00:01.182185] [INFO ] [MAIN ] FOA: Full history saved. [2025-11-04T19:00:01.182266] [INFO ] [MAIN ] Update Progress: 859/2367 symbols completed. FOA took 33.8s. Estimated remaining: 3h 30m 36s. Rate: 7.2 symbols/min [2025-11-04T19:00:01.359486] [INFO ] [MAIN ] FOUR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:00:01.403665] [DEBUG] [LOCAL ] FOUR: Loaded 49710 existing records from primary storage. [2025-11-04T19:00:01.404053] [INFO ] [MAIN ] FOUR: Loaded 49710 existing records. [2025-11-04T19:00:01.602361] [WARN ] [MAIN ] FOUR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:00:01.602618] [INFO ] [MAIN ] FOUR: Wiping existing primary data before full refresh. [2025-11-04T19:00:01.602716] [INFO ] [LOCAL ] FOUR: Starting deletion of primary data. [2025-11-04T19:00:01.603634] [DEBUG] [LOCAL ] FOUR: Deleted primary data file. [2025-11-04T19:00:01.603742] [INFO ] [MAIN ] FOUR: Starting full primary refresh. [2025-11-04T19:00:01.779177] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-10. [2025-11-04T19:00:02.080216] [INFO ] [REMOTE ] FOUR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:00:02.287569] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T19:00:02.388745] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-11-04T19:00:02.567051] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-09. [2025-11-04T19:00:03.057444] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-11-04T19:00:03.367091] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-01. [2025-11-04T19:00:04.167075] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-08. [2025-11-04T19:00:04.774514] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T19:00:04.967072] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-02. [2025-11-04T19:00:05.267113] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-11-04T19:00:05.767072] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-09. [2025-11-04T19:00:06.123218] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T19:00:06.567067] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-11. [2025-11-04T19:00:06.830397] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T19:00:07.337297] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T19:00:07.338429] [INFO ] [REMOTE ] FOR: Full fetch complete (29237 records). [2025-11-04T19:00:07.338506] [DEBUG] [VALIDATOR ] FOR: Validating primary data. [2025-11-04T19:00:07.372164] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-03. [2025-11-04T19:00:07.729326] [DEBUG] [VALIDATOR ] FOR: All expected trading days are present. [2025-11-04T19:00:07.729408] [DEBUG] [VALIDATOR ] FOR: Primary data validation passed. [2025-11-04T19:00:07.729930] [INFO ] [MAIN ] FOR: Fetched primary data is valid. [2025-11-04T19:00:07.729992] [DEBUG] [LOCAL ] FOR: Saving primary data. [2025-11-04T19:00:07.730828] [DEBUG] [LOCAL ] FOR: Trimmed 29237 records to 18649 (keeping 200 days of data) [2025-11-04T19:00:07.845313] [DEBUG] [LOCAL ] FOR: Primary data saved (18649 records). [2025-11-04T19:00:07.845455] [INFO ] [MAIN ] FOR: Full history saved. [2025-11-04T19:00:07.845534] [INFO ] [MAIN ] Update Progress: 860/2367 symbols completed. FOR took 23.6s. Estimated remaining: 3h 30m 25s. Rate: 7.2 symbols/min [2025-11-04T19:00:08.010450] [INFO ] [MAIN ] FPF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:00:08.027629] [DEBUG] [LOCAL ] FPF: Loaded 18536 existing records from primary storage. [2025-11-04T19:00:08.027871] [INFO ] [MAIN ] FPF: Loaded 18536 existing records. [2025-11-04T19:00:08.172187] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-10. [2025-11-04T19:00:08.208997] [WARN ] [MAIN ] FPF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:00:08.209437] [INFO ] [MAIN ] FPF: Wiping existing primary data before full refresh. [2025-11-04T19:00:08.209487] [INFO ] [LOCAL ] FPF: Starting deletion of primary data. [2025-11-04T19:00:08.209942] [DEBUG] [LOCAL ] FPF: Deleted primary data file. [2025-11-04T19:00:08.209999] [INFO ] [MAIN ] FPF: Starting full primary refresh. [2025-11-04T19:00:08.575908] [INFO ] [REMOTE ] FPF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:00:08.761370] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T19:00:08.967053] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-04. [2025-11-04T19:00:09.618362] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T19:00:09.767072] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-10. [2025-11-04T19:00:10.280787] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-11-04T19:00:10.362815] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T19:00:10.567054] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-05. [2025-11-04T19:00:11.367060] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-11. [2025-11-04T19:00:11.782264] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T19:00:12.167046] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-01. [2025-11-04T19:00:12.770278] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-11 (1.40 seconds). [2025-11-04T19:00:12.772994] [INFO ] [REMOTE ] FNV: Full fetch complete (67091 records). [2025-11-04T19:00:12.773197] [DEBUG] [VALIDATOR ] FNV: Validating primary data. [2025-11-04T19:00:12.979096] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-11. [2025-11-04T19:00:13.072695] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-11-04T19:00:13.259605] [DEBUG] [VALIDATOR ] FNV: All expected trading days are present. [2025-11-04T19:00:13.259732] [DEBUG] [VALIDATOR ] FNV: Primary data validation passed. [2025-11-04T19:00:13.260666] [INFO ] [MAIN ] FNV: Fetched primary data is valid. [2025-11-04T19:00:13.260748] [DEBUG] [LOCAL ] FNV: Saving primary data. [2025-11-04T19:00:13.262231] [DEBUG] [LOCAL ] FNV: Trimmed 67091 records to 45652 (keeping 200 days of data) [2025-11-04T19:00:13.548954] [DEBUG] [LOCAL ] FNV: Primary data saved (45652 records). [2025-11-04T19:00:13.549089] [INFO ] [MAIN ] FNV: Full history saved. [2025-11-04T19:00:13.549171] [INFO ] [MAIN ] Update Progress: 861/2367 symbols completed. FNV took 48.9s. Estimated remaining: 3h 30m 12s. Rate: 7.2 symbols/min [2025-11-04T19:00:13.655400] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T19:00:13.664741] [INFO ] [REMOTE ] FOF: Full fetch complete (11914 records). [2025-11-04T19:00:13.685408] [DEBUG] [VALIDATOR ] FOF: Validating primary data. [2025-11-04T19:00:13.734005] [INFO ] [MAIN ] FPH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:00:13.773230] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-06. [2025-11-04T19:00:13.915731] [DEBUG] [LOCAL ] FPH: Loaded 19136 existing records from primary storage. [2025-11-04T19:00:13.938733] [INFO ] [MAIN ] FPH: Loaded 19136 existing records. [2025-11-04T19:00:14.333927] [DEBUG] [VALIDATOR ] FOF: All expected trading days are present. [2025-11-04T19:00:14.334249] [DEBUG] [VALIDATOR ] FOF: Primary data validation passed. [2025-11-04T19:00:14.333808] [WARN ] [MAIN ] FPH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:00:14.334700] [INFO ] [MAIN ] FOF: Fetched primary data is valid. [2025-11-04T19:00:14.334797] [DEBUG] [LOCAL ] FOF: Saving primary data. [2025-11-04T19:00:14.334952] [INFO ] [MAIN ] FPH: Wiping existing primary data before full refresh. [2025-11-04T19:00:14.335200] [INFO ] [LOCAL ] FPH: Starting deletion of primary data. [2025-11-04T19:00:14.335778] [DEBUG] [LOCAL ] FPH: Deleted primary data file. [2025-11-04T19:00:14.335870] [DEBUG] [LOCAL ] FOF: Trimmed 11914 records to 8098 (keeping 200 days of data) [2025-11-04T19:00:14.335996] [INFO ] [MAIN ] FPH: Starting full primary refresh. [2025-11-04T19:00:14.568424] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-02. [2025-11-04T19:00:14.614440] [DEBUG] [LOCAL ] FOF: Primary data saved (8098 records). [2025-11-04T19:00:14.644672] [INFO ] [MAIN ] FOF: Full history saved. [2025-11-04T19:00:14.667384] [INFO ] [MAIN ] Update Progress: 862/2367 symbols completed. FOF took 39.2s. Estimated remaining: 3h 29m 51s. Rate: 7.2 symbols/min [2025-11-04T19:00:14.912293] [INFO ] [REMOTE ] FPH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:00:14.971071] [INFO ] [MAIN ] FPI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:00:14.971395] [DEBUG] [LOCAL ] FPI: No existing data file found in primary storage. [2025-11-04T19:00:14.971462] [WARN ] [MAIN ] FPI: No existing data found, performing full refresh instead. [2025-11-04T19:00:14.971743] [INFO ] [MAIN ] FPI: Wiping existing primary data before full refresh. [2025-11-04T19:00:14.971818] [INFO ] [LOCAL ] FPI: Starting deletion of primary data. [2025-11-04T19:00:14.971912] [DEBUG] [LOCAL ] FPI: No primary data file found to delete. [2025-11-04T19:00:14.972098] [INFO ] [MAIN ] FPI: Starting full primary refresh. [2025-11-04T19:00:15.350629] [INFO ] [REMOTE ] FPI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:00:15.367577] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-01. [2025-11-04T19:00:15.380443] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-11-04T19:00:15.390740] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-11-04T19:00:16.062146] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T19:00:16.167021] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-02. [2025-11-04T19:00:16.921381] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T19:00:16.967003] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-03. [2025-11-04T19:00:17.760058] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T19:00:17.768286] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-01. [2025-11-04T19:00:18.575715] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-03. [2025-11-04T19:00:18.581369] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T19:00:19.287959] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T19:00:19.367030] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-02. [2025-11-04T19:00:20.029627] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-11-04T19:00:20.167042] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-07. [2025-11-04T19:00:20.967104] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-03. [2025-11-04T19:00:21.368448] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T19:00:21.673706] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T19:00:21.767030] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-04. [2025-11-04T19:00:22.567126] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-04. [2025-11-04T19:00:22.692565] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-11-04T19:00:23.314826] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T19:00:23.367003] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-05. [2025-11-04T19:00:24.032187] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T19:00:24.167043] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-05. [2025-11-04T19:00:24.967013] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-08. [2025-11-04T19:00:25.242398] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-11-04T19:00:25.767059] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-04. [2025-11-04T19:00:26.522208] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-11-04T19:00:26.572140] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-05. [2025-11-04T19:00:26.599034] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-11-04T19:00:27.328499] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-11-04T19:00:27.367023] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-09. [2025-11-04T19:00:28.167032] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-06. [2025-11-04T19:00:28.835523] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-11-04T19:00:28.953076] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T19:00:28.966999] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-07. [2025-11-04T19:00:29.742623] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-11-04T19:00:29.767004] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-08. [2025-11-04T19:00:30.443868] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-11-04T19:00:30.567050] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-06. [2025-11-04T19:00:31.367027] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-06. [2025-11-04T19:00:31.472245] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T19:00:31.873722] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T19:00:32.167056] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-09. [2025-11-04T19:00:32.835476] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-11-04T19:00:32.967049] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-07. [2025-11-04T19:00:33.637323] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T19:00:33.767056] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-10. [2025-11-04T19:00:34.572146] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-07. [2025-11-04T19:00:34.586395] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T19:00:35.367072] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-08. [2025-11-04T19:00:35.633415] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-11-04T19:00:35.994314] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-11-04T19:00:36.167032] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-11. [2025-11-04T19:00:36.880774] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T19:00:36.882074] [INFO ] [REMOTE ] FPF: Full fetch complete (26728 records). [2025-11-04T19:00:36.882146] [DEBUG] [VALIDATOR ] FPF: Validating primary data. [2025-11-04T19:00:36.972230] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-09. [2025-11-04T19:00:37.260805] [DEBUG] [VALIDATOR ] FPF: All expected trading days are present. [2025-11-04T19:00:37.260886] [DEBUG] [VALIDATOR ] FPF: Primary data validation passed. [2025-11-04T19:00:37.261441] [INFO ] [MAIN ] FPF: Fetched primary data is valid. [2025-11-04T19:00:37.261481] [DEBUG] [LOCAL ] FPF: Saving primary data. [2025-11-04T19:00:37.262203] [DEBUG] [LOCAL ] FPF: Trimmed 26728 records to 18598 (keeping 200 days of data) [2025-11-04T19:00:37.372724] [DEBUG] [LOCAL ] FPF: Primary data saved (18598 records). [2025-11-04T19:00:37.372862] [INFO ] [MAIN ] FPF: Full history saved. [2025-11-04T19:00:37.372940] [INFO ] [MAIN ] Update Progress: 863/2367 symbols completed. FPF took 29.5s. Estimated remaining: 3h 30m 7s. Rate: 7.2 symbols/min [2025-11-04T19:00:37.547212] [INFO ] [MAIN ] FR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:00:37.589477] [DEBUG] [LOCAL ] FR: Loaded 46273 existing records from primary storage. [2025-11-04T19:00:37.589824] [INFO ] [MAIN ] FR: Loaded 46273 existing records. [2025-11-04T19:00:37.767878] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-10. [2025-11-04T19:00:37.767782] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-11-04T19:00:37.786932] [WARN ] [MAIN ] FR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:00:37.792160] [INFO ] [MAIN ] FR: Wiping existing primary data before full refresh. [2025-11-04T19:00:37.792265] [INFO ] [LOCAL ] FR: Starting deletion of primary data. [2025-11-04T19:00:37.793341] [DEBUG] [LOCAL ] FR: Deleted primary data file. [2025-11-04T19:00:37.793452] [INFO ] [MAIN ] FR: Starting full primary refresh. [2025-11-04T19:00:38.154989] [INFO ] [REMOTE ] FR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:00:38.567108] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-10. [2025-11-04T19:00:39.174427] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-11-04T19:00:39.233419] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T19:00:39.367049] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-01. [2025-11-04T19:00:40.167038] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-08. [2025-11-04T19:00:40.347919] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T19:00:40.967050] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-11. [2025-11-04T19:00:40.996543] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-11-04T19:00:41.767078] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-09. [2025-11-04T19:00:42.480279] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-11 (1.51 seconds). [2025-11-04T19:00:42.483150] [INFO ] [REMOTE ] FOUR: Full fetch complete (72433 records). [2025-11-04T19:00:42.483462] [DEBUG] [VALIDATOR ] FOUR: Validating primary data. [2025-11-04T19:00:42.572018] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-02. [2025-11-04T19:00:42.846362] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T19:00:42.926449] [DEBUG] [VALIDATOR ] FOUR: All expected trading days are present. [2025-11-04T19:00:42.926538] [DEBUG] [VALIDATOR ] FOUR: Primary data validation passed. [2025-11-04T19:00:42.927554] [INFO ] [MAIN ] FOUR: Fetched primary data is valid. [2025-11-04T19:00:42.927622] [DEBUG] [LOCAL ] FOUR: Saving primary data. [2025-11-04T19:00:42.929002] [DEBUG] [LOCAL ] FOUR: Trimmed 72433 records to 49751 (keeping 200 days of data) [2025-11-04T19:00:43.226578] [DEBUG] [LOCAL ] FOUR: Primary data saved (49751 records). [2025-11-04T19:00:43.226763] [INFO ] [MAIN ] FOUR: Full history saved. [2025-11-04T19:00:43.226858] [INFO ] [MAIN ] Update Progress: 864/2367 symbols completed. FOUR took 42.0s. Estimated remaining: 3h 29m 54s. Rate: 7.2 symbols/min [2025-11-04T19:00:43.372699] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-11. [2025-11-04T19:00:43.399167] [INFO ] [MAIN ] FRA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:00:43.412887] [DEBUG] [LOCAL ] FRA: Loaded 13234 existing records from primary storage. [2025-11-04T19:00:43.413178] [INFO ] [MAIN ] FRA: Loaded 13234 existing records. [2025-11-04T19:00:43.602776] [WARN ] [MAIN ] FRA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:00:43.603044] [INFO ] [MAIN ] FRA: Wiping existing primary data before full refresh. [2025-11-04T19:00:43.603100] [INFO ] [LOCAL ] FRA: Starting deletion of primary data. [2025-11-04T19:00:43.603475] [DEBUG] [LOCAL ] FRA: Deleted primary data file. [2025-11-04T19:00:43.603545] [INFO ] [MAIN ] FRA: Starting full primary refresh. [2025-11-04T19:00:44.014778] [INFO ] [REMOTE ] FRA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:00:44.044617] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T19:00:44.047052] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-11-04T19:00:44.058599] [INFO ] [REMOTE ] FPH: Full fetch complete (30511 records). [2025-11-04T19:00:44.058851] [DEBUG] [VALIDATOR ] FPH: Validating primary data. [2025-11-04T19:00:44.167123] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-01. [2025-11-04T19:00:44.462362] [DEBUG] [VALIDATOR ] FPH: All expected trading days are present. [2025-11-04T19:00:44.462468] [DEBUG] [VALIDATOR ] FPH: Primary data validation passed. [2025-11-04T19:00:44.463260] [INFO ] [MAIN ] FPH: Fetched primary data is valid. [2025-11-04T19:00:44.463326] [DEBUG] [LOCAL ] FPH: Saving primary data. [2025-11-04T19:00:44.464395] [DEBUG] [LOCAL ] FPH: Trimmed 30511 records to 19093 (keeping 200 days of data) [2025-11-04T19:00:44.581711] [DEBUG] [LOCAL ] FPH: Primary data saved (19093 records). [2025-11-04T19:00:44.581925] [INFO ] [MAIN ] FPH: Full history saved. [2025-11-04T19:00:44.582031] [INFO ] [MAIN ] Update Progress: 865/2367 symbols completed. FPH took 31.0s. Estimated remaining: 3h 29m 34s. Rate: 7.2 symbols/min [2025-11-04T19:00:44.765412] [INFO ] [MAIN ] FRGE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:00:44.782534] [DEBUG] [LOCAL ] FRGE: Loaded 15026 existing records from primary storage. [2025-11-04T19:00:44.782893] [INFO ] [MAIN ] FRGE: Loaded 15026 existing records. [2025-11-04T19:00:44.983453] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-10. [2025-11-04T19:00:44.991356] [WARN ] [MAIN ] FRGE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:00:44.993018] [INFO ] [MAIN ] FRGE: Wiping existing primary data before full refresh. [2025-11-04T19:00:44.993243] [INFO ] [LOCAL ] FRGE: Starting deletion of primary data. [2025-11-04T19:00:44.999006] [DEBUG] [LOCAL ] FRGE: Deleted primary data file. [2025-11-04T19:00:45.010145] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-11-04T19:00:45.009421] [INFO ] [MAIN ] FRGE: Starting full primary refresh. [2025-11-04T19:00:45.394748] [INFO ] [REMOTE ] FRGE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:00:45.767083] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-01. [2025-11-04T19:00:45.902532] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-11-04T19:00:46.567051] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-02. [2025-11-04T19:00:46.692931] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-11-04T19:00:47.216890] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-11-04T19:00:47.367079] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-02. [2025-11-04T19:00:48.167070] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-11. [2025-11-04T19:00:48.215311] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T19:00:48.967023] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-03. [2025-11-04T19:00:48.972294] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T19:00:48.973890] [INFO ] [REMOTE ] FPI: Full fetch complete (43992 records). [2025-11-04T19:00:48.973966] [DEBUG] [VALIDATOR ] FPI: Validating primary data. [2025-11-04T19:00:49.356315] [DEBUG] [VALIDATOR ] FPI: All expected trading days are present. [2025-11-04T19:00:49.356392] [DEBUG] [VALIDATOR ] FPI: Primary data validation passed. [2025-11-04T19:00:49.357076] [INFO ] [MAIN ] FPI: Fetched primary data is valid. [2025-11-04T19:00:49.357138] [DEBUG] [LOCAL ] FPI: Saving primary data. [2025-11-04T19:00:49.358080] [DEBUG] [LOCAL ] FPI: Trimmed 43992 records to 31071 (keeping 200 days of data) [2025-11-04T19:00:49.541188] [DEBUG] [LOCAL ] FPI: Primary data saved (31071 records). [2025-11-04T19:00:49.541314] [INFO ] [MAIN ] FPI: Full history saved. [2025-11-04T19:00:49.541384] [INFO ] [MAIN ] Update Progress: 866/2367 symbols completed. FPI took 34.9s. Estimated remaining: 3h 29m 20s. Rate: 7.2 symbols/min [2025-11-04T19:00:49.710567] [INFO ] [MAIN ] FRO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:00:49.757751] [DEBUG] [LOCAL ] FRO: Loaded 51564 existing records from primary storage. [2025-11-04T19:00:49.758132] [INFO ] [MAIN ] FRO: Loaded 51564 existing records. [2025-11-04T19:00:49.772174] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-03. [2025-11-04T19:00:49.941820] [WARN ] [MAIN ] FRO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:00:49.942224] [INFO ] [MAIN ] FRO: Wiping existing primary data before full refresh. [2025-11-04T19:00:49.942296] [INFO ] [LOCAL ] FRO: Starting deletion of primary data. [2025-11-04T19:00:49.949032] [DEBUG] [LOCAL ] FRO: Deleted primary data file. [2025-11-04T19:00:49.960457] [INFO ] [MAIN ] FRO: Starting full primary refresh. [2025-11-04T19:00:49.961041] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T19:00:50.346501] [INFO ] [REMOTE ] FRO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:00:50.567098] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-04. [2025-11-04T19:00:50.988998] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T19:00:51.270168] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T19:00:51.367041] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-05. [2025-11-04T19:00:52.037375] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T19:00:52.167036] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-04. [2025-11-04T19:00:52.967087] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-01. [2025-11-04T19:00:53.494137] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T19:00:53.767123] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-06. [2025-11-04T19:00:54.321842] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-11-04T19:00:54.408722] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T19:00:54.567046] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-03. [2025-11-04T19:00:55.133525] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T19:00:55.367055] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-02. [2025-11-04T19:00:56.167053] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-07. [2025-11-04T19:00:56.760677] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-11-04T19:00:56.779747] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-11-04T19:00:56.967041] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-04. [2025-11-04T19:00:57.684040] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-11-04T19:00:57.767038] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-05. [2025-11-04T19:00:58.202466] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-11-04T19:00:58.567103] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-06. [2025-11-04T19:00:59.185064] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-11-04T19:00:59.367044] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-08. [2025-11-04T19:01:00.010389] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-11-04T19:01:00.167051] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-09. [2025-11-04T19:01:00.727251] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T19:01:00.967043] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-07. [2025-11-04T19:01:01.545148] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-11-04T19:01:01.767072] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-10. [2025-11-04T19:01:02.391306] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-11-04T19:01:02.567057] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-05. [2025-11-04T19:01:03.367066] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-03. [2025-11-04T19:01:03.645913] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-11-04T19:01:04.167083] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-06. [2025-11-04T19:01:04.786119] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T19:01:04.967080] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-04. [2025-11-04T19:01:05.269127] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T19:01:05.767053] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-08. [2025-11-04T19:01:06.256464] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-11-04T19:01:06.529954] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T19:01:06.567042] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-05. [2025-11-04T19:01:07.367020] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-07. [2025-11-04T19:01:07.969413] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T19:01:08.167077] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-06. [2025-11-04T19:01:08.626658] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T19:01:08.967128] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-08. [2025-11-04T19:01:09.425173] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-11-04T19:01:09.767050] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-11. [2025-11-04T19:01:10.210564] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T19:01:10.455734] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-11 (0.69 seconds). [2025-11-04T19:01:10.457122] [INFO ] [REMOTE ] FRGE: Full fetch complete (29234 records). [2025-11-04T19:01:10.457231] [DEBUG] [VALIDATOR ] FRGE: Validating primary data. [2025-11-04T19:01:10.572196] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-07. [2025-11-04T19:01:10.832300] [DEBUG] [VALIDATOR ] FRGE: All expected trading days are present. [2025-11-04T19:01:10.832374] [DEBUG] [VALIDATOR ] FRGE: Primary data validation passed. [2025-11-04T19:01:10.832886] [INFO ] [MAIN ] FRGE: Fetched primary data is valid. [2025-11-04T19:01:10.832947] [DEBUG] [LOCAL ] FRGE: Saving primary data. [2025-11-04T19:01:10.833559] [DEBUG] [LOCAL ] FRGE: Trimmed 29234 records to 15127 (keeping 200 days of data) [2025-11-04T19:01:10.916792] [DEBUG] [LOCAL ] FRGE: Primary data saved (15127 records). [2025-11-04T19:01:10.916918] [INFO ] [MAIN ] FRGE: Full history saved. [2025-11-04T19:01:10.916995] [INFO ] [MAIN ] Update Progress: 867/2367 symbols completed. FRGE took 26.3s. Estimated remaining: 3h 29m 34s. Rate: 7.2 symbols/min [2025-11-04T19:01:11.084079] [INFO ] [MAIN ] FRT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:01:11.119708] [DEBUG] [LOCAL ] FRT: Loaded 41024 existing records from primary storage. [2025-11-04T19:01:11.119982] [INFO ] [MAIN ] FRT: Loaded 41024 existing records. [2025-11-04T19:01:11.295218] [WARN ] [MAIN ] FRT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:01:11.295452] [INFO ] [MAIN ] FRT: Wiping existing primary data before full refresh. [2025-11-04T19:01:11.295499] [INFO ] [LOCAL ] FRT: Starting deletion of primary data. [2025-11-04T19:01:11.295707] [DEBUG] [LOCAL ] FRT: Deleted primary data file. [2025-11-04T19:01:11.295762] [INFO ] [MAIN ] FRT: Starting full primary refresh. [2025-11-04T19:01:11.372239] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-09. [2025-11-04T19:01:11.648484] [INFO ] [REMOTE ] FRT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:01:12.047992] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T19:01:12.167004] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-08. [2025-11-04T19:01:12.518248] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T19:01:12.967062] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-09. [2025-11-04T19:01:13.512390] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-11-04T19:01:13.549504] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T19:01:13.767043] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-09. [2025-11-04T19:01:14.567050] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-10. [2025-11-04T19:01:15.209360] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T19:01:15.367027] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-10. [2025-11-04T19:01:15.812446] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T19:01:16.167069] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-11. [2025-11-04T19:01:16.817069] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T19:01:16.967079] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-01. [2025-11-04T19:01:17.108472] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T19:01:17.110283] [INFO ] [REMOTE ] FR: Full fetch complete (67529 records). [2025-11-04T19:01:17.110369] [DEBUG] [VALIDATOR ] FR: Validating primary data. [2025-11-04T19:01:17.491292] [DEBUG] [VALIDATOR ] FR: All expected trading days are present. [2025-11-04T19:01:17.491379] [DEBUG] [VALIDATOR ] FR: Primary data validation passed. [2025-11-04T19:01:17.492325] [INFO ] [MAIN ] FR: Fetched primary data is valid. [2025-11-04T19:01:17.492389] [DEBUG] [LOCAL ] FR: Saving primary data. [2025-11-04T19:01:17.493749] [DEBUG] [LOCAL ] FR: Trimmed 67529 records to 46213 (keeping 200 days of data) [2025-11-04T19:01:17.760068] [DEBUG] [LOCAL ] FR: Primary data saved (46213 records). [2025-11-04T19:01:17.760189] [INFO ] [MAIN ] FR: Full history saved. [2025-11-04T19:01:17.760267] [INFO ] [MAIN ] Update Progress: 868/2367 symbols completed. FR took 40.4s. Estimated remaining: 3h 29m 23s. Rate: 7.2 symbols/min [2025-11-04T19:01:17.772128] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-10. [2025-11-04T19:01:17.938075] [INFO ] [MAIN ] FSCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:01:17.972167] [DEBUG] [LOCAL ] FSCO: Loaded 38338 existing records from primary storage. [2025-11-04T19:01:17.972565] [INFO ] [MAIN ] FSCO: Loaded 38338 existing records. [2025-11-04T19:01:18.159970] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T19:01:18.181077] [WARN ] [MAIN ] FSCO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:01:18.189479] [INFO ] [MAIN ] FSCO: Wiping existing primary data before full refresh. [2025-11-04T19:01:18.189535] [INFO ] [LOCAL ] FSCO: Starting deletion of primary data. [2025-11-04T19:01:18.189779] [DEBUG] [LOCAL ] FSCO: Deleted primary data file. [2025-11-04T19:01:18.189835] [INFO ] [MAIN ] FSCO: Starting full primary refresh. [2025-11-04T19:01:18.532135] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-11-04T19:01:18.551281] [INFO ] [REMOTE ] FSCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:01:18.566995] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-01. [2025-11-04T19:01:19.367102] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-11. [2025-11-04T19:01:19.600196] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-11-04T19:01:20.167076] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-11. [2025-11-04T19:01:20.618505] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T19:01:20.620754] [INFO ] [REMOTE ] FRO: Full fetch complete (75266 records). [2025-11-04T19:01:20.620875] [DEBUG] [VALIDATOR ] FRO: Validating primary data. [2025-11-04T19:01:20.752719] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-11 (0.59 seconds). [2025-11-04T19:01:20.783803] [INFO ] [REMOTE ] FRA: Full fetch complete (21884 records). [2025-11-04T19:01:20.805332] [DEBUG] [VALIDATOR ] FRA: Validating primary data. [2025-11-04T19:01:20.969861] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-02. [2025-11-04T19:01:21.204246] [DEBUG] [VALIDATOR ] FRO: All expected trading days are present. [2025-11-04T19:01:21.257713] [DEBUG] [VALIDATOR ] FRO: Primary data validation passed. [2025-11-04T19:01:21.291119] [INFO ] [MAIN ] FRO: Fetched primary data is valid. [2025-11-04T19:01:21.319972] [DEBUG] [LOCAL ] FRO: Saving primary data. [2025-11-04T19:01:21.377766] [DEBUG] [LOCAL ] FRO: Trimmed 75266 records to 51566 (keeping 200 days of data) [2025-11-04T19:01:21.404015] [DEBUG] [VALIDATOR ] FRA: All expected trading days are present. [2025-11-04T19:01:21.404090] [DEBUG] [VALIDATOR ] FRA: Primary data validation passed. [2025-11-04T19:01:21.404313] [INFO ] [MAIN ] FRA: Fetched primary data is valid. [2025-11-04T19:01:21.404357] [DEBUG] [LOCAL ] FRA: Saving primary data. [2025-11-04T19:01:21.465175] [DEBUG] [LOCAL ] FRA: Trimmed 21884 records to 13265 (keeping 200 days of data) [2025-11-04T19:01:21.773671] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-02. [2025-11-04T19:01:21.808558] [DEBUG] [LOCAL ] FRA: Primary data saved (13265 records). [2025-11-04T19:01:21.811795] [INFO ] [MAIN ] FRA: Full history saved. [2025-11-04T19:01:21.813102] [INFO ] [MAIN ] Update Progress: 869/2367 symbols completed. FRA took 38.6s. Estimated remaining: 3h 29m 7s. Rate: 7.2 symbols/min [2025-11-04T19:01:21.894162] [DEBUG] [LOCAL ] FRO: Primary data saved (51566 records). [2025-11-04T19:01:21.911576] [INFO ] [MAIN ] FRO: Full history saved. [2025-11-04T19:01:21.923390] [INFO ] [MAIN ] Update Progress: 870/2367 symbols completed. FRO took 32.4s. Estimated remaining: 3h 28m 44s. Rate: 7.2 symbols/min [2025-11-04T19:01:22.087488] [INFO ] [MAIN ] FSK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:01:22.181775] [INFO ] [MAIN ] FSLY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:01:22.201822] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T19:01:22.243339] [DEBUG] [LOCAL ] FSK: Loaded 49338 existing records from primary storage. [2025-11-04T19:01:22.255430] [INFO ] [MAIN ] FSK: Loaded 49338 existing records. [2025-11-04T19:01:22.392547] [DEBUG] [LOCAL ] FSLY: Loaded 51207 existing records from primary storage. [2025-11-04T19:01:22.430849] [INFO ] [MAIN ] FSLY: Loaded 51207 existing records. [2025-11-04T19:01:22.520174] [WARN ] [MAIN ] FSK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:01:22.536839] [INFO ] [MAIN ] FSK: Wiping existing primary data before full refresh. [2025-11-04T19:01:22.547718] [INFO ] [LOCAL ] FSK: Starting deletion of primary data. [2025-11-04T19:01:22.564020] [DEBUG] [LOCAL ] FSK: Deleted primary data file. [2025-11-04T19:01:22.569290] [INFO ] [MAIN ] FSK: Starting full primary refresh. [2025-11-04T19:01:22.569179] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-03. [2025-11-04T19:01:22.724888] [WARN ] [MAIN ] FSLY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:01:22.748227] [INFO ] [MAIN ] FSLY: Wiping existing primary data before full refresh. [2025-11-04T19:01:22.763884] [INFO ] [LOCAL ] FSLY: Starting deletion of primary data. [2025-11-04T19:01:22.793903] [DEBUG] [LOCAL ] FSLY: Deleted primary data file. [2025-11-04T19:01:22.841064] [INFO ] [MAIN ] FSLY: Starting full primary refresh. [2025-11-04T19:01:23.127707] [INFO ] [REMOTE ] FSK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:01:23.381635] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-11-04T19:01:23.381751] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-01. [2025-11-04T19:01:23.436210] [INFO ] [REMOTE ] FSLY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:01:23.678170] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T19:01:24.167030] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-03. [2025-11-04T19:01:24.571451] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T19:01:24.967124] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-02. [2025-11-04T19:01:25.366160] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T19:01:25.767075] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-04. [2025-11-04T19:01:25.941438] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T19:01:26.567097] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-01. [2025-11-04T19:01:26.904173] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T19:01:27.367101] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-05. [2025-11-04T19:01:27.876251] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T19:01:28.167045] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-04. [2025-11-04T19:01:28.402998] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-11-04T19:01:28.967089] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-02. [2025-11-04T19:01:29.424538] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T19:01:29.767042] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-05. [2025-11-04T19:01:30.176792] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T19:01:30.567030] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-06. [2025-11-04T19:01:30.890719] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-11-04T19:01:31.367022] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-03. [2025-11-04T19:01:31.478598] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T19:01:32.167088] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-06. [2025-11-04T19:01:32.657710] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-11-04T19:01:32.967065] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-04. [2025-11-04T19:01:33.237750] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T19:01:33.767045] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-03. [2025-11-04T19:01:34.254751] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T19:01:34.567021] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-07. [2025-11-04T19:01:35.099754] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T19:01:35.367173] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-04. [2025-11-04T19:01:35.512997] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-11-04T19:01:36.167102] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-08. [2025-11-04T19:01:36.629290] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T19:01:36.967114] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-05. [2025-11-04T19:01:37.238952] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T19:01:37.767095] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-09. [2025-11-04T19:01:38.317637] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-11-04T19:01:38.567045] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-07. [2025-11-04T19:01:38.829375] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T19:01:39.367059] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-10. [2025-11-04T19:01:39.742039] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T19:01:40.167079] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-05. [2025-11-04T19:01:40.557236] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-11-04T19:01:40.967077] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-08. [2025-11-04T19:01:41.575433] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T19:01:41.767060] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-06. [2025-11-04T19:01:42.185744] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T19:01:42.567241] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-06. [2025-11-04T19:01:42.812076] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T19:01:43.367053] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-09. [2025-11-04T19:01:43.680366] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T19:01:44.167055] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-07. [2025-11-04T19:01:44.439088] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-11-04T19:01:44.967296] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-10. [2025-11-04T19:01:45.379256] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-11-04T19:01:45.767109] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-08. [2025-11-04T19:01:46.128208] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T19:01:46.567053] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-07. [2025-11-04T19:01:47.106992] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T19:01:47.367075] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-09. [2025-11-04T19:01:47.897493] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T19:01:48.167075] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-11. [2025-11-04T19:01:48.841268] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-11-04T19:01:48.967181] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-08. [2025-11-04T19:01:49.112851] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-11 (0.95 seconds). [2025-11-04T19:01:49.114599] [INFO ] [REMOTE ] FSCO: Full fetch complete (55282 records). [2025-11-04T19:01:49.114708] [DEBUG] [VALIDATOR ] FSCO: Validating primary data. [2025-11-04T19:01:49.486849] [DEBUG] [VALIDATOR ] FSCO: All expected trading days are present. [2025-11-04T19:01:49.486924] [DEBUG] [VALIDATOR ] FSCO: Primary data validation passed. [2025-11-04T19:01:49.487684] [INFO ] [MAIN ] FSCO: Fetched primary data is valid. [2025-11-04T19:01:49.487744] [DEBUG] [LOCAL ] FSCO: Saving primary data. [2025-11-04T19:01:49.488760] [DEBUG] [LOCAL ] FSCO: Trimmed 55282 records to 38385 (keeping 200 days of data) [2025-11-04T19:01:49.706243] [DEBUG] [LOCAL ] FSCO: Primary data saved (38385 records). [2025-11-04T19:01:49.706383] [INFO ] [MAIN ] FSCO: Full history saved. [2025-11-04T19:01:49.706459] [INFO ] [MAIN ] Update Progress: 871/2367 symbols completed. FSCO took 31.9s. Estimated remaining: 3h 29m 9s. Rate: 7.2 symbols/min [2025-11-04T19:01:49.772266] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-11. [2025-11-04T19:01:49.876948] [INFO ] [MAIN ] FSM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:01:49.925017] [DEBUG] [LOCAL ] FSM: Loaded 53294 existing records from primary storage. [2025-11-04T19:01:49.925328] [INFO ] [MAIN ] FSM: Loaded 53294 existing records. [2025-11-04T19:01:50.106502] [WARN ] [MAIN ] FSM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:01:50.106951] [INFO ] [MAIN ] FSM: Wiping existing primary data before full refresh. [2025-11-04T19:01:50.107011] [INFO ] [LOCAL ] FSM: Starting deletion of primary data. [2025-11-04T19:01:50.107341] [DEBUG] [LOCAL ] FSM: Deleted primary data file. [2025-11-04T19:01:50.107397] [INFO ] [MAIN ] FSM: Starting full primary refresh. [2025-11-04T19:01:50.498619] [INFO ] [REMOTE ] FSM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:01:50.530196] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-11-04T19:01:50.567040] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-09. [2025-11-04T19:01:50.732899] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-11 (0.96 seconds). [2025-11-04T19:01:50.734723] [INFO ] [REMOTE ] FRT: Full fetch complete (60537 records). [2025-11-04T19:01:50.734815] [DEBUG] [VALIDATOR ] FRT: Validating primary data. [2025-11-04T19:01:51.120965] [DEBUG] [VALIDATOR ] FRT: All expected trading days are present. [2025-11-04T19:01:51.121044] [DEBUG] [VALIDATOR ] FRT: Primary data validation passed. [2025-11-04T19:01:51.121795] [INFO ] [MAIN ] FRT: Fetched primary data is valid. [2025-11-04T19:01:51.121856] [DEBUG] [LOCAL ] FRT: Saving primary data. [2025-11-04T19:01:51.123047] [DEBUG] [LOCAL ] FRT: Trimmed 60537 records to 41074 (keeping 200 days of data) [2025-11-04T19:01:51.367482] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-01. [2025-11-04T19:01:51.370094] [DEBUG] [LOCAL ] FRT: Primary data saved (41074 records). [2025-11-04T19:01:51.370258] [INFO ] [MAIN ] FRT: Full history saved. [2025-11-04T19:01:51.370339] [INFO ] [MAIN ] Update Progress: 872/2367 symbols completed. FRT took 40.5s. Estimated remaining: 3h 28m 49s. Rate: 7.2 symbols/min [2025-11-04T19:01:51.548896] [INFO ] [MAIN ] FSS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:01:51.582541] [DEBUG] [LOCAL ] FSS: Loaded 32715 existing records from primary storage. [2025-11-04T19:01:51.582855] [INFO ] [MAIN ] FSS: Loaded 32715 existing records. [2025-11-04T19:01:51.761470] [WARN ] [MAIN ] FSS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:01:51.761820] [INFO ] [MAIN ] FSS: Wiping existing primary data before full refresh. [2025-11-04T19:01:51.761945] [INFO ] [LOCAL ] FSS: Starting deletion of primary data. [2025-11-04T19:01:51.762143] [DEBUG] [LOCAL ] FSS: Deleted primary data file. [2025-11-04T19:01:51.762198] [INFO ] [MAIN ] FSS: Starting full primary refresh. [2025-11-04T19:01:52.126312] [INFO ] [REMOTE ] FSS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:01:52.155394] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T19:01:52.167003] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-01. [2025-11-04T19:01:52.641256] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T19:01:52.962978] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-11-04T19:01:52.969033] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-02. [2025-11-04T19:01:53.767044] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-10. [2025-11-04T19:01:53.964268] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T19:01:54.567073] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-03. [2025-11-04T19:01:55.212519] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T19:01:55.367076] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-10. [2025-11-04T19:01:55.610041] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T19:01:56.167082] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-11. [2025-11-04T19:01:56.661851] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T19:01:56.967091] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-02. [2025-11-04T19:01:57.281150] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T19:01:57.283203] [INFO ] [REMOTE ] FSK: Full fetch complete (72284 records). [2025-11-04T19:01:57.283314] [DEBUG] [VALIDATOR ] FSK: Validating primary data. [2025-11-04T19:01:57.689081] [DEBUG] [VALIDATOR ] FSK: All expected trading days are present. [2025-11-04T19:01:57.689177] [DEBUG] [VALIDATOR ] FSK: Primary data validation passed. [2025-11-04T19:01:57.690032] [INFO ] [MAIN ] FSK: Fetched primary data is valid. [2025-11-04T19:01:57.690094] [DEBUG] [LOCAL ] FSK: Saving primary data. [2025-11-04T19:01:57.691560] [DEBUG] [LOCAL ] FSK: Trimmed 72284 records to 49339 (keeping 200 days of data) [2025-11-04T19:01:57.779929] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-11. [2025-11-04T19:01:58.008297] [DEBUG] [LOCAL ] FSK: Primary data saved (49339 records). [2025-11-04T19:01:58.008428] [INFO ] [MAIN ] FSK: Full history saved. [2025-11-04T19:01:58.008505] [INFO ] [MAIN ] Update Progress: 873/2367 symbols completed. FSK took 36.2s. Estimated remaining: 3h 28m 38s. Rate: 7.2 symbols/min [2025-11-04T19:01:58.179379] [INFO ] [MAIN ] FT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:01:58.187893] [DEBUG] [LOCAL ] FT: Loaded 6750 existing records from primary storage. [2025-11-04T19:01:58.188090] [INFO ] [MAIN ] FT: Loaded 6750 existing records. [2025-11-04T19:01:58.393136] [WARN ] [MAIN ] FT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:01:58.393541] [INFO ] [MAIN ] FT: Wiping existing primary data before full refresh. [2025-11-04T19:01:58.393977] [INFO ] [LOCAL ] FT: Starting deletion of primary data. [2025-11-04T19:01:58.395277] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-11-04T19:01:58.398895] [DEBUG] [LOCAL ] FT: Deleted primary data file. [2025-11-04T19:01:58.404916] [INFO ] [MAIN ] FT: Starting full primary refresh. [2025-11-04T19:01:58.576076] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-04. [2025-11-04T19:01:58.769296] [INFO ] [REMOTE ] FT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:01:59.159884] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-11 (1.38 seconds). [2025-11-04T19:01:59.162017] [INFO ] [REMOTE ] FSLY: Full fetch complete (74871 records). [2025-11-04T19:01:59.162133] [DEBUG] [VALIDATOR ] FSLY: Validating primary data. [2025-11-04T19:01:59.372217] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-03. [2025-11-04T19:01:59.572126] [DEBUG] [VALIDATOR ] FSLY: All expected trading days are present. [2025-11-04T19:01:59.572204] [DEBUG] [VALIDATOR ] FSLY: Primary data validation passed. [2025-11-04T19:01:59.573048] [INFO ] [MAIN ] FSLY: Fetched primary data is valid. [2025-11-04T19:01:59.573109] [DEBUG] [LOCAL ] FSLY: Saving primary data. [2025-11-04T19:01:59.574553] [DEBUG] [LOCAL ] FSLY: Trimmed 74871 records to 51234 (keeping 200 days of data) [2025-11-04T19:01:59.870482] [DEBUG] [LOCAL ] FSLY: Primary data saved (51234 records). [2025-11-04T19:01:59.870737] [INFO ] [MAIN ] FSLY: Full history saved. [2025-11-04T19:01:59.870970] [INFO ] [MAIN ] Update Progress: 874/2367 symbols completed. FSLY took 37.9s. Estimated remaining: 3h 28m 18s. Rate: 7.2 symbols/min [2025-11-04T19:02:00.068498] [INFO ] [MAIN ] FTHY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:02:00.081165] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T19:02:00.100446] [DEBUG] [LOCAL ] FTHY: Loaded 11711 existing records from primary storage. [2025-11-04T19:02:00.100752] [INFO ] [MAIN ] FTHY: Loaded 11711 existing records. [2025-11-04T19:02:00.172223] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-01. [2025-11-04T19:02:00.290954] [WARN ] [MAIN ] FTHY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:02:00.291189] [INFO ] [MAIN ] FTHY: Wiping existing primary data before full refresh. [2025-11-04T19:02:00.291239] [INFO ] [LOCAL ] FTHY: Starting deletion of primary data. [2025-11-04T19:02:00.291790] [DEBUG] [LOCAL ] FTHY: Deleted primary data file. [2025-11-04T19:02:00.291848] [INFO ] [MAIN ] FTHY: Starting full primary refresh. [2025-11-04T19:02:00.666884] [INFO ] [REMOTE ] FTHY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:02:00.755535] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T19:02:00.809100] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-11-04T19:02:00.967061] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-05. [2025-11-04T19:02:01.767035] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-04. [2025-11-04T19:02:01.990060] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T19:02:02.567051] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-02. [2025-11-04T19:02:02.621004] [DEBUG] [REMOTE ] FT: Error fetching primary data for 2025-02: 'API call returned an empty response.' [2025-11-04T19:02:02.621305] [WARN ] [REMOTE ] FT: No data returned for month 2025-02 [2025-11-04T19:02:02.621392] [FATAL] [REMOTE ] FT: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:02:02.621481] [FATAL] [MAIN ] FT: Failed to fetch primary data. [2025-11-04T19:02:02.621547] [WARN ] [MAIN ] FT: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:02:02.621614] [INFO ] [LOCAL ] FT: Starting deletion of primary data. [2025-11-04T19:02:02.621709] [DEBUG] [LOCAL ] FT: No primary data file found to delete. [2025-11-04T19:02:02.621756] [INFO ] [LOCAL ] FT: Starting deletion of derivative data. [2025-11-04T19:02:02.621902] [DEBUG] [LOCAL ] FT: Deleted derivative data file. [2025-11-04T19:02:02.621995] [INFO ] [MAIN ] Update Progress: 875/2367 symbols completed. FT took 4.6s. Estimated remaining: 3h 28m 0s. Rate: 7.2 symbols/min [2025-11-04T19:02:02.794334] [INFO ] [MAIN ] FTI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:02:02.840307] [DEBUG] [LOCAL ] FTI: Loaded 53290 existing records from primary storage. [2025-11-04T19:02:02.840622] [INFO ] [MAIN ] FTI: Loaded 53290 existing records. [2025-11-04T19:02:03.025670] [WARN ] [MAIN ] FTI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:02:03.026123] [INFO ] [MAIN ] FTI: Wiping existing primary data before full refresh. [2025-11-04T19:02:03.026234] [INFO ] [LOCAL ] FTI: Starting deletion of primary data. [2025-11-04T19:02:03.027205] [DEBUG] [LOCAL ] FTI: Deleted primary data file. [2025-11-04T19:02:03.027352] [INFO ] [MAIN ] FTI: Starting full primary refresh. [2025-11-04T19:02:03.230944] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-11-04T19:02:03.372208] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-01. [2025-11-04T19:02:03.435900] [INFO ] [REMOTE ] FTI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:02:03.983800] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-11-04T19:02:04.167066] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-02. [2025-11-04T19:02:04.587591] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-11-04T19:02:04.967049] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-05. [2025-11-04T19:02:05.767033] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-03. [2025-11-04T19:02:06.185507] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T19:02:06.292165] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-11-04T19:02:06.567046] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-06. [2025-11-04T19:02:07.367054] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-01. [2025-11-04T19:02:08.141225] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-11-04T19:02:08.167016] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-06. [2025-11-04T19:02:08.512930] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T19:02:08.967020] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-07. [2025-11-04T19:02:09.108586] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-11-04T19:02:09.767062] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-04. [2025-11-04T19:02:10.381180] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T19:02:10.403293] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-11-04T19:02:10.567011] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-08. [2025-11-04T19:02:11.367043] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-07. [2025-11-04T19:02:12.060697] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T19:02:12.167040] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-05. [2025-11-04T19:02:12.420003] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-11-04T19:02:12.658170] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-11-04T19:02:12.967051] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-02. [2025-11-04T19:02:13.767040] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-09. [2025-11-04T19:02:14.217770] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-11-04T19:02:14.567053] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-08. [2025-11-04T19:02:15.162088] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T19:02:15.367025] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-10. [2025-11-04T19:02:15.558460] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-11-04T19:02:16.167047] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-03. [2025-11-04T19:02:16.787504] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-11-04T19:02:16.967041] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-11. [2025-11-04T19:02:17.574044] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T19:02:17.767034] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-04. [2025-11-04T19:02:18.369040] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-11 (1.40 seconds). [2025-11-04T19:02:18.371525] [INFO ] [REMOTE ] FSM: Full fetch complete (77287 records). [2025-11-04T19:02:18.371667] [DEBUG] [VALIDATOR ] FSM: Validating primary data. [2025-11-04T19:02:18.575200] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-09. [2025-11-04T19:02:18.783858] [DEBUG] [VALIDATOR ] FSM: All expected trading days are present. [2025-11-04T19:02:18.783951] [DEBUG] [VALIDATOR ] FSM: Primary data validation passed. [2025-11-04T19:02:18.784841] [INFO ] [MAIN ] FSM: Fetched primary data is valid. [2025-11-04T19:02:18.784904] [DEBUG] [LOCAL ] FSM: Saving primary data. [2025-11-04T19:02:18.786321] [DEBUG] [LOCAL ] FSM: Trimmed 77287 records to 53293 (keeping 200 days of data) [2025-11-04T19:02:19.122796] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:02:19.157231] [DEBUG] [LOCAL ] FSM: Primary data saved (53293 records). [2025-11-04T19:02:19.157400] [INFO ] [MAIN ] FSM: Full history saved. [2025-11-04T19:02:19.157498] [INFO ] [MAIN ] Update Progress: 876/2367 symbols completed. FSM took 29.5s. Estimated remaining: 3h 28m 6s. Rate: 7.2 symbols/min [2025-11-04T19:02:19.325406] [INFO ] [MAIN ] FTK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:02:19.358886] [DEBUG] [LOCAL ] FTK: Loaded 33542 existing records from primary storage. [2025-11-04T19:02:19.359180] [INFO ] [MAIN ] FTK: Loaded 33542 existing records. [2025-11-04T19:02:19.372681] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-05. [2025-11-04T19:02:19.551523] [WARN ] [MAIN ] FTK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:02:19.551915] [INFO ] [MAIN ] FTK: Wiping existing primary data before full refresh. [2025-11-04T19:02:19.552085] [INFO ] [LOCAL ] FTK: Starting deletion of primary data. [2025-11-04T19:02:19.558157] [DEBUG] [LOCAL ] FTK: Deleted primary data file. [2025-11-04T19:02:19.568403] [INFO ] [MAIN ] FTK: Starting full primary refresh. [2025-11-04T19:02:19.569041] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T19:02:19.945910] [INFO ] [REMOTE ] FTK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:02:20.167021] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-10. [2025-11-04T19:02:20.967047] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-06. [2025-11-04T19:02:20.991526] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T19:02:21.069544] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-11-04T19:02:21.403550] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-11-04T19:02:21.767068] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-06. [2025-11-04T19:02:22.567069] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-11. [2025-11-04T19:02:23.026902] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-11-04T19:02:23.367027] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-07. [2025-11-04T19:02:23.397101] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T19:02:23.398564] [INFO ] [REMOTE ] FSS: Full fetch complete (48784 records). [2025-11-04T19:02:23.398692] [DEBUG] [VALIDATOR ] FSS: Validating primary data. [2025-11-04T19:02:23.777354] [DEBUG] [VALIDATOR ] FSS: All expected trading days are present. [2025-11-04T19:02:23.777472] [DEBUG] [VALIDATOR ] FSS: Primary data validation passed. [2025-11-04T19:02:23.778268] [INFO ] [MAIN ] FSS: Fetched primary data is valid. [2025-11-04T19:02:23.778335] [DEBUG] [LOCAL ] FSS: Saving primary data. [2025-11-04T19:02:23.779596] [DEBUG] [LOCAL ] FSS: Trimmed 48784 records to 32629 (keeping 200 days of data) [2025-11-04T19:02:23.893170] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-11-04T19:02:23.997890] [DEBUG] [LOCAL ] FSS: Primary data saved (32629 records). [2025-11-04T19:02:23.998034] [INFO ] [MAIN ] FSS: Full history saved. [2025-11-04T19:02:23.998114] [INFO ] [MAIN ] Update Progress: 877/2367 symbols completed. FSS took 32.6s. Estimated remaining: 3h 27m 52s. Rate: 7.2 symbols/min [2025-11-04T19:02:24.167087] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-01. [2025-11-04T19:02:24.878527] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T19:02:24.967047] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-07. [2025-11-04T19:02:25.767047] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-02. [2025-11-04T19:02:26.470230] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T19:02:26.567455] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-03. [2025-11-04T19:02:26.571036] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-11-04T19:02:27.371990] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-08. [2025-11-04T19:02:27.374316] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-11-04T19:02:28.167029] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-04. [2025-11-04T19:02:28.957162] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-11-04T19:02:28.966993] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-05. [2025-11-04T19:02:29.767009] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-08. [2025-11-04T19:02:29.956430] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T19:02:30.260923] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-11-04T19:02:30.567058] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-09. [2025-11-04T19:02:31.131576] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T19:02:31.367055] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-10. [2025-11-04T19:02:31.873007] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-11-04T19:02:32.167089] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-11. [2025-11-04T19:02:32.730850] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-11 (0.56 seconds). [2025-11-04T19:02:32.732033] [INFO ] [REMOTE ] FTHY: Full fetch complete (17241 records). [2025-11-04T19:02:32.732126] [DEBUG] [VALIDATOR ] FTHY: Validating primary data. [2025-11-04T19:02:32.980722] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-06. [2025-11-04T19:02:33.099766] [DEBUG] [VALIDATOR ] FTHY: All expected trading days are present. [2025-11-04T19:02:33.099849] [DEBUG] [VALIDATOR ] FTHY: Primary data validation passed. [2025-11-04T19:02:33.100264] [INFO ] [MAIN ] FTHY: Fetched primary data is valid. [2025-11-04T19:02:33.100303] [DEBUG] [LOCAL ] FTHY: Saving primary data. [2025-11-04T19:02:33.100869] [DEBUG] [LOCAL ] FTHY: Trimmed 17241 records to 11838 (keeping 200 days of data) [2025-11-04T19:02:33.172714] [DEBUG] [LOCAL ] FTHY: Primary data saved (11838 records). [2025-11-04T19:02:33.172850] [INFO ] [MAIN ] FTHY: Full history saved. [2025-11-04T19:02:33.172928] [INFO ] [MAIN ] Update Progress: 878/2367 symbols completed. FTHY took 33.3s. Estimated remaining: 3h 27m 45s. Rate: 7.2 symbols/min [2025-11-04T19:02:34.077687] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T19:02:34.085813] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-07. [2025-11-04T19:02:35.042821] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T19:02:35.051174] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-08. [2025-11-04T19:02:36.033992] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-11-04T19:02:36.041592] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-09. [2025-11-04T19:02:37.116963] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T19:02:37.129135] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-10. [2025-11-04T19:02:38.071060] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T19:02:38.079356] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-11. [2025-11-04T19:02:39.189046] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T19:02:39.190489] [INFO ] [REMOTE ] FTK: Full fetch complete (44543 records). [2025-11-04T19:02:39.190560] [DEBUG] [VALIDATOR ] FTK: Validating primary data. [2025-11-04T19:02:39.563584] [DEBUG] [VALIDATOR ] FTK: All expected trading days are present. [2025-11-04T19:02:39.563683] [DEBUG] [VALIDATOR ] FTK: Primary data validation passed. [2025-11-04T19:02:39.564301] [INFO ] [MAIN ] FTK: Fetched primary data is valid. [2025-11-04T19:02:39.564341] [DEBUG] [LOCAL ] FTK: Saving primary data. [2025-11-04T19:02:39.565252] [DEBUG] [LOCAL ] FTK: Trimmed 44543 records to 33634 (keeping 200 days of data) [2025-11-04T19:02:39.758963] [DEBUG] [LOCAL ] FTK: Primary data saved (33634 records). [2025-11-04T19:02:39.759137] [INFO ] [MAIN ] FTK: Full history saved. [2025-11-04T19:02:39.759241] [INFO ] [MAIN ] Update Progress: 879/2367 symbols completed. FTK took 20.6s. Estimated remaining: 3h 27m 33s. Rate: 7.2 symbols/min [2025-11-04T19:03:20.173742] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-08 (52.80 seconds). [2025-11-04T19:03:20.187371] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-09. [2025-11-04T19:03:21.690529] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T19:03:21.703400] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-10. [2025-11-04T19:03:23.068245] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-11-04T19:03:23.081557] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-11. [2025-11-04T19:03:24.244186] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T19:03:24.246294] [INFO ] [REMOTE ] FTI: Full fetch complete (77340 records). [2025-11-04T19:03:24.246392] [DEBUG] [VALIDATOR ] FTI: Validating primary data. [2025-11-04T19:03:24.640228] [DEBUG] [VALIDATOR ] FTI: All expected trading days are present. [2025-11-04T19:03:24.640319] [DEBUG] [VALIDATOR ] FTI: Primary data validation passed. [2025-11-04T19:03:24.641283] [INFO ] [MAIN ] FTI: Fetched primary data is valid. [2025-11-04T19:03:24.641349] [DEBUG] [LOCAL ] FTI: Saving primary data. [2025-11-04T19:03:24.642846] [DEBUG] [LOCAL ] FTI: Trimmed 77340 records to 53290 (keeping 200 days of data) [2025-11-04T19:03:24.961261] [DEBUG] [LOCAL ] FTI: Primary data saved (53290 records). [2025-11-04T19:03:24.961598] [INFO ] [MAIN ] FTI: Full history saved. [2025-11-04T19:03:24.961700] [INFO ] [MAIN ] Update Progress: 880/2367 symbols completed. FTI took 1m 22s. Estimated remaining: 3h 28m 27s. Rate: 7.1 symbols/min [2025-11-04T19:03:25.908997] [INFO ] [MAIN ] FTWO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:25.917603] [INFO ] [MAIN ] FTW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:25.975093] [INFO ] [MAIN ] FTS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:25.998461] [DEBUG] [LOCAL ] FTW: No existing data file found in primary storage. [2025-11-04T19:03:26.017176] [INFO ] [MAIN ] FTV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:26.014042] [WARN ] [MAIN ] FTW: No existing data found, performing full refresh instead. [2025-11-04T19:03:26.019138] [INFO ] [MAIN ] FTW: Wiping existing primary data before full refresh. [2025-11-04T19:03:26.023056] [INFO ] [LOCAL ] FTW: Starting deletion of primary data. [2025-11-04T19:03:26.022635] [DEBUG] [LOCAL ] FTWO: Loaded 1952 existing records from primary storage. [2025-11-04T19:03:26.023366] [DEBUG] [LOCAL ] FTW: No primary data file found to delete. [2025-11-04T19:03:26.023598] [INFO ] [MAIN ] FTWO: Loaded 1952 existing records. [2025-11-04T19:03:26.024095] [INFO ] [MAIN ] FTW: Starting full primary refresh. [2025-11-04T19:03:26.358213] [DEBUG] [LOCAL ] FTS: Loaded 42334 existing records from primary storage. [2025-11-04T19:03:26.392748] [INFO ] [MAIN ] FTS: Loaded 42334 existing records. [2025-11-04T19:03:26.577211] [WARN ] [MAIN ] FTWO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:26.661033] [INFO ] [MAIN ] FTWO: Wiping existing primary data before full refresh. [2025-11-04T19:03:26.727828] [DEBUG] [LOCAL ] FTV: Loaded 53020 existing records from primary storage. [2025-11-04T19:03:26.752578] [INFO ] [LOCAL ] FTWO: Starting deletion of primary data. [2025-11-04T19:03:26.777497] [INFO ] [MAIN ] FTV: Loaded 53020 existing records. [2025-11-04T19:03:26.786829] [DEBUG] [LOCAL ] FTWO: Deleted primary data file. [2025-11-04T19:03:26.841393] [INFO ] [MAIN ] FTWO: Starting full primary refresh. [2025-11-04T19:03:26.925817] [WARN ] [MAIN ] FTS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:26.978893] [INFO ] [MAIN ] FTS: Wiping existing primary data before full refresh. [2025-11-04T19:03:27.004719] [INFO ] [LOCAL ] FTS: Starting deletion of primary data. [2025-11-04T19:03:27.106427] [DEBUG] [LOCAL ] FTS: Deleted primary data file. [2025-11-04T19:03:27.067690] [INFO ] [REMOTE ] FTW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:27.156564] [INFO ] [MAIN ] FTS: Starting full primary refresh. [2025-11-04T19:03:27.219351] [INFO ] [REMOTE ] FTW: Fetching primary data for 2025-01. [2025-11-04T19:03:27.205732] [WARN ] [MAIN ] FTV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:27.242375] [INFO ] [MAIN ] FTV: Wiping existing primary data before full refresh. [2025-11-04T19:03:27.252895] [INFO ] [LOCAL ] FTV: Starting deletion of primary data. [2025-11-04T19:03:27.338679] [DEBUG] [LOCAL ] FTV: Deleted primary data file. [2025-11-04T19:03:27.399159] [INFO ] [MAIN ] FTV: Starting full primary refresh. [2025-11-04T19:03:27.479219] [DEBUG] [REMOTE ] FTW: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T19:03:27.568386] [WARN ] [REMOTE ] FTW: No data returned for month 2025-01 [2025-11-04T19:03:27.594318] [FATAL] [REMOTE ] FTW: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:03:27.703556] [FATAL] [MAIN ] FTW: Failed to fetch primary data. [2025-11-04T19:03:27.847969] [WARN ] [MAIN ] FTW: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:03:27.880734] [INFO ] [REMOTE ] FTWO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:27.906868] [INFO ] [LOCAL ] FTW: Starting deletion of primary data. [2025-11-04T19:03:27.929545] [DEBUG] [LOCAL ] FTW: No primary data file found to delete. [2025-11-04T19:03:27.907147] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-01. [2025-11-04T19:03:27.953185] [INFO ] [LOCAL ] FTW: Starting deletion of derivative data. [2025-11-04T19:03:27.977453] [DEBUG] [LOCAL ] FTW: No derivative data file found to delete. [2025-11-04T19:03:28.079102] [INFO ] [MAIN ] Update Progress: 881/2367 symbols completed. FTW took 3.1s. Estimated remaining: 3h 28m 10s. Rate: 7.1 symbols/min [2025-11-04T19:03:28.262832] [INFO ] [REMOTE ] FTS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:28.288859] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-01. [2025-11-04T19:03:28.449551] [INFO ] [MAIN ] FUBO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:28.470298] [INFO ] [REMOTE ] FTV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:28.470689] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-01. [2025-11-04T19:03:28.481893] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-11-04T19:03:28.483533] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-02. [2025-11-04T19:03:28.525539] [DEBUG] [LOCAL ] FUBO: Loaded 53714 existing records from primary storage. [2025-11-04T19:03:28.525877] [INFO ] [MAIN ] FUBO: Loaded 53714 existing records. [2025-11-04T19:03:28.700198] [WARN ] [MAIN ] FUBO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:28.700474] [INFO ] [MAIN ] FUBO: Wiping existing primary data before full refresh. [2025-11-04T19:03:28.700523] [INFO ] [LOCAL ] FUBO: Starting deletion of primary data. [2025-11-04T19:03:28.700735] [DEBUG] [LOCAL ] FUBO: Deleted primary data file. [2025-11-04T19:03:28.700792] [INFO ] [MAIN ] FUBO: Starting full primary refresh. [2025-11-04T19:03:28.892219] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-11-04T19:03:28.953232] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-03. [2025-11-04T19:03:29.070589] [INFO ] [REMOTE ] FUBO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:29.070934] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-01. [2025-11-04T19:03:29.222055] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-11-04T19:03:29.230009] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-02. [2025-11-04T19:03:29.279960] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-11-04T19:03:29.281150] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-04. [2025-11-04T19:03:29.560073] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-11-04T19:03:29.561380] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-05. [2025-11-04T19:03:29.671535] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T19:03:29.683016] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-02. [2025-11-04T19:03:29.767928] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-05 (0.21 seconds). [2025-11-04T19:03:29.769177] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-06. [2025-11-04T19:03:30.115792] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-11-04T19:03:30.117310] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-07. [2025-11-04T19:03:30.310127] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T19:03:30.318297] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-03. [2025-11-04T19:03:30.411353] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-07 (0.29 seconds). [2025-11-04T19:03:30.413061] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-08. [2025-11-04T19:03:30.642132] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-08 (0.23 seconds). [2025-11-04T19:03:30.643761] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-09. [2025-11-04T19:03:30.927610] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-09 (0.28 seconds). [2025-11-04T19:03:30.931341] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-11-04T19:03:30.944280] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-10. [2025-11-04T19:03:30.944819] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-02. [2025-11-04T19:03:31.235832] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-10 (0.29 seconds). [2025-11-04T19:03:31.237887] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-11. [2025-11-04T19:03:31.472852] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T19:03:31.481300] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-04. [2025-11-04T19:03:31.756161] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-11 (0.52 seconds). [2025-11-04T19:03:31.757053] [INFO ] [REMOTE ] FTWO: Full fetch complete (3244 records). [2025-11-04T19:03:31.757128] [DEBUG] [VALIDATOR ] FTWO: Validating primary data. [2025-11-04T19:03:32.145840] [DEBUG] [VALIDATOR ] FTWO: All expected trading days are present. [2025-11-04T19:03:32.145931] [DEBUG] [VALIDATOR ] FTWO: Primary data validation passed. [2025-11-04T19:03:32.146050] [INFO ] [MAIN ] FTWO: Fetched primary data is valid. [2025-11-04T19:03:32.146088] [DEBUG] [LOCAL ] FTWO: Saving primary data. [2025-11-04T19:03:32.146503] [DEBUG] [LOCAL ] FTWO: Trimmed 3244 records to 1953 (keeping 200 days of data) [2025-11-04T19:03:32.158997] [DEBUG] [LOCAL ] FTWO: Primary data saved (1953 records). [2025-11-04T19:03:32.159124] [INFO ] [MAIN ] FTWO: Full history saved. [2025-11-04T19:03:32.159203] [INFO ] [MAIN ] Update Progress: 882/2367 symbols completed. FTWO took 7.2s. Estimated remaining: 3h 27m 54s. Rate: 7.1 symbols/min [2025-11-04T19:03:32.336808] [INFO ] [MAIN ] FUL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:32.366466] [DEBUG] [LOCAL ] FUL: Loaded 31350 existing records from primary storage. [2025-11-04T19:03:32.366862] [INFO ] [MAIN ] FUL: Loaded 31350 existing records. [2025-11-04T19:03:32.510182] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-02 (2.83 seconds). [2025-11-04T19:03:32.603071] [WARN ] [MAIN ] FUL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:32.597450] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-03. [2025-11-04T19:03:32.604140] [INFO ] [MAIN ] FUL: Wiping existing primary data before full refresh. [2025-11-04T19:03:32.604398] [INFO ] [LOCAL ] FUL: Starting deletion of primary data. [2025-11-04T19:03:32.604766] [DEBUG] [LOCAL ] FUL: Deleted primary data file. [2025-11-04T19:03:32.604963] [INFO ] [MAIN ] FUL: Starting full primary refresh. [2025-11-04T19:03:32.911669] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T19:03:32.951297] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-05. [2025-11-04T19:03:33.063318] [INFO ] [REMOTE ] FUL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:33.056329] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-02 (2.11 seconds). [2025-11-04T19:03:33.063833] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-01. [2025-11-04T19:03:33.074731] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-03. [2025-11-04T19:03:33.748493] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T19:03:33.759420] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-04. [2025-11-04T19:03:33.872042] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T19:03:33.878988] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-02. [2025-11-04T19:03:34.254688] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T19:03:34.263896] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-06. [2025-11-04T19:03:34.627072] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T19:03:34.638896] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-04. [2025-11-04T19:03:34.780114] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T19:03:34.786856] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-03. [2025-11-04T19:03:34.846807] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T19:03:34.857952] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-05. [2025-11-04T19:03:35.424918] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T19:03:35.433881] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-07. [2025-11-04T19:03:36.128339] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-11-04T19:03:36.140490] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-06. [2025-11-04T19:03:36.362020] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-11-04T19:03:36.370967] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-04. [2025-11-04T19:03:36.406423] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-11-04T19:03:36.419711] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-05. [2025-11-04T19:03:36.614505] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T19:03:36.624302] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-08. [2025-11-04T19:03:37.245043] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T19:03:37.261802] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-07. [2025-11-04T19:03:37.604348] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-11-04T19:03:37.612737] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-05. [2025-11-04T19:03:37.712391] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-11-04T19:03:37.721492] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-09. [2025-11-04T19:03:38.096744] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-11-04T19:03:38.107765] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-06. [2025-11-04T19:03:38.410937] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-11-04T19:03:38.416891] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-06. [2025-11-04T19:03:38.654408] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-11-04T19:03:38.663282] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-10. [2025-11-04T19:03:38.820164] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-11-04T19:03:38.831596] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-08. [2025-11-04T19:03:39.262079] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T19:03:39.268510] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-07. [2025-11-04T19:03:39.765932] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T19:03:39.776314] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-11. [2025-11-04T19:03:40.118625] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T19:03:40.146059] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-08. [2025-11-04T19:03:40.150718] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-06 (2.04 seconds). [2025-11-04T19:03:40.163619] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-07. [2025-11-04T19:03:40.319849] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T19:03:40.331963] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-09. [2025-11-04T19:03:40.801199] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T19:03:40.803499] [INFO ] [REMOTE ] FTS: Full fetch complete (61116 records). [2025-11-04T19:03:40.803718] [DEBUG] [VALIDATOR ] FTS: Validating primary data. [2025-11-04T19:03:41.090176] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-11-04T19:03:41.156206] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-09. [2025-11-04T19:03:41.306190] [DEBUG] [VALIDATOR ] FTS: All expected trading days are present. [2025-11-04T19:03:41.306265] [DEBUG] [VALIDATOR ] FTS: Primary data validation passed. [2025-11-04T19:03:41.307064] [INFO ] [MAIN ] FTS: Fetched primary data is valid. [2025-11-04T19:03:41.307128] [DEBUG] [LOCAL ] FTS: Saving primary data. [2025-11-04T19:03:41.308341] [DEBUG] [LOCAL ] FTS: Trimmed 61116 records to 42360 (keeping 200 days of data) [2025-11-04T19:03:41.558869] [DEBUG] [LOCAL ] FTS: Primary data saved (42360 records). [2025-11-04T19:03:41.559004] [INFO ] [MAIN ] FTS: Full history saved. [2025-11-04T19:03:41.559115] [INFO ] [MAIN ] Update Progress: 883/2367 symbols completed. FTS took 16.6s. Estimated remaining: 3h 27m 47s. Rate: 7.1 symbols/min [2025-11-04T19:03:41.764322] [INFO ] [MAIN ] FUN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:41.770172] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T19:03:41.783916] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-10. [2025-11-04T19:03:41.830082] [DEBUG] [LOCAL ] FUN: Loaded 52256 existing records from primary storage. [2025-11-04T19:03:41.841220] [INFO ] [MAIN ] FUN: Loaded 52256 existing records. [2025-11-04T19:03:41.931068] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-11-04T19:03:41.985074] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-08. [2025-11-04T19:03:42.053899] [WARN ] [MAIN ] FUN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:42.054105] [INFO ] [MAIN ] FUN: Wiping existing primary data before full refresh. [2025-11-04T19:03:42.054167] [INFO ] [LOCAL ] FUN: Starting deletion of primary data. [2025-11-04T19:03:42.054377] [DEBUG] [LOCAL ] FUN: Deleted primary data file. [2025-11-04T19:03:42.054434] [INFO ] [MAIN ] FUN: Starting full primary refresh. [2025-11-04T19:03:42.414263] [INFO ] [REMOTE ] FUN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:42.414623] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-01. [2025-11-04T19:03:42.543326] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T19:03:42.551171] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-10. [2025-11-04T19:03:43.665589] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-11-04T19:03:43.682413] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-11. [2025-11-04T19:03:43.750882] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-11-04T19:03:43.778890] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T19:03:43.789068] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-09. [2025-11-04T19:03:43.792137] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-11. [2025-11-04T19:03:43.991081] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-11-04T19:03:44.000017] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-02. [2025-11-04T19:03:44.779277] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T19:03:44.780785] [INFO ] [REMOTE ] FUL: Full fetch complete (47852 records). [2025-11-04T19:03:44.780862] [DEBUG] [VALIDATOR ] FUL: Validating primary data. [2025-11-04T19:03:45.026045] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-11 (1.34 seconds). [2025-11-04T19:03:45.073146] [INFO ] [REMOTE ] FTV: Full fetch complete (76665 records). [2025-11-04T19:03:45.091591] [DEBUG] [VALIDATOR ] FTV: Validating primary data. [2025-11-04T19:03:45.314171] [DEBUG] [VALIDATOR ] FUL: All expected trading days are present. [2025-11-04T19:03:45.314332] [DEBUG] [VALIDATOR ] FUL: Primary data validation passed. [2025-11-04T19:03:45.325883] [INFO ] [MAIN ] FUL: Fetched primary data is valid. [2025-11-04T19:03:45.344838] [DEBUG] [LOCAL ] FUL: Saving primary data. [2025-11-04T19:03:45.486768] [DEBUG] [LOCAL ] FUL: Trimmed 47852 records to 31264 (keeping 200 days of data) [2025-11-04T19:03:45.503866] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-11-04T19:03:45.625426] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-03. [2025-11-04T19:03:45.775580] [DEBUG] [VALIDATOR ] FTV: All expected trading days are present. [2025-11-04T19:03:45.783078] [DEBUG] [VALIDATOR ] FTV: Primary data validation passed. [2025-11-04T19:03:45.784032] [INFO ] [MAIN ] FTV: Fetched primary data is valid. [2025-11-04T19:03:45.784356] [DEBUG] [LOCAL ] FTV: Saving primary data. [2025-11-04T19:03:45.799774] [DEBUG] [LOCAL ] FTV: Trimmed 76665 records to 53033 (keeping 200 days of data) [2025-11-04T19:03:45.888710] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-11-04T19:03:45.906028] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-10. [2025-11-04T19:03:45.975394] [DEBUG] [LOCAL ] FUL: Primary data saved (31264 records). [2025-11-04T19:03:46.004370] [INFO ] [MAIN ] FUL: Full history saved. [2025-11-04T19:03:46.012276] [INFO ] [MAIN ] Update Progress: 884/2367 symbols completed. FUL took 13.9s. Estimated remaining: 3h 27m 32s. Rate: 7.1 symbols/min [2025-11-04T19:03:46.325815] [INFO ] [MAIN ] FVR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:46.434331] [DEBUG] [LOCAL ] FVR: Loaded 20320 existing records from primary storage. [2025-11-04T19:03:46.442217] [INFO ] [MAIN ] FVR: Loaded 20320 existing records. [2025-11-04T19:03:46.442088] [DEBUG] [LOCAL ] FTV: Primary data saved (53033 records). [2025-11-04T19:03:46.506147] [INFO ] [MAIN ] FTV: Full history saved. [2025-11-04T19:03:46.518228] [INFO ] [MAIN ] Update Progress: 885/2367 symbols completed. FTV took 21.5s. Estimated remaining: 3h 27m 11s. Rate: 7.2 symbols/min [2025-11-04T19:03:46.728819] [WARN ] [MAIN ] FVR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:46.745123] [INFO ] [MAIN ] FVR: Wiping existing primary data before full refresh. [2025-11-04T19:03:46.768931] [INFO ] [LOCAL ] FVR: Starting deletion of primary data. [2025-11-04T19:03:46.800905] [INFO ] [MAIN ] FVRR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:46.801019] [DEBUG] [LOCAL ] FVR: Deleted primary data file. [2025-11-04T19:03:46.801143] [INFO ] [MAIN ] FVR: Starting full primary refresh. [2025-11-04T19:03:47.117808] [DEBUG] [LOCAL ] FVRR: Loaded 40654 existing records from primary storage. [2025-11-04T19:03:47.134062] [INFO ] [MAIN ] FVRR: Loaded 40654 existing records. [2025-11-04T19:03:47.245423] [INFO ] [REMOTE ] FVR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:47.278354] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-01. [2025-11-04T19:03:47.280007] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-11-04T19:03:47.357746] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-04. [2025-11-04T19:03:47.435313] [WARN ] [MAIN ] FVRR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:47.435500] [INFO ] [MAIN ] FVRR: Wiping existing primary data before full refresh. [2025-11-04T19:03:47.435549] [INFO ] [LOCAL ] FVRR: Starting deletion of primary data. [2025-11-04T19:03:47.435816] [DEBUG] [LOCAL ] FVRR: Deleted primary data file. [2025-11-04T19:03:47.435872] [INFO ] [MAIN ] FVRR: Starting full primary refresh. [2025-11-04T19:03:47.834320] [INFO ] [REMOTE ] FVRR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:47.835123] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-01. [2025-11-04T19:03:47.872593] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-11-04T19:03:47.884998] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-11. [2025-11-04T19:03:47.998388] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-11-04T19:03:48.002369] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-02. [2025-11-04T19:03:48.614070] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-11-04T19:03:48.619154] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-03. [2025-11-04T19:03:48.712590] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:03:48.722979] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-05. [2025-11-04T19:03:48.886048] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-11-04T19:03:48.894745] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-02. [2025-11-04T19:03:49.358817] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T19:03:49.363945] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-04. [2025-11-04T19:03:49.438362] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-11 (1.55 seconds). [2025-11-04T19:03:49.440415] [INFO ] [REMOTE ] FUBO: Full fetch complete (77865 records). [2025-11-04T19:03:49.440518] [DEBUG] [VALIDATOR ] FUBO: Validating primary data. [2025-11-04T19:03:49.848067] [DEBUG] [VALIDATOR ] FUBO: All expected trading days are present. [2025-11-04T19:03:49.848186] [DEBUG] [VALIDATOR ] FUBO: Primary data validation passed. [2025-11-04T19:03:49.849516] [INFO ] [MAIN ] FUBO: Fetched primary data is valid. [2025-11-04T19:03:49.849734] [DEBUG] [LOCAL ] FUBO: Saving primary data. [2025-11-04T19:03:49.851619] [DEBUG] [LOCAL ] FUBO: Trimmed 77865 records to 53715 (keeping 200 days of data) [2025-11-04T19:03:49.873389] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-11-04T19:03:49.962598] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-03. [2025-11-04T19:03:50.068010] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T19:03:50.128975] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-05. [2025-11-04T19:03:50.223773] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-11-04T19:03:50.244777] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-06. [2025-11-04T19:03:50.258074] [DEBUG] [LOCAL ] FUBO: Primary data saved (53715 records). [2025-11-04T19:03:50.258234] [INFO ] [MAIN ] FUBO: Full history saved. [2025-11-04T19:03:50.258324] [INFO ] [MAIN ] Update Progress: 886/2367 symbols completed. FUBO took 22.2s. Estimated remaining: 3h 26m 55s. Rate: 7.2 symbols/min [2025-11-04T19:03:50.426441] [INFO ] [MAIN ] FXED: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:50.430170] [DEBUG] [LOCAL ] FXED: Loaded 1205 existing records from primary storage. [2025-11-04T19:03:50.430338] [INFO ] [MAIN ] FXED: Loaded 1205 existing records. [2025-11-04T19:03:50.610638] [WARN ] [MAIN ] FXED: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:50.610886] [INFO ] [MAIN ] FXED: Wiping existing primary data before full refresh. [2025-11-04T19:03:50.610934] [INFO ] [LOCAL ] FXED: Starting deletion of primary data. [2025-11-04T19:03:50.611064] [DEBUG] [LOCAL ] FXED: Deleted primary data file. [2025-11-04T19:03:50.611103] [INFO ] [MAIN ] FXED: Starting full primary refresh. [2025-11-04T19:03:50.986665] [INFO ] [REMOTE ] FXED: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:50.989798] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-01. [2025-11-04T19:03:50.992406] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-11-04T19:03:50.999614] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-06. [2025-11-04T19:03:51.053392] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T19:03:51.062408] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-04. [2025-11-04T19:03:51.271730] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-01 (0.28 seconds). [2025-11-04T19:03:51.273068] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-02. [2025-11-04T19:03:51.512702] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-02 (0.24 seconds). [2025-11-04T19:03:51.513725] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-03. [2025-11-04T19:03:51.652209] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-11-04T19:03:51.664086] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-07. [2025-11-04T19:03:51.781990] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T19:03:51.788209] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-07. [2025-11-04T19:03:51.841028] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-11-04T19:03:51.842041] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-04. [2025-11-04T19:03:52.141051] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-11-04T19:03:52.145839] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-05. [2025-11-04T19:03:52.144224] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T19:03:52.156567] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-05. [2025-11-04T19:03:52.401993] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-05 (0.26 seconds). [2025-11-04T19:03:52.403517] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-06. [2025-11-04T19:03:52.419225] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-11-04T19:03:52.425071] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-08. [2025-11-04T19:03:52.684670] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-06 (0.28 seconds). [2025-11-04T19:03:52.685755] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-07. [2025-11-04T19:03:52.910332] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T19:03:52.925241] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-08. [2025-11-04T19:03:52.925357] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-07 (0.24 seconds). [2025-11-04T19:03:52.926598] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-08. [2025-11-04T19:03:53.139425] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T19:03:53.148824] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-08 (0.22 seconds). [2025-11-04T19:03:53.148698] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-09. [2025-11-04T19:03:53.150191] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-09. [2025-11-04T19:03:53.245632] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-11-04T19:03:53.254720] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-06. [2025-11-04T19:03:53.422070] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-09 (0.27 seconds). [2025-11-04T19:03:53.423123] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-10. [2025-11-04T19:03:53.705662] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-10 (0.28 seconds). [2025-11-04T19:03:53.707317] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-11. [2025-11-04T19:03:53.780334] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:03:53.785465] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-10. [2025-11-04T19:03:54.070828] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T19:03:54.071677] [INFO ] [REMOTE ] FXED: Full fetch complete (1780 records). [2025-11-04T19:03:54.071756] [DEBUG] [VALIDATOR ] FXED: Validating primary data. [2025-11-04T19:03:54.425926] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T19:03:54.465154] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-07. [2025-11-04T19:03:54.575086] [DEBUG] [VALIDATOR ] FXED: All expected trading days are present. [2025-11-04T19:03:54.578961] [DEBUG] [VALIDATOR ] FXED: Primary data validation passed. [2025-11-04T19:03:54.580996] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-11-04T19:03:54.581419] [INFO ] [MAIN ] FXED: Fetched primary data is valid. [2025-11-04T19:03:54.597685] [DEBUG] [LOCAL ] FXED: Saving primary data. [2025-11-04T19:03:54.608931] [DEBUG] [LOCAL ] FXED: Trimmed 1780 records to 1218 (keeping 200 days of data) [2025-11-04T19:03:54.609539] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T19:03:54.615906] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-09. [2025-11-04T19:03:54.622823] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-11. [2025-11-04T19:03:54.624958] [DEBUG] [LOCAL ] FXED: Primary data saved (1218 records). [2025-11-04T19:03:54.627171] [INFO ] [MAIN ] FXED: Full history saved. [2025-11-04T19:03:54.627311] [INFO ] [MAIN ] Update Progress: 887/2367 symbols completed. FXED took 4.4s. Estimated remaining: 3h 26m 40s. Rate: 7.2 symbols/min [2025-11-04T19:03:54.805406] [INFO ] [MAIN ] G: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:54.852955] [DEBUG] [LOCAL ] G: Loaded 50130 existing records from primary storage. [2025-11-04T19:03:54.853391] [INFO ] [MAIN ] G: Loaded 50130 existing records. [2025-11-04T19:03:55.049595] [WARN ] [MAIN ] G: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:55.049954] [INFO ] [MAIN ] G: Wiping existing primary data before full refresh. [2025-11-04T19:03:55.050004] [INFO ] [LOCAL ] G: Starting deletion of primary data. [2025-11-04T19:03:55.050980] [DEBUG] [LOCAL ] G: Deleted primary data file. [2025-11-04T19:03:55.051039] [INFO ] [MAIN ] G: Starting full primary refresh. [2025-11-04T19:03:55.247373] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T19:03:55.294299] [INFO ] [REMOTE ] FVR: Full fetch complete (29130 records). [2025-11-04T19:03:55.318762] [DEBUG] [VALIDATOR ] FVR: Validating primary data. [2025-11-04T19:03:55.601397] [INFO ] [REMOTE ] G: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:55.607913] [INFO ] [REMOTE ] G: Fetching primary data for 2025-01. [2025-11-04T19:03:55.884562] [DEBUG] [VALIDATOR ] FVR: All expected trading days are present. [2025-11-04T19:03:55.884705] [DEBUG] [VALIDATOR ] FVR: Primary data validation passed. [2025-11-04T19:03:55.885551] [INFO ] [MAIN ] FVR: Fetched primary data is valid. [2025-11-04T19:03:55.885599] [DEBUG] [LOCAL ] FVR: Saving primary data. [2025-11-04T19:03:55.887015] [DEBUG] [LOCAL ] FVR: Trimmed 29130 records to 20271 (keeping 200 days of data) [2025-11-04T19:03:56.031459] [DEBUG] [LOCAL ] FVR: Primary data saved (20271 records). [2025-11-04T19:03:56.031690] [INFO ] [MAIN ] FVR: Full history saved. [2025-11-04T19:03:56.031820] [INFO ] [MAIN ] Update Progress: 888/2367 symbols completed. FVR took 10.0s. Estimated remaining: 3h 26m 20s. Rate: 7.2 symbols/min [2025-11-04T19:03:56.260416] [INFO ] [MAIN ] GAB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:03:56.256186] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-11-04T19:03:56.271610] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-08. [2025-11-04T19:03:56.300187] [DEBUG] [LOCAL ] GAB: Loaded 26588 existing records from primary storage. [2025-11-04T19:03:56.300562] [INFO ] [MAIN ] GAB: Loaded 26588 existing records. [2025-11-04T19:03:56.493376] [WARN ] [MAIN ] GAB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:03:56.493964] [INFO ] [MAIN ] GAB: Wiping existing primary data before full refresh. [2025-11-04T19:03:56.494050] [INFO ] [LOCAL ] GAB: Starting deletion of primary data. [2025-11-04T19:03:56.495088] [DEBUG] [LOCAL ] GAB: Deleted primary data file. [2025-11-04T19:03:56.495224] [INFO ] [MAIN ] GAB: Starting full primary refresh. [2025-11-04T19:03:56.677513] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-11-04T19:03:56.939978] [INFO ] [REMOTE ] GAB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:03:56.966929] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T19:03:56.967002] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-01. [2025-11-04T19:03:57.426840] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T19:03:57.719032] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T19:03:57.767072] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-02. [2025-11-04T19:03:58.482723] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T19:03:58.567080] [INFO ] [REMOTE ] G: Fetching primary data for 2025-02. [2025-11-04T19:03:59.367094] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-10. [2025-11-04T19:03:59.862694] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T19:04:00.167084] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-09. [2025-11-04T19:04:00.864106] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-11-04T19:04:00.967115] [INFO ] [REMOTE ] G: Fetching primary data for 2025-03. [2025-11-04T19:04:01.279756] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T19:04:01.767050] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-03. [2025-11-04T19:04:02.534433] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-11-04T19:04:02.567040] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-10. [2025-11-04T19:04:02.624590] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-11-04T19:04:03.367042] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-04. [2025-11-04T19:04:03.821437] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T19:04:04.167007] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-11. [2025-11-04T19:04:04.215116] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-11-04T19:04:04.967048] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-11. [2025-11-04T19:04:05.206111] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T19:04:05.207994] [INFO ] [REMOTE ] FVRR: Full fetch complete (59375 records). [2025-11-04T19:04:05.208104] [DEBUG] [VALIDATOR ] FVRR: Validating primary data. [2025-11-04T19:04:05.582821] [DEBUG] [VALIDATOR ] FVRR: All expected trading days are present. [2025-11-04T19:04:05.582904] [DEBUG] [VALIDATOR ] FVRR: Primary data validation passed. [2025-11-04T19:04:05.583659] [INFO ] [MAIN ] FVRR: Fetched primary data is valid. [2025-11-04T19:04:05.583724] [DEBUG] [LOCAL ] FVRR: Saving primary data. [2025-11-04T19:04:05.584924] [DEBUG] [LOCAL ] FVRR: Trimmed 59375 records to 40810 (keeping 200 days of data) [2025-11-04T19:04:05.779285] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-05. [2025-11-04T19:04:05.820371] [DEBUG] [LOCAL ] FVRR: Primary data saved (40810 records). [2025-11-04T19:04:05.820497] [INFO ] [MAIN ] FVRR: Full history saved. [2025-11-04T19:04:05.820572] [INFO ] [MAIN ] Update Progress: 889/2367 symbols completed. FVRR took 19.3s. Estimated remaining: 3h 26m 14s. Rate: 7.2 symbols/min [2025-11-04T19:04:06.002157] [INFO ] [MAIN ] GAM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:04:06.007120] [DEBUG] [LOCAL ] GAM: Loaded 3697 existing records from primary storage. [2025-11-04T19:04:06.007279] [INFO ] [MAIN ] GAM: Loaded 3697 existing records. [2025-11-04T19:04:06.193582] [WARN ] [MAIN ] GAM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:04:06.193853] [INFO ] [MAIN ] GAM: Wiping existing primary data before full refresh. [2025-11-04T19:04:06.193901] [INFO ] [LOCAL ] GAM: Starting deletion of primary data. [2025-11-04T19:04:06.194169] [DEBUG] [LOCAL ] GAM: Deleted primary data file. [2025-11-04T19:04:06.194230] [INFO ] [MAIN ] GAM: Starting full primary refresh. [2025-11-04T19:04:06.306524] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-11 (1.34 seconds). [2025-11-04T19:04:06.343708] [INFO ] [REMOTE ] FUN: Full fetch complete (73955 records). [2025-11-04T19:04:06.354423] [DEBUG] [VALIDATOR ] FUN: Validating primary data. [2025-11-04T19:04:06.572251] [INFO ] [REMOTE ] G: Fetching primary data for 2025-04. [2025-11-04T19:04:06.724239] [INFO ] [REMOTE ] GAM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:04:06.965576] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T19:04:07.012962] [DEBUG] [VALIDATOR ] FUN: All expected trading days are present. [2025-11-04T19:04:07.013044] [DEBUG] [VALIDATOR ] FUN: Primary data validation passed. [2025-11-04T19:04:07.014014] [INFO ] [MAIN ] FUN: Fetched primary data is valid. [2025-11-04T19:04:07.014079] [DEBUG] [LOCAL ] FUN: Saving primary data. [2025-11-04T19:04:07.015390] [DEBUG] [LOCAL ] FUN: Trimmed 73955 records to 52281 (keeping 200 days of data) [2025-11-04T19:04:07.353046] [DEBUG] [LOCAL ] FUN: Primary data saved (52281 records). [2025-11-04T19:04:07.353191] [INFO ] [MAIN ] FUN: Full history saved. [2025-11-04T19:04:07.353285] [INFO ] [MAIN ] Update Progress: 890/2367 symbols completed. FUN took 25.8s. Estimated remaining: 3h 25m 54s. Rate: 7.2 symbols/min [2025-11-04T19:04:07.375714] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-06. [2025-11-04T19:04:07.530296] [INFO ] [MAIN ] GAP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:04:07.576803] [DEBUG] [LOCAL ] GAP: Loaded 53653 existing records from primary storage. [2025-11-04T19:04:07.577196] [INFO ] [MAIN ] GAP: Loaded 53653 existing records. [2025-11-04T19:04:07.751878] [WARN ] [MAIN ] GAP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:04:07.752078] [INFO ] [MAIN ] GAP: Wiping existing primary data before full refresh. [2025-11-04T19:04:07.752124] [INFO ] [LOCAL ] GAP: Starting deletion of primary data. [2025-11-04T19:04:07.753037] [DEBUG] [LOCAL ] GAP: Deleted primary data file. [2025-11-04T19:04:07.753094] [INFO ] [MAIN ] GAP: Starting full primary refresh. [2025-11-04T19:04:08.111219] [INFO ] [REMOTE ] GAP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:04:08.138272] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-11-04T19:04:08.166997] [INFO ] [REMOTE ] G: Fetching primary data for 2025-05. [2025-11-04T19:04:08.225662] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T19:04:08.967035] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-01. [2025-11-04T19:04:09.350402] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-01 (0.38 seconds). [2025-11-04T19:04:09.387995] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T19:04:09.767047] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-01. [2025-11-04T19:04:10.567065] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-02. [2025-11-04T19:04:11.056118] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-11-04T19:04:11.061619] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T19:04:11.367056] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-07. [2025-11-04T19:04:12.167023] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-03. [2025-11-04T19:04:12.231140] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-11-04T19:04:12.660812] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-11-04T19:04:12.967133] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-08. [2025-11-04T19:04:13.624309] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-11-04T19:04:13.767013] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-09. [2025-11-04T19:04:14.567042] [INFO ] [REMOTE ] G: Fetching primary data for 2025-06. [2025-11-04T19:04:14.842825] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T19:04:15.367055] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-10. [2025-11-04T19:04:15.685017] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T19:04:16.159438] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T19:04:16.167726] [INFO ] [REMOTE ] G: Fetching primary data for 2025-07. [2025-11-04T19:04:16.967049] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-11. [2025-11-04T19:04:17.369586] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T19:04:17.731156] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-11 (0.76 seconds). [2025-11-04T19:04:17.732773] [INFO ] [REMOTE ] GAB: Full fetch complete (39246 records). [2025-11-04T19:04:17.732861] [DEBUG] [VALIDATOR ] GAB: Validating primary data. [2025-11-04T19:04:17.772972] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-02. [2025-11-04T19:04:18.111723] [DEBUG] [VALIDATOR ] GAB: All expected trading days are present. [2025-11-04T19:04:18.111794] [DEBUG] [VALIDATOR ] GAB: Primary data validation passed. [2025-11-04T19:04:18.112404] [INFO ] [MAIN ] GAB: Fetched primary data is valid. [2025-11-04T19:04:18.112449] [DEBUG] [LOCAL ] GAB: Saving primary data. [2025-11-04T19:04:18.113308] [DEBUG] [LOCAL ] GAB: Trimmed 39246 records to 26540 (keeping 200 days of data) [2025-11-04T19:04:18.263285] [DEBUG] [LOCAL ] GAB: Primary data saved (26540 records). [2025-11-04T19:04:18.263411] [INFO ] [MAIN ] GAB: Full history saved. [2025-11-04T19:04:18.263492] [INFO ] [MAIN ] Update Progress: 891/2367 symbols completed. GAB took 22.2s. Estimated remaining: 3h 25m 50s. Rate: 7.2 symbols/min [2025-11-04T19:04:18.435462] [INFO ] [MAIN ] GATX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:04:18.454706] [DEBUG] [LOCAL ] GATX: Loaded 17634 existing records from primary storage. [2025-11-04T19:04:18.454926] [INFO ] [MAIN ] GATX: Loaded 17634 existing records. [2025-11-04T19:04:18.572236] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-04. [2025-11-04T19:04:18.637315] [WARN ] [MAIN ] GATX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:04:18.637530] [INFO ] [MAIN ] GATX: Wiping existing primary data before full refresh. [2025-11-04T19:04:18.637581] [INFO ] [LOCAL ] GATX: Starting deletion of primary data. [2025-11-04T19:04:18.637820] [DEBUG] [LOCAL ] GATX: Deleted primary data file. [2025-11-04T19:04:18.637877] [INFO ] [MAIN ] GATX: Starting full primary refresh. [2025-11-04T19:04:19.000236] [INFO ] [REMOTE ] GATX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:04:19.036565] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T19:04:19.083170] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-11-04T19:04:19.367071] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-05. [2025-11-04T19:04:19.695036] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-11-04T19:04:20.167038] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-06. [2025-11-04T19:04:20.530504] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-06 (0.36 seconds). [2025-11-04T19:04:20.967101] [INFO ] [REMOTE ] G: Fetching primary data for 2025-08. [2025-11-04T19:04:21.767063] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-01. [2025-11-04T19:04:22.111202] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T19:04:22.511993] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-11-04T19:04:22.567009] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-02. [2025-11-04T19:04:23.107386] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-11-04T19:04:23.367100] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-03. [2025-11-04T19:04:23.990992] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-11-04T19:04:24.167076] [INFO ] [REMOTE ] G: Fetching primary data for 2025-09. [2025-11-04T19:04:24.967057] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-07. [2025-11-04T19:04:25.378274] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-11-04T19:04:25.384165] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T19:04:25.767074] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-08. [2025-11-04T19:04:26.143563] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-11-04T19:04:26.567051] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-03. [2025-11-04T19:04:27.367100] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-04. [2025-11-04T19:04:27.966406] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T19:04:28.099031] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T19:04:28.166999] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-04. [2025-11-04T19:04:28.967076] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-09. [2025-11-04T19:04:29.293618] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-09 (0.33 seconds). [2025-11-04T19:04:29.548464] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T19:04:29.767059] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-05. [2025-11-04T19:04:30.422961] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T19:04:30.567016] [INFO ] [REMOTE ] G: Fetching primary data for 2025-10. [2025-11-04T19:04:31.367125] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-05. [2025-11-04T19:04:32.080428] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T19:04:32.167004] [INFO ] [REMOTE ] G: Fetching primary data for 2025-11. [2025-11-04T19:04:32.849480] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T19:04:32.967053] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-06. [2025-11-04T19:04:33.191516] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T19:04:33.194415] [INFO ] [REMOTE ] G: Full fetch complete (72557 records). [2025-11-04T19:04:33.194601] [DEBUG] [VALIDATOR ] G: Validating primary data. [2025-11-04T19:04:33.753520] [DEBUG] [VALIDATOR ] G: All expected trading days are present. [2025-11-04T19:04:33.753693] [DEBUG] [VALIDATOR ] G: Primary data validation passed. [2025-11-04T19:04:33.755306] [INFO ] [MAIN ] G: Fetched primary data is valid. [2025-11-04T19:04:33.755400] [DEBUG] [LOCAL ] G: Saving primary data. [2025-11-04T19:04:33.757365] [DEBUG] [LOCAL ] G: Trimmed 72557 records to 50147 (keeping 200 days of data) [2025-11-04T19:04:33.827074] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-10. [2025-11-04T19:04:34.106561] [DEBUG] [LOCAL ] G: Primary data saved (50147 records). [2025-11-04T19:04:34.106783] [INFO ] [MAIN ] G: Full history saved. [2025-11-04T19:04:34.106886] [INFO ] [MAIN ] Update Progress: 892/2367 symbols completed. G took 39.5s. Estimated remaining: 3h 25m 54s. Rate: 7.2 symbols/min [2025-11-04T19:04:34.286209] [INFO ] [MAIN ] GBAB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:04:34.292630] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-11-04T19:04:34.305983] [DEBUG] [LOCAL ] GBAB: Loaded 12279 existing records from primary storage. [2025-11-04T19:04:34.306236] [INFO ] [MAIN ] GBAB: Loaded 12279 existing records. [2025-11-04T19:04:34.487749] [WARN ] [MAIN ] GBAB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:04:34.487999] [INFO ] [MAIN ] GBAB: Wiping existing primary data before full refresh. [2025-11-04T19:04:34.488054] [INFO ] [LOCAL ] GBAB: Starting deletion of primary data. [2025-11-04T19:04:34.488364] [DEBUG] [LOCAL ] GBAB: Deleted primary data file. [2025-11-04T19:04:34.488424] [INFO ] [MAIN ] GBAB: Starting full primary refresh. [2025-11-04T19:04:34.568590] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-06. [2025-11-04T19:04:34.804071] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-11-04T19:04:34.895335] [INFO ] [REMOTE ] GBAB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:04:35.267229] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T19:04:35.367057] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-07. [2025-11-04T19:04:36.167044] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-11. [2025-11-04T19:04:36.528320] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T19:04:36.529042] [INFO ] [REMOTE ] GAM: Full fetch complete (6382 records). [2025-11-04T19:04:36.529110] [DEBUG] [VALIDATOR ] GAM: Validating primary data. [2025-11-04T19:04:36.891028] [DEBUG] [VALIDATOR ] GAM: All expected trading days are present. [2025-11-04T19:04:36.891149] [DEBUG] [VALIDATOR ] GAM: Primary data validation passed. [2025-11-04T19:04:36.891343] [INFO ] [MAIN ] GAM: Fetched primary data is valid. [2025-11-04T19:04:36.891478] [DEBUG] [LOCAL ] GAM: Saving primary data. [2025-11-04T19:04:36.892090] [DEBUG] [LOCAL ] GAM: Trimmed 6382 records to 3702 (keeping 200 days of data) [2025-11-04T19:04:36.914879] [DEBUG] [LOCAL ] GAM: Primary data saved (3702 records). [2025-11-04T19:04:36.915076] [INFO ] [MAIN ] GAM: Full history saved. [2025-11-04T19:04:36.915164] [INFO ] [MAIN ] Update Progress: 893/2367 symbols completed. GAM took 31.1s. Estimated remaining: 3h 25m 36s. Rate: 7.2 symbols/min [2025-11-04T19:04:36.980892] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-01. [2025-11-04T19:04:37.064785] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-11-04T19:04:37.142749] [INFO ] [MAIN ] GBCI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:04:37.179930] [DEBUG] [LOCAL ] GBCI: Loaded 41260 existing records from primary storage. [2025-11-04T19:04:37.180229] [INFO ] [MAIN ] GBCI: Loaded 41260 existing records. [2025-11-04T19:04:37.370846] [WARN ] [MAIN ] GBCI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:04:37.371059] [INFO ] [MAIN ] GBCI: Wiping existing primary data before full refresh. [2025-11-04T19:04:37.371107] [INFO ] [LOCAL ] GBCI: Starting deletion of primary data. [2025-11-04T19:04:37.371288] [DEBUG] [LOCAL ] GBCI: Deleted primary data file. [2025-11-04T19:04:37.371325] [INFO ] [MAIN ] GBCI: Starting full primary refresh. [2025-11-04T19:04:37.677837] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-11-04T19:04:37.754794] [INFO ] [REMOTE ] GBCI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:04:37.767001] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-02. [2025-11-04T19:04:38.257475] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-11-04T19:04:38.567127] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-03. [2025-11-04T19:04:39.080864] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-11-04T19:04:39.367063] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-04. [2025-11-04T19:04:39.943771] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-11-04T19:04:40.167025] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-01. [2025-11-04T19:04:40.967025] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-08. [2025-11-04T19:04:41.117417] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T19:04:41.767080] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-07. [2025-11-04T19:04:42.480472] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T19:04:42.567011] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-05. [2025-11-04T19:04:42.611982] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-11-04T19:04:43.101125] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-11-04T19:04:43.367051] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-06. [2025-11-04T19:04:44.071545] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T19:04:44.167044] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-09. [2025-11-04T19:04:44.967081] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-02. [2025-11-04T19:04:45.628469] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-11-04T19:04:45.767040] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-10. [2025-11-04T19:04:45.972745] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-11-04T19:04:46.567059] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-07. [2025-11-04T19:04:47.080282] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-11-04T19:04:47.190931] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-11-04T19:04:47.367009] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-03. [2025-11-04T19:04:48.167034] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-08. [2025-11-04T19:04:48.441238] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T19:04:48.604668] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T19:04:48.967053] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-04. [2025-11-04T19:04:49.767079] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-09. [2025-11-04T19:04:50.050082] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T19:04:50.274139] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-11-04T19:04:50.567120] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-11. [2025-11-04T19:04:51.367057] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-08. [2025-11-04T19:04:51.865960] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T19:04:51.868363] [INFO ] [REMOTE ] GAP: Full fetch complete (77817 records). [2025-11-04T19:04:51.868481] [DEBUG] [VALIDATOR ] GAP: Validating primary data. [2025-11-04T19:04:52.086360] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-11-04T19:04:52.172258] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-05. [2025-11-04T19:04:52.359021] [DEBUG] [VALIDATOR ] GAP: All expected trading days are present. [2025-11-04T19:04:52.359131] [DEBUG] [VALIDATOR ] GAP: Primary data validation passed. [2025-11-04T19:04:52.360200] [INFO ] [MAIN ] GAP: Fetched primary data is valid. [2025-11-04T19:04:52.360284] [DEBUG] [LOCAL ] GAP: Saving primary data. [2025-11-04T19:04:52.361900] [DEBUG] [LOCAL ] GAP: Trimmed 77817 records to 53653 (keeping 200 days of data) [2025-11-04T19:04:52.720638] [DEBUG] [LOCAL ] GAP: Primary data saved (53653 records). [2025-11-04T19:04:52.720803] [INFO ] [MAIN ] GAP: Full history saved. [2025-11-04T19:04:52.720885] [INFO ] [MAIN ] Update Progress: 894/2367 symbols completed. GAP took 45.4s. Estimated remaining: 3h 25m 40s. Rate: 7.2 symbols/min [2025-11-04T19:04:52.925217] [INFO ] [MAIN ] GBTG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:04:52.961557] [DEBUG] [LOCAL ] GBTG: Loaded 39763 existing records from primary storage. [2025-11-04T19:04:52.961896] [INFO ] [MAIN ] GBTG: Loaded 39763 existing records. [2025-11-04T19:04:52.975383] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-10. [2025-11-04T19:04:53.145459] [WARN ] [MAIN ] GBTG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:04:53.145713] [INFO ] [MAIN ] GBTG: Wiping existing primary data before full refresh. [2025-11-04T19:04:53.145770] [INFO ] [LOCAL ] GBTG: Starting deletion of primary data. [2025-11-04T19:04:53.146389] [DEBUG] [LOCAL ] GBTG: Deleted primary data file. [2025-11-04T19:04:53.146445] [INFO ] [MAIN ] GBTG: Starting full primary refresh. [2025-11-04T19:04:53.523334] [INFO ] [REMOTE ] GBTG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:04:53.527159] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-11-04T19:04:53.620695] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-11-04T19:04:53.767052] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-01. [2025-11-04T19:04:54.567100] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-11. [2025-11-04T19:04:54.758240] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T19:04:55.162294] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-11 (0.60 seconds). [2025-11-04T19:04:55.163265] [INFO ] [REMOTE ] GBAB: Full fetch complete (18486 records). [2025-11-04T19:04:55.163332] [DEBUG] [VALIDATOR ] GBAB: Validating primary data. [2025-11-04T19:04:55.372238] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-02. [2025-11-04T19:04:55.521027] [DEBUG] [VALIDATOR ] GBAB: All expected trading days are present. [2025-11-04T19:04:55.521093] [DEBUG] [VALIDATOR ] GBAB: Primary data validation passed. [2025-11-04T19:04:55.521450] [INFO ] [MAIN ] GBAB: Fetched primary data is valid. [2025-11-04T19:04:55.521488] [DEBUG] [LOCAL ] GBAB: Saving primary data. [2025-11-04T19:04:55.522013] [DEBUG] [LOCAL ] GBAB: Trimmed 18486 records to 12261 (keeping 200 days of data) [2025-11-04T19:04:55.592523] [DEBUG] [LOCAL ] GBAB: Primary data saved (12261 records). [2025-11-04T19:04:55.592633] [INFO ] [MAIN ] GBAB: Full history saved. [2025-11-04T19:04:55.592753] [INFO ] [MAIN ] Update Progress: 895/2367 symbols completed. GBAB took 21.5s. Estimated remaining: 3h 25m 23s. Rate: 7.2 symbols/min [2025-11-04T19:04:55.756584] [INFO ] [MAIN ] GBX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:04:55.780217] [DEBUG] [LOCAL ] GBX: Loaded 27463 existing records from primary storage. [2025-11-04T19:04:55.780450] [INFO ] [MAIN ] GBX: Loaded 27463 existing records. [2025-11-04T19:04:55.955683] [WARN ] [MAIN ] GBX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:04:55.955909] [INFO ] [MAIN ] GBX: Wiping existing primary data before full refresh. [2025-11-04T19:04:55.955961] [INFO ] [LOCAL ] GBX: Starting deletion of primary data. [2025-11-04T19:04:55.956564] [DEBUG] [LOCAL ] GBX: Deleted primary data file. [2025-11-04T19:04:55.956620] [INFO ] [MAIN ] GBX: Starting full primary refresh. [2025-11-04T19:04:56.175723] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-06. [2025-11-04T19:04:56.318538] [INFO ] [REMOTE ] GBX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:04:56.422447] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T19:04:56.967062] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-09. [2025-11-04T19:04:57.421812] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-11-04T19:04:57.767013] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-07. [2025-11-04T19:04:57.882957] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T19:04:58.567071] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-10. [2025-11-04T19:04:59.022030] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T19:04:59.237637] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-11-04T19:04:59.367007] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-11. [2025-11-04T19:05:00.141794] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T19:05:00.142917] [INFO ] [REMOTE ] GATX: Full fetch complete (25244 records). [2025-11-04T19:05:00.142988] [DEBUG] [VALIDATOR ] GATX: Validating primary data. [2025-11-04T19:05:00.172739] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-01. [2025-11-04T19:05:00.514057] [DEBUG] [VALIDATOR ] GATX: All expected trading days are present. [2025-11-04T19:05:00.514130] [DEBUG] [VALIDATOR ] GATX: Primary data validation passed. [2025-11-04T19:05:00.514564] [INFO ] [MAIN ] GATX: Fetched primary data is valid. [2025-11-04T19:05:00.514623] [DEBUG] [LOCAL ] GATX: Saving primary data. [2025-11-04T19:05:00.515299] [DEBUG] [LOCAL ] GATX: Trimmed 25244 records to 17685 (keeping 200 days of data) [2025-11-04T19:05:00.623505] [DEBUG] [LOCAL ] GATX: Primary data saved (17685 records). [2025-11-04T19:05:00.623626] [INFO ] [MAIN ] GATX: Full history saved. [2025-11-04T19:05:00.623735] [INFO ] [MAIN ] Update Progress: 896/2367 symbols completed. GATX took 42.4s. Estimated remaining: 3h 25m 9s. Rate: 7.2 symbols/min [2025-11-04T19:05:00.791467] [INFO ] [MAIN ] GCI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:05:00.827338] [DEBUG] [LOCAL ] GCI: Loaded 43651 existing records from primary storage. [2025-11-04T19:05:00.827681] [INFO ] [MAIN ] GCI: Loaded 43651 existing records. [2025-11-04T19:05:00.972224] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-08. [2025-11-04T19:05:01.007250] [WARN ] [MAIN ] GCI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:05:01.007474] [INFO ] [MAIN ] GCI: Wiping existing primary data before full refresh. [2025-11-04T19:05:01.007529] [INFO ] [LOCAL ] GCI: Starting deletion of primary data. [2025-11-04T19:05:01.008189] [DEBUG] [LOCAL ] GCI: Deleted primary data file. [2025-11-04T19:05:01.008245] [INFO ] [MAIN ] GCI: Starting full primary refresh. [2025-11-04T19:05:01.373725] [INFO ] [REMOTE ] GCI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:05:01.390929] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-11-04T19:05:01.767040] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-01. [2025-11-04T19:05:02.088129] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-11-04T19:05:02.567057] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-02. [2025-11-04T19:05:02.884926] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T19:05:03.367017] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-02. [2025-11-04T19:05:03.406532] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-11-04T19:05:04.167024] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-03. [2025-11-04T19:05:04.550138] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T19:05:04.967110] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-09. [2025-11-04T19:05:05.128196] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T19:05:05.767039] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-03. [2025-11-04T19:05:06.053355] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-11-04T19:05:06.567041] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-04. [2025-11-04T19:05:06.965070] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T19:05:07.367031] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-04. [2025-11-04T19:05:07.963914] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T19:05:08.167023] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-03. [2025-11-04T19:05:08.457522] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T19:05:08.967052] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-05. [2025-11-04T19:05:09.283455] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T19:05:09.773467] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-04. [2025-11-04T19:05:09.775270] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T19:05:10.567037] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-10. [2025-11-04T19:05:10.878154] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T19:05:11.367033] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-06. [2025-11-04T19:05:11.935574] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T19:05:12.089700] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-11-04T19:05:12.167004] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-07. [2025-11-04T19:05:12.967036] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-11. [2025-11-04T19:05:13.301533] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T19:05:13.767077] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-05. [2025-11-04T19:05:14.097634] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:05:14.099718] [INFO ] [REMOTE ] GBCI: Full fetch complete (59585 records). [2025-11-04T19:05:14.099864] [DEBUG] [VALIDATOR ] GBCI: Validating primary data. [2025-11-04T19:05:14.510808] [DEBUG] [VALIDATOR ] GBCI: All expected trading days are present. [2025-11-04T19:05:14.510901] [DEBUG] [VALIDATOR ] GBCI: Primary data validation passed. [2025-11-04T19:05:14.511684] [INFO ] [MAIN ] GBCI: Fetched primary data is valid. [2025-11-04T19:05:14.511746] [DEBUG] [LOCAL ] GBCI: Saving primary data. [2025-11-04T19:05:14.513000] [DEBUG] [LOCAL ] GBCI: Trimmed 59585 records to 41291 (keeping 200 days of data) [2025-11-04T19:05:14.572273] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-05. [2025-11-04T19:05:14.766095] [DEBUG] [LOCAL ] GBCI: Primary data saved (41291 records). [2025-11-04T19:05:14.766375] [INFO ] [MAIN ] GBCI: Full history saved. [2025-11-04T19:05:14.766523] [INFO ] [MAIN ] Update Progress: 897/2367 symbols completed. GBCI took 37.8s. Estimated remaining: 3h 25m 10s. Rate: 7.2 symbols/min [2025-11-04T19:05:14.967920] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T19:05:14.972383] [INFO ] [MAIN ] GCO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:05:15.008353] [DEBUG] [LOCAL ] GCO: Loaded 21336 existing records from primary storage. [2025-11-04T19:05:15.008688] [INFO ] [MAIN ] GCO: Loaded 21336 existing records. [2025-11-04T19:05:15.190947] [WARN ] [MAIN ] GCO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:05:15.191198] [INFO ] [MAIN ] GCO: Wiping existing primary data before full refresh. [2025-11-04T19:05:15.191247] [INFO ] [LOCAL ] GCO: Starting deletion of primary data. [2025-11-04T19:05:15.191617] [DEBUG] [LOCAL ] GCO: Deleted primary data file. [2025-11-04T19:05:15.191708] [INFO ] [MAIN ] GCO: Starting full primary refresh. [2025-11-04T19:05:15.369673] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-06. [2025-11-04T19:05:15.554189] [INFO ] [REMOTE ] GCO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:05:15.674780] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-11-04T19:05:16.167025] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-08. [2025-11-04T19:05:16.684075] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T19:05:16.958372] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T19:05:16.967002] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-07. [2025-11-04T19:05:17.767091] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-06. [2025-11-04T19:05:18.239244] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T19:05:18.567046] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-08. [2025-11-04T19:05:18.809153] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T19:05:19.367043] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-09. [2025-11-04T19:05:19.811999] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T19:05:20.131418] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-11-04T19:05:20.167013] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-10. [2025-11-04T19:05:20.967071] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-07. [2025-11-04T19:05:21.079733] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-11-04T19:05:21.767099] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-09. [2025-11-04T19:05:22.121391] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-11-04T19:05:22.567080] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-01. [2025-11-04T19:05:22.842728] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T19:05:23.274118] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T19:05:23.367045] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-02. [2025-11-04T19:05:24.091267] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T19:05:24.167047] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-03. [2025-11-04T19:05:24.967006] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-08. [2025-11-04T19:05:25.011806] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-11-04T19:05:25.767036] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-10. [2025-11-04T19:05:25.940726] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-11-04T19:05:26.567156] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-09. [2025-11-04T19:05:26.951478] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T19:05:27.367025] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-04. [2025-11-04T19:05:27.595317] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-11-04T19:05:28.167034] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-11. [2025-11-04T19:05:28.721573] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:05:28.967041] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-05. [2025-11-04T19:05:29.166761] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T19:05:29.168611] [INFO ] [REMOTE ] GCI: Full fetch complete (63660 records). [2025-11-04T19:05:29.168741] [DEBUG] [VALIDATOR ] GCI: Validating primary data. [2025-11-04T19:05:29.614371] [DEBUG] [VALIDATOR ] GCI: All expected trading days are present. [2025-11-04T19:05:29.614464] [DEBUG] [VALIDATOR ] GCI: Primary data validation passed. [2025-11-04T19:05:29.615483] [INFO ] [MAIN ] GCI: Fetched primary data is valid. [2025-11-04T19:05:29.615553] [DEBUG] [LOCAL ] GCI: Saving primary data. [2025-11-04T19:05:29.616972] [DEBUG] [LOCAL ] GCI: Trimmed 63660 records to 43656 (keeping 200 days of data) [2025-11-04T19:05:29.788941] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-11. [2025-11-04T19:05:29.847858] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T19:05:29.899620] [DEBUG] [LOCAL ] GCI: Primary data saved (43656 records). [2025-11-04T19:05:29.899770] [INFO ] [MAIN ] GCI: Full history saved. [2025-11-04T19:05:29.899853] [INFO ] [MAIN ] Update Progress: 898/2367 symbols completed. GCI took 29.3s. Estimated remaining: 3h 25m 12s. Rate: 7.2 symbols/min [2025-11-04T19:05:30.070575] [INFO ] [MAIN ] GCTS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:05:30.089377] [DEBUG] [LOCAL ] GCTS: Loaded 20298 existing records from primary storage. [2025-11-04T19:05:30.089596] [INFO ] [MAIN ] GCTS: Loaded 20298 existing records. [2025-11-04T19:05:30.264383] [WARN ] [MAIN ] GCTS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:05:30.264597] [INFO ] [MAIN ] GCTS: Wiping existing primary data before full refresh. [2025-11-04T19:05:30.264675] [INFO ] [LOCAL ] GCTS: Starting deletion of primary data. [2025-11-04T19:05:30.265085] [DEBUG] [LOCAL ] GCTS: Deleted primary data file. [2025-11-04T19:05:30.265141] [INFO ] [MAIN ] GCTS: Starting full primary refresh. [2025-11-04T19:05:30.570727] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-10. [2025-11-04T19:05:30.671304] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-11 (0.88 seconds). [2025-11-04T19:05:30.678245] [INFO ] [REMOTE ] GCTS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:05:30.681007] [INFO ] [REMOTE ] GBX: Full fetch complete (42777 records). [2025-11-04T19:05:30.681167] [DEBUG] [VALIDATOR ] GBX: Validating primary data. [2025-11-04T19:05:31.068047] [DEBUG] [VALIDATOR ] GBX: All expected trading days are present. [2025-11-04T19:05:31.068138] [DEBUG] [VALIDATOR ] GBX: Primary data validation passed. [2025-11-04T19:05:31.068810] [INFO ] [MAIN ] GBX: Fetched primary data is valid. [2025-11-04T19:05:31.068873] [DEBUG] [LOCAL ] GBX: Saving primary data. [2025-11-04T19:05:31.069933] [DEBUG] [LOCAL ] GBX: Trimmed 42777 records to 27370 (keeping 200 days of data) [2025-11-04T19:05:31.228821] [DEBUG] [LOCAL ] GBX: Primary data saved (27370 records). [2025-11-04T19:05:31.228948] [INFO ] [MAIN ] GBX: Full history saved. [2025-11-04T19:05:31.229024] [INFO ] [MAIN ] Update Progress: 899/2367 symbols completed. GBX took 35.6s. Estimated remaining: 3h 24m 53s. Rate: 7.2 symbols/min [2025-11-04T19:05:31.374068] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-01. [2025-11-04T19:05:31.400707] [INFO ] [MAIN ] GCV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:05:31.410562] [DEBUG] [LOCAL ] GCV: Loaded 7878 existing records from primary storage. [2025-11-04T19:05:31.410801] [INFO ] [MAIN ] GCV: Loaded 7878 existing records. [2025-11-04T19:05:31.596842] [WARN ] [MAIN ] GCV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:05:31.597133] [INFO ] [MAIN ] GCV: Wiping existing primary data before full refresh. [2025-11-04T19:05:31.597205] [INFO ] [LOCAL ] GCV: Starting deletion of primary data. [2025-11-04T19:05:31.597682] [DEBUG] [LOCAL ] GCV: Deleted primary data file. [2025-11-04T19:05:31.597741] [INFO ] [MAIN ] GCV: Starting full primary refresh. [2025-11-04T19:05:31.969304] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T19:05:32.003042] [INFO ] [REMOTE ] GCV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:05:32.111525] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-11-04T19:05:32.167034] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-11. [2025-11-04T19:05:32.967043] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-02. [2025-11-04T19:05:33.183207] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T19:05:33.185409] [INFO ] [REMOTE ] GBTG: Full fetch complete (57036 records). [2025-11-04T19:05:33.185540] [DEBUG] [VALIDATOR ] GBTG: Validating primary data. [2025-11-04T19:05:33.592612] [DEBUG] [VALIDATOR ] GBTG: All expected trading days are present. [2025-11-04T19:05:33.592739] [DEBUG] [VALIDATOR ] GBTG: Primary data validation passed. [2025-11-04T19:05:33.593558] [INFO ] [MAIN ] GBTG: Fetched primary data is valid. [2025-11-04T19:05:33.593603] [DEBUG] [LOCAL ] GBTG: Saving primary data. [2025-11-04T19:05:33.594886] [DEBUG] [LOCAL ] GBTG: Trimmed 57036 records to 39829 (keeping 200 days of data) [2025-11-04T19:05:33.697762] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-11-04T19:05:33.768193] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-03. [2025-11-04T19:05:33.824434] [DEBUG] [LOCAL ] GBTG: Primary data saved (39829 records). [2025-11-04T19:05:33.824544] [INFO ] [MAIN ] GBTG: Full history saved. [2025-11-04T19:05:33.824620] [INFO ] [MAIN ] Update Progress: 900/2367 symbols completed. GBTG took 41.1s. Estimated remaining: 3h 24m 35s. Rate: 7.2 symbols/min [2025-11-04T19:05:33.992969] [INFO ] [MAIN ] GD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:05:34.031070] [DEBUG] [LOCAL ] GD: Loaded 46225 existing records from primary storage. [2025-11-04T19:05:34.031391] [INFO ] [MAIN ] GD: Loaded 46225 existing records. [2025-11-04T19:05:34.206130] [WARN ] [MAIN ] GD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:05:34.206343] [INFO ] [MAIN ] GD: Wiping existing primary data before full refresh. [2025-11-04T19:05:34.206390] [INFO ] [LOCAL ] GD: Starting deletion of primary data. [2025-11-04T19:05:34.207282] [DEBUG] [LOCAL ] GD: Deleted primary data file. [2025-11-04T19:05:34.207340] [INFO ] [MAIN ] GD: Starting full primary refresh. [2025-11-04T19:05:34.570628] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-06. [2025-11-04T19:05:34.575724] [INFO ] [REMOTE ] GD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:05:34.587087] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-11-04T19:05:35.367051] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-01. [2025-11-04T19:05:35.445475] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T19:05:35.708434] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-11-04T19:05:36.167023] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-04. [2025-11-04T19:05:36.864404] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T19:05:36.967019] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-01. [2025-11-04T19:05:37.767087] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-07. [2025-11-04T19:05:38.370359] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T19:05:38.401631] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-11-04T19:05:38.567017] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-02. [2025-11-04T19:05:39.367038] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-08. [2025-11-04T19:05:39.853366] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T19:05:40.152629] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T19:05:40.166989] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-09. [2025-11-04T19:05:40.959700] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T19:05:40.967005] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-05. [2025-11-04T19:05:41.767044] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-10. [2025-11-04T19:05:41.843279] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T19:05:42.376291] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-11-04T19:05:42.567082] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-11. [2025-11-04T19:05:43.108567] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T19:05:43.109538] [INFO ] [REMOTE ] GCO: Full fetch complete (33986 records). [2025-11-04T19:05:43.109608] [DEBUG] [VALIDATOR ] GCO: Validating primary data. [2025-11-04T19:05:43.372180] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-06. [2025-11-04T19:05:43.485330] [DEBUG] [VALIDATOR ] GCO: All expected trading days are present. [2025-11-04T19:05:43.485405] [DEBUG] [VALIDATOR ] GCO: Primary data validation passed. [2025-11-04T19:05:43.485968] [INFO ] [MAIN ] GCO: Fetched primary data is valid. [2025-11-04T19:05:43.486029] [DEBUG] [LOCAL ] GCO: Saving primary data. [2025-11-04T19:05:43.486804] [DEBUG] [LOCAL ] GCO: Trimmed 33986 records to 21169 (keeping 200 days of data) [2025-11-04T19:05:43.603661] [DEBUG] [LOCAL ] GCO: Primary data saved (21169 records). [2025-11-04T19:05:43.603783] [INFO ] [MAIN ] GCO: Full history saved. [2025-11-04T19:05:43.603857] [INFO ] [MAIN ] Update Progress: 901/2367 symbols completed. GCO took 28.8s. Estimated remaining: 3h 24m 29s. Rate: 7.2 symbols/min [2025-11-04T19:05:43.782142] [INFO ] [MAIN ] GDDY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:05:43.824595] [DEBUG] [LOCAL ] GDDY: Loaded 49415 existing records from primary storage. [2025-11-04T19:05:43.824928] [INFO ] [MAIN ] GDDY: Loaded 49415 existing records. [2025-11-04T19:05:44.000052] [WARN ] [MAIN ] GDDY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:05:44.000306] [INFO ] [MAIN ] GDDY: Wiping existing primary data before full refresh. [2025-11-04T19:05:44.000353] [INFO ] [LOCAL ] GDDY: Starting deletion of primary data. [2025-11-04T19:05:44.001295] [DEBUG] [LOCAL ] GDDY: Deleted primary data file. [2025-11-04T19:05:44.001352] [INFO ] [MAIN ] GDDY: Starting full primary refresh. [2025-11-04T19:05:44.172195] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-03. [2025-11-04T19:05:44.380778] [INFO ] [REMOTE ] GDDY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:05:44.466595] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T19:05:44.967039] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-02. [2025-11-04T19:05:45.426347] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-11-04T19:05:45.563931] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T19:05:45.767088] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-01. [2025-11-04T19:05:46.567100] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-07. [2025-11-04T19:05:46.982501] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-11-04T19:05:47.358426] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-11-04T19:05:47.366988] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-08. [2025-11-04T19:05:47.963207] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-11-04T19:05:48.167076] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-03. [2025-11-04T19:05:48.569230] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T19:05:48.967049] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-04. [2025-11-04T19:05:49.479122] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-11-04T19:05:49.767035] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-05. [2025-11-04T19:05:50.121878] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-11-04T19:05:50.567050] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-02. [2025-11-04T19:05:51.367055] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-09. [2025-11-04T19:05:51.784940] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T19:05:52.108496] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-11-04T19:05:52.167005] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-10. [2025-11-04T19:05:52.939864] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-11-04T19:05:52.966994] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-11. [2025-11-04T19:05:53.640892] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T19:05:53.641945] [INFO ] [REMOTE ] GCTS: Full fetch complete (26909 records). [2025-11-04T19:05:53.642016] [DEBUG] [VALIDATOR ] GCTS: Validating primary data. [2025-11-04T19:05:53.783176] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-06. [2025-11-04T19:05:54.061980] [DEBUG] [VALIDATOR ] GCTS: All expected trading days are present. [2025-11-04T19:05:54.062066] [DEBUG] [VALIDATOR ] GCTS: Primary data validation passed. [2025-11-04T19:05:54.062690] [INFO ] [MAIN ] GCTS: Fetched primary data is valid. [2025-11-04T19:05:54.062775] [DEBUG] [LOCAL ] GCTS: Saving primary data. [2025-11-04T19:05:54.063522] [DEBUG] [LOCAL ] GCTS: Trimmed 26909 records to 20256 (keeping 200 days of data) [2025-11-04T19:05:54.188301] [DEBUG] [LOCAL ] GCTS: Primary data saved (20256 records). [2025-11-04T19:05:54.188439] [INFO ] [MAIN ] GCTS: Full history saved. [2025-11-04T19:05:54.188518] [INFO ] [MAIN ] Update Progress: 902/2367 symbols completed. GCTS took 24.3s. Estimated remaining: 3h 24m 24s. Rate: 7.2 symbols/min [2025-11-04T19:05:54.333786] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-11-04T19:05:54.409381] [INFO ] [MAIN ] GDIV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:05:54.415639] [DEBUG] [LOCAL ] GDIV: Loaded 3331 existing records from primary storage. [2025-11-04T19:05:54.415901] [INFO ] [MAIN ] GDIV: Loaded 3331 existing records. [2025-11-04T19:05:54.572122] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-07. [2025-11-04T19:05:54.644211] [WARN ] [MAIN ] GDIV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:05:54.644441] [INFO ] [MAIN ] GDIV: Wiping existing primary data before full refresh. [2025-11-04T19:05:54.644511] [INFO ] [LOCAL ] GDIV: Starting deletion of primary data. [2025-11-04T19:05:54.644778] [DEBUG] [LOCAL ] GDIV: Deleted primary data file. [2025-11-04T19:05:54.644838] [INFO ] [MAIN ] GDIV: Starting full primary refresh. [2025-11-04T19:05:55.079691] [INFO ] [REMOTE ] GDIV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:05:55.129592] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-11-04T19:05:55.367026] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-04. [2025-11-04T19:05:56.167093] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-08. [2025-11-04T19:05:56.648108] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-11-04T19:05:56.762694] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T19:05:56.967034] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-05. [2025-11-04T19:05:57.767043] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-03. [2025-11-04T19:05:58.411028] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-11-04T19:05:58.567065] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-06. [2025-11-04T19:05:59.065800] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T19:05:59.367044] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-01. [2025-11-04T19:05:59.687125] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T19:05:59.753500] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-11-04T19:06:00.167072] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-02. [2025-11-04T19:06:00.629469] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-11-04T19:06:00.967057] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-07. [2025-11-04T19:06:01.767164] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-03. [2025-11-04T19:06:02.093961] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-11-04T19:06:02.556477] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-11-04T19:06:02.567040] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-08. [2025-11-04T19:06:03.367108] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-09. [2025-11-04T19:06:03.803580] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T19:06:03.911251] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T19:06:04.167063] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-04. [2025-11-04T19:06:04.712213] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-11-04T19:06:04.967064] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-05. [2025-11-04T19:06:05.413869] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-11-04T19:06:05.767048] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-09. [2025-11-04T19:06:06.567017] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-10. [2025-11-04T19:06:07.048470] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T19:06:07.108102] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-11-04T19:06:07.367115] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-06. [2025-11-04T19:06:07.697311] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-11-04T19:06:08.167110] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-04. [2025-11-04T19:06:08.967076] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-11. [2025-11-04T19:06:09.513078] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:06:09.560480] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-11 (0.59 seconds). [2025-11-04T19:06:09.561278] [INFO ] [REMOTE ] GCV: Full fetch complete (10335 records). [2025-11-04T19:06:09.561347] [DEBUG] [VALIDATOR ] GCV: Validating primary data. [2025-11-04T19:06:09.772209] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-07. [2025-11-04T19:06:09.950074] [DEBUG] [VALIDATOR ] GCV: All expected trading days are present. [2025-11-04T19:06:09.950156] [DEBUG] [VALIDATOR ] GCV: Primary data validation passed. [2025-11-04T19:06:09.950441] [INFO ] [MAIN ] GCV: Fetched primary data is valid. [2025-11-04T19:06:09.950481] [DEBUG] [LOCAL ] GCV: Saving primary data. [2025-11-04T19:06:09.951065] [DEBUG] [LOCAL ] GCV: Trimmed 10335 records to 7842 (keeping 200 days of data) [2025-11-04T19:06:09.997302] [DEBUG] [LOCAL ] GCV: Primary data saved (7842 records). [2025-11-04T19:06:09.997443] [INFO ] [MAIN ] GCV: Full history saved. [2025-11-04T19:06:09.997527] [INFO ] [MAIN ] Update Progress: 903/2367 symbols completed. GCV took 38.8s. Estimated remaining: 3h 24m 28s. Rate: 7.2 symbols/min [2025-11-04T19:06:10.175857] [INFO ] [MAIN ] GDL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:06:10.182966] [DEBUG] [LOCAL ] GDL: Loaded 2339 existing records from primary storage. [2025-11-04T19:06:10.183195] [INFO ] [MAIN ] GDL: Loaded 2339 existing records. [2025-11-04T19:06:10.257052] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-11-04T19:06:10.380507] [WARN ] [MAIN ] GDL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:06:10.380777] [INFO ] [MAIN ] GDL: Wiping existing primary data before full refresh. [2025-11-04T19:06:10.380838] [INFO ] [LOCAL ] GDL: Starting deletion of primary data. [2025-11-04T19:06:10.381068] [DEBUG] [LOCAL ] GDL: Deleted primary data file. [2025-11-04T19:06:10.381137] [INFO ] [MAIN ] GDL: Starting full primary refresh. [2025-11-04T19:06:10.572109] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-05. [2025-11-04T19:06:10.755949] [INFO ] [REMOTE ] GDL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:06:11.367063] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-10. [2025-11-04T19:06:11.863713] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T19:06:12.167085] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-08. [2025-11-04T19:06:12.482460] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-11-04T19:06:12.832378] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T19:06:12.967078] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-06. [2025-11-04T19:06:13.767140] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-01. [2025-11-04T19:06:14.062782] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T19:06:14.094997] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-11-04T19:06:14.567026] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-07. [2025-11-04T19:06:15.367060] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-02. [2025-11-04T19:06:15.679486] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-11-04T19:06:15.939744] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T19:06:16.167054] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-03. [2025-11-04T19:06:16.517185] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-11-04T19:06:16.967082] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-09. [2025-11-04T19:06:17.247284] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-09 (0.28 seconds). [2025-11-04T19:06:17.767096] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-04. [2025-11-04T19:06:18.080673] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-11-04T19:06:18.567092] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-10. [2025-11-04T19:06:18.903876] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-10 (0.34 seconds). [2025-11-04T19:06:19.367094] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-11. [2025-11-04T19:06:19.689769] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-11 (0.32 seconds). [2025-11-04T19:06:19.691836] [INFO ] [REMOTE ] GD: Full fetch complete (69494 records). [2025-11-04T19:06:19.691938] [DEBUG] [VALIDATOR ] GD: Validating primary data. [2025-11-04T19:06:20.080911] [DEBUG] [VALIDATOR ] GD: All expected trading days are present. [2025-11-04T19:06:20.080986] [DEBUG] [VALIDATOR ] GD: Primary data validation passed. [2025-11-04T19:06:20.081807] [INFO ] [MAIN ] GD: Fetched primary data is valid. [2025-11-04T19:06:20.081868] [DEBUG] [LOCAL ] GD: Saving primary data. [2025-11-04T19:06:20.083110] [DEBUG] [LOCAL ] GD: Trimmed 69494 records to 46226 (keeping 200 days of data) [2025-11-04T19:06:20.171933] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-11. [2025-11-04T19:06:20.368958] [DEBUG] [LOCAL ] GD: Primary data saved (46226 records). [2025-11-04T19:06:20.369078] [INFO ] [MAIN ] GD: Full history saved. [2025-11-04T19:06:20.369153] [INFO ] [MAIN ] Update Progress: 904/2367 symbols completed. GD took 46.5s. Estimated remaining: 3h 24m 22s. Rate: 7.2 symbols/min [2025-11-04T19:06:20.536399] [INFO ] [MAIN ] GDO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:06:20.543211] [DEBUG] [LOCAL ] GDO: Loaded 5803 existing records from primary storage. [2025-11-04T19:06:20.543381] [INFO ] [MAIN ] GDO: Loaded 5803 existing records. [2025-11-04T19:06:20.718188] [WARN ] [MAIN ] GDO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:06:20.718391] [INFO ] [MAIN ] GDO: Wiping existing primary data before full refresh. [2025-11-04T19:06:20.718437] [INFO ] [LOCAL ] GDO: Starting deletion of primary data. [2025-11-04T19:06:20.718681] [DEBUG] [LOCAL ] GDO: Deleted primary data file. [2025-11-04T19:06:20.718739] [INFO ] [MAIN ] GDO: Starting full primary refresh. [2025-11-04T19:06:20.800216] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T19:06:20.821784] [INFO ] [REMOTE ] GDIV: Full fetch complete (5368 records). [2025-11-04T19:06:20.821905] [DEBUG] [VALIDATOR ] GDIV: Validating primary data. [2025-11-04T19:06:20.971743] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-08. [2025-11-04T19:06:21.418322] [INFO ] [REMOTE ] GDO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:06:21.439100] [DEBUG] [VALIDATOR ] GDIV: All expected trading days are present. [2025-11-04T19:06:21.439152] [DEBUG] [VALIDATOR ] GDIV: Primary data validation passed. [2025-11-04T19:06:21.439292] [INFO ] [MAIN ] GDIV: Fetched primary data is valid. [2025-11-04T19:06:21.439327] [DEBUG] [LOCAL ] GDIV: Saving primary data. [2025-11-04T19:06:21.439700] [DEBUG] [LOCAL ] GDIV: Trimmed 5368 records to 3334 (keeping 200 days of data) [2025-11-04T19:06:21.459210] [DEBUG] [LOCAL ] GDIV: Primary data saved (3334 records). [2025-11-04T19:06:21.459304] [INFO ] [MAIN ] GDIV: Full history saved. [2025-11-04T19:06:21.459386] [INFO ] [MAIN ] Update Progress: 905/2367 symbols completed. GDIV took 27.3s. Estimated remaining: 3h 24m 2s. Rate: 7.2 symbols/min [2025-11-04T19:06:21.624549] [INFO ] [MAIN ] GDOT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:06:21.657867] [DEBUG] [LOCAL ] GDOT: Loaded 37171 existing records from primary storage. [2025-11-04T19:06:21.658128] [INFO ] [MAIN ] GDOT: Loaded 37171 existing records. [2025-11-04T19:06:21.772194] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-05. [2025-11-04T19:06:21.836201] [WARN ] [MAIN ] GDOT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:06:21.836364] [INFO ] [MAIN ] GDOT: Wiping existing primary data before full refresh. [2025-11-04T19:06:21.836408] [INFO ] [LOCAL ] GDOT: Starting deletion of primary data. [2025-11-04T19:06:21.836988] [DEBUG] [LOCAL ] GDOT: Deleted primary data file. [2025-11-04T19:06:21.837045] [INFO ] [MAIN ] GDOT: Starting full primary refresh. [2025-11-04T19:06:22.183896] [INFO ] [REMOTE ] GDOT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:06:22.186692] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-11-04T19:06:22.424722] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T19:06:22.567045] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-09. [2025-11-04T19:06:23.367034] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-01. [2025-11-04T19:06:23.669342] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T19:06:23.716550] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-01 (0.35 seconds). [2025-11-04T19:06:24.167091] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-01. [2025-11-04T19:06:24.967077] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-02. [2025-11-04T19:06:25.026122] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T19:06:25.328427] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-11-04T19:06:25.767053] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-03. [2025-11-04T19:06:26.210061] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-11-04T19:06:26.567048] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-06. [2025-11-04T19:06:26.909318] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-11-04T19:06:27.367097] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-10. [2025-11-04T19:06:28.167040] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-04. [2025-11-04T19:06:28.712325] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-11-04T19:06:28.751135] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T19:06:28.967065] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-02. [2025-11-04T19:06:29.767088] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-11. [2025-11-04T19:06:29.840502] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T19:06:30.567054] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-07. [2025-11-04T19:06:30.879340] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-07 (0.31 seconds). [2025-11-04T19:06:30.895536] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:06:30.897529] [INFO ] [REMOTE ] GDDY: Full fetch complete (72116 records). [2025-11-04T19:06:30.897668] [DEBUG] [VALIDATOR ] GDDY: Validating primary data. [2025-11-04T19:06:31.296566] [DEBUG] [VALIDATOR ] GDDY: All expected trading days are present. [2025-11-04T19:06:31.296768] [DEBUG] [VALIDATOR ] GDDY: Primary data validation passed. [2025-11-04T19:06:31.297665] [INFO ] [MAIN ] GDDY: Fetched primary data is valid. [2025-11-04T19:06:31.297766] [DEBUG] [LOCAL ] GDDY: Saving primary data. [2025-11-04T19:06:31.299151] [DEBUG] [LOCAL ] GDDY: Trimmed 72116 records to 49471 (keeping 200 days of data) [2025-11-04T19:06:31.382752] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-08. [2025-11-04T19:06:31.592185] [DEBUG] [LOCAL ] GDDY: Primary data saved (49471 records). [2025-11-04T19:06:31.592328] [INFO ] [MAIN ] GDDY: Full history saved. [2025-11-04T19:06:31.592414] [INFO ] [MAIN ] Update Progress: 906/2367 symbols completed. GDDY took 48.0s. Estimated remaining: 3h 23m 57s. Rate: 7.2 symbols/min [2025-11-04T19:06:31.764858] [INFO ] [MAIN ] GDV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:06:31.765025] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-11-04T19:06:31.783872] [DEBUG] [LOCAL ] GDV: Loaded 18516 existing records from primary storage. [2025-11-04T19:06:31.784121] [INFO ] [MAIN ] GDV: Loaded 18516 existing records. [2025-11-04T19:06:31.959749] [WARN ] [MAIN ] GDV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:06:31.959989] [INFO ] [MAIN ] GDV: Wiping existing primary data before full refresh. [2025-11-04T19:06:31.960038] [INFO ] [LOCAL ] GDV: Starting deletion of primary data. [2025-11-04T19:06:31.960458] [DEBUG] [LOCAL ] GDV: Deleted primary data file. [2025-11-04T19:06:31.960515] [INFO ] [MAIN ] GDV: Starting full primary refresh. [2025-11-04T19:06:32.172292] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-03. [2025-11-04T19:06:32.306878] [INFO ] [REMOTE ] GDV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:06:32.967053] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-09. [2025-11-04T19:06:33.310433] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T19:06:33.325434] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-09 (0.36 seconds). [2025-11-04T19:06:33.767017] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-10. [2025-11-04T19:06:34.092845] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-10 (0.33 seconds). [2025-11-04T19:06:34.567078] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-11. [2025-11-04T19:06:34.937588] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-11 (0.37 seconds). [2025-11-04T19:06:34.938454] [INFO ] [REMOTE ] GDL: Full fetch complete (3467 records). [2025-11-04T19:06:34.938527] [DEBUG] [VALIDATOR ] GDL: Validating primary data. [2025-11-04T19:06:35.325452] [DEBUG] [VALIDATOR ] GDL: All expected trading days are present. [2025-11-04T19:06:35.325554] [DEBUG] [VALIDATOR ] GDL: Primary data validation passed. [2025-11-04T19:06:35.325751] [INFO ] [MAIN ] GDL: Fetched primary data is valid. [2025-11-04T19:06:35.325815] [DEBUG] [LOCAL ] GDL: Saving primary data. [2025-11-04T19:06:35.326363] [DEBUG] [LOCAL ] GDL: Trimmed 3467 records to 2335 (keeping 200 days of data) [2025-11-04T19:06:35.342081] [DEBUG] [LOCAL ] GDL: Primary data saved (2335 records). [2025-11-04T19:06:35.342247] [INFO ] [MAIN ] GDL: Full history saved. [2025-11-04T19:06:35.342346] [INFO ] [MAIN ] Update Progress: 907/2367 symbols completed. GDL took 25.3s. Estimated remaining: 3h 23m 41s. Rate: 7.2 symbols/min [2025-11-04T19:06:35.372156] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-05. [2025-11-04T19:06:35.514440] [INFO ] [MAIN ] GE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:06:35.569361] [DEBUG] [LOCAL ] GE: Loaded 53683 existing records from primary storage. [2025-11-04T19:06:35.569787] [INFO ] [MAIN ] GE: Loaded 53683 existing records. [2025-11-04T19:06:35.750406] [WARN ] [MAIN ] GE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:06:35.750783] [INFO ] [MAIN ] GE: Wiping existing primary data before full refresh. [2025-11-04T19:06:35.750834] [INFO ] [LOCAL ] GE: Starting deletion of primary data. [2025-11-04T19:06:35.751958] [DEBUG] [LOCAL ] GE: Deleted primary data file. [2025-11-04T19:06:35.752083] [INFO ] [MAIN ] GE: Starting full primary refresh. [2025-11-04T19:06:35.877577] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-11-04T19:06:36.175756] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-06. [2025-11-04T19:06:36.182567] [INFO ] [REMOTE ] GE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:06:36.569182] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-11-04T19:06:36.967119] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-07. [2025-11-04T19:06:37.389340] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-11-04T19:06:37.767084] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-08. [2025-11-04T19:06:38.228282] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-11-04T19:06:38.567095] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-04. [2025-11-04T19:06:39.367016] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-01. [2025-11-04T19:06:39.565226] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-11-04T19:06:40.041272] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-11-04T19:06:40.167068] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-09. [2025-11-04T19:06:40.477210] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-09 (0.31 seconds). [2025-11-04T19:06:40.967068] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-01. [2025-11-04T19:06:41.767049] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-05. [2025-11-04T19:06:42.567067] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-10. [2025-11-04T19:06:42.721145] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-11-04T19:06:42.792577] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-11-04T19:06:42.899798] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-10 (0.33 seconds). [2025-11-04T19:06:43.367053] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-06. [2025-11-04T19:06:44.167115] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-02. [2025-11-04T19:06:44.349193] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T19:06:44.814085] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-11-04T19:06:44.967076] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-07. [2025-11-04T19:06:45.767069] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-11. [2025-11-04T19:06:46.050349] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T19:06:46.225024] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-11 (0.46 seconds). [2025-11-04T19:06:46.227314] [INFO ] [REMOTE ] GDO: Full fetch complete (8465 records). [2025-11-04T19:06:46.227510] [DEBUG] [VALIDATOR ] GDO: Validating primary data. [2025-11-04T19:06:46.567122] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-02. [2025-11-04T19:06:46.592878] [DEBUG] [VALIDATOR ] GDO: All expected trading days are present. [2025-11-04T19:06:46.592972] [DEBUG] [VALIDATOR ] GDO: Primary data validation passed. [2025-11-04T19:06:46.593267] [INFO ] [MAIN ] GDO: Fetched primary data is valid. [2025-11-04T19:06:46.593307] [DEBUG] [LOCAL ] GDO: Saving primary data. [2025-11-04T19:06:46.593887] [DEBUG] [LOCAL ] GDO: Trimmed 8465 records to 5854 (keeping 200 days of data) [2025-11-04T19:06:46.630183] [DEBUG] [LOCAL ] GDO: Primary data saved (5854 records). [2025-11-04T19:06:46.630333] [INFO ] [MAIN ] GDO: Full history saved. [2025-11-04T19:06:46.630418] [INFO ] [MAIN ] Update Progress: 908/2367 symbols completed. GDO took 26.3s. Estimated remaining: 3h 23m 37s. Rate: 7.2 symbols/min [2025-11-04T19:06:46.799326] [INFO ] [MAIN ] GEF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:06:46.817394] [DEBUG] [LOCAL ] GEF: Loaded 20586 existing records from primary storage. [2025-11-04T19:06:46.817664] [INFO ] [MAIN ] GEF: Loaded 20586 existing records. [2025-11-04T19:06:46.989595] [WARN ] [MAIN ] GEF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:06:46.989823] [INFO ] [MAIN ] GEF: Wiping existing primary data before full refresh. [2025-11-04T19:06:46.989870] [INFO ] [LOCAL ] GEF: Starting deletion of primary data. [2025-11-04T19:06:46.990288] [DEBUG] [LOCAL ] GEF: Deleted primary data file. [2025-11-04T19:06:46.990344] [INFO ] [MAIN ] GEF: Starting full primary refresh. [2025-11-04T19:06:47.345524] [INFO ] [REMOTE ] GEF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:06:47.367038] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-01. [2025-11-04T19:06:48.092105] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-11-04T19:06:48.093724] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-11-04T19:06:48.167071] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-08. [2025-11-04T19:06:48.967008] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-02. [2025-11-04T19:06:49.203537] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T19:06:49.592410] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-11-04T19:06:49.767087] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-09. [2025-11-04T19:06:50.567085] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-03. [2025-11-04T19:06:50.845932] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T19:06:51.264895] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T19:06:51.367042] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-04. [2025-11-04T19:06:52.167048] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-10. [2025-11-04T19:06:52.205870] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-11-04T19:06:52.967081] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-03. [2025-11-04T19:06:53.167387] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-11-04T19:06:53.669992] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T19:06:53.767057] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-04. [2025-11-04T19:06:54.534397] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-11-04T19:06:54.567110] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-11. [2025-11-04T19:06:55.367077] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-03. [2025-11-04T19:06:55.485343] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T19:06:55.486837] [INFO ] [REMOTE ] GDOT: Full fetch complete (53698 records). [2025-11-04T19:06:55.486925] [DEBUG] [VALIDATOR ] GDOT: Validating primary data. [2025-11-04T19:06:55.909240] [DEBUG] [VALIDATOR ] GDOT: All expected trading days are present. [2025-11-04T19:06:55.909365] [DEBUG] [VALIDATOR ] GDOT: Primary data validation passed. [2025-11-04T19:06:55.910783] [INFO ] [MAIN ] GDOT: Fetched primary data is valid. [2025-11-04T19:06:55.910853] [DEBUG] [LOCAL ] GDOT: Saving primary data. [2025-11-04T19:06:55.912469] [DEBUG] [LOCAL ] GDOT: Trimmed 53698 records to 37151 (keeping 200 days of data) [2025-11-04T19:06:56.138049] [DEBUG] [LOCAL ] GDOT: Primary data saved (37151 records). [2025-11-04T19:06:56.138254] [INFO ] [MAIN ] GDOT: Full history saved. [2025-11-04T19:06:56.138368] [INFO ] [MAIN ] Update Progress: 909/2367 symbols completed. GDOT took 34.7s. Estimated remaining: 3h 23m 31s. Rate: 7.2 symbols/min [2025-11-04T19:06:56.173578] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-05. [2025-11-04T19:06:56.329741] [INFO ] [MAIN ] GEL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:06:56.365014] [DEBUG] [LOCAL ] GEL: Loaded 30807 existing records from primary storage. [2025-11-04T19:06:56.365374] [INFO ] [MAIN ] GEL: Loaded 30807 existing records. [2025-11-04T19:06:56.554609] [WARN ] [MAIN ] GEL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:06:56.554919] [INFO ] [MAIN ] GEL: Wiping existing primary data before full refresh. [2025-11-04T19:06:56.554972] [INFO ] [LOCAL ] GEL: Starting deletion of primary data. [2025-11-04T19:06:56.555192] [DEBUG] [LOCAL ] GEL: Deleted primary data file. [2025-11-04T19:06:56.555234] [INFO ] [MAIN ] GEL: Starting full primary refresh. [2025-11-04T19:06:56.954161] [INFO ] [REMOTE ] GEL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:06:56.969159] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-01. [2025-11-04T19:06:56.972188] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-11-04T19:06:57.116080] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-11-04T19:06:57.767151] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-06. [2025-11-04T19:06:58.003041] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-11-04T19:06:58.427841] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-11-04T19:06:58.567099] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-02. [2025-11-04T19:06:59.367079] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-05. [2025-11-04T19:06:59.566751] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-11-04T19:06:59.918383] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T19:07:00.167073] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-07. [2025-11-04T19:07:00.971869] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-06. [2025-11-04T19:07:00.977571] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-11-04T19:07:01.767017] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-03. [2025-11-04T19:07:01.929064] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-11-04T19:07:02.567058] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-08. [2025-11-04T19:07:02.722631] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-11-04T19:07:03.221336] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-11-04T19:07:03.367059] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-04. [2025-11-04T19:07:04.167017] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-07. [2025-11-04T19:07:04.244235] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-11-04T19:07:04.972217] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-09. [2025-11-04T19:07:04.985254] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T19:07:05.632160] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-11-04T19:07:05.767043] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-10. [2025-11-04T19:07:06.445590] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-11-04T19:07:06.567052] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-08. [2025-11-04T19:07:07.317336] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T19:07:07.367049] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-09. [2025-11-04T19:07:08.167055] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-11. [2025-11-04T19:07:08.249790] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T19:07:08.849784] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T19:07:08.850882] [INFO ] [REMOTE ] GDV: Full fetch complete (28128 records). [2025-11-04T19:07:08.850953] [DEBUG] [VALIDATOR ] GDV: Validating primary data. [2025-11-04T19:07:08.972107] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-10. [2025-11-04T19:07:09.251794] [DEBUG] [VALIDATOR ] GDV: All expected trading days are present. [2025-11-04T19:07:09.251875] [DEBUG] [VALIDATOR ] GDV: Primary data validation passed. [2025-11-04T19:07:09.252366] [INFO ] [MAIN ] GDV: Fetched primary data is valid. [2025-11-04T19:07:09.252405] [DEBUG] [LOCAL ] GDV: Saving primary data. [2025-11-04T19:07:09.253154] [DEBUG] [LOCAL ] GDV: Trimmed 28128 records to 18521 (keeping 200 days of data) [2025-11-04T19:07:09.364662] [DEBUG] [LOCAL ] GDV: Primary data saved (18521 records). [2025-11-04T19:07:09.364811] [INFO ] [MAIN ] GDV: Full history saved. [2025-11-04T19:07:09.364889] [INFO ] [MAIN ] Update Progress: 910/2367 symbols completed. GDV took 37.8s. Estimated remaining: 3h 23m 30s. Rate: 7.2 symbols/min [2025-11-04T19:07:09.552489] [INFO ] [MAIN ] GENI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:07:09.598964] [DEBUG] [LOCAL ] GENI: Loaded 52861 existing records from primary storage. [2025-11-04T19:07:09.599275] [INFO ] [MAIN ] GENI: Loaded 52861 existing records. [2025-11-04T19:07:09.781519] [WARN ] [MAIN ] GENI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:07:09.777868] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-04. [2025-11-04T19:07:09.781910] [INFO ] [MAIN ] GENI: Wiping existing primary data before full refresh. [2025-11-04T19:07:09.783142] [INFO ] [LOCAL ] GENI: Starting deletion of primary data. [2025-11-04T19:07:09.788883] [DEBUG] [LOCAL ] GENI: Deleted primary data file. [2025-11-04T19:07:09.796268] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T19:07:09.796459] [INFO ] [MAIN ] GENI: Starting full primary refresh. [2025-11-04T19:07:10.150525] [INFO ] [REMOTE ] GENI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:07:10.567059] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-05. [2025-11-04T19:07:11.367090] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-11. [2025-11-04T19:07:11.403337] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-11-04T19:07:11.571610] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-11-04T19:07:12.167062] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-05. [2025-11-04T19:07:12.210181] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T19:07:12.211507] [INFO ] [REMOTE ] GEF: Full fetch complete (29429 records). [2025-11-04T19:07:12.211605] [DEBUG] [VALIDATOR ] GEF: Validating primary data. [2025-11-04T19:07:12.578076] [DEBUG] [VALIDATOR ] GEF: All expected trading days are present. [2025-11-04T19:07:12.578150] [DEBUG] [VALIDATOR ] GEF: Primary data validation passed. [2025-11-04T19:07:12.578660] [INFO ] [MAIN ] GEF: Fetched primary data is valid. [2025-11-04T19:07:12.578724] [DEBUG] [LOCAL ] GEF: Saving primary data. [2025-11-04T19:07:12.579387] [DEBUG] [LOCAL ] GEF: Trimmed 29429 records to 20649 (keeping 200 days of data) [2025-11-04T19:07:12.700819] [DEBUG] [LOCAL ] GEF: Primary data saved (20649 records). [2025-11-04T19:07:12.700938] [INFO ] [MAIN ] GEF: Full history saved. [2025-11-04T19:07:12.701009] [INFO ] [MAIN ] Update Progress: 911/2367 symbols completed. GEF took 26.1s. Estimated remaining: 3h 23m 14s. Rate: 7.2 symbols/min [2025-11-04T19:07:12.868390] [INFO ] [MAIN ] GEO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:07:12.912029] [DEBUG] [LOCAL ] GEO: Loaded 51520 existing records from primary storage. [2025-11-04T19:07:12.912327] [INFO ] [MAIN ] GEO: Loaded 51520 existing records. [2025-11-04T19:07:12.979162] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-06. [2025-11-04T19:07:13.094269] [WARN ] [MAIN ] GEO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:07:13.094475] [INFO ] [MAIN ] GEO: Wiping existing primary data before full refresh. [2025-11-04T19:07:13.094521] [INFO ] [LOCAL ] GEO: Starting deletion of primary data. [2025-11-04T19:07:13.095319] [DEBUG] [LOCAL ] GEO: Deleted primary data file. [2025-11-04T19:07:13.095376] [INFO ] [MAIN ] GEO: Starting full primary refresh. [2025-11-04T19:07:13.441478] [INFO ] [REMOTE ] GEO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:07:13.767097] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-01. [2025-11-04T19:07:13.817811] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-11-04T19:07:14.053382] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T19:07:14.567071] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-06. [2025-11-04T19:07:15.243381] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-11-04T19:07:15.367070] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-07. [2025-11-04T19:07:16.174559] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-01. [2025-11-04T19:07:16.196069] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-11-04T19:07:16.401734] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T19:07:16.967054] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-02. [2025-11-04T19:07:17.352048] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T19:07:17.767084] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-07. [2025-11-04T19:07:18.220304] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-11-04T19:07:18.567099] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-03. [2025-11-04T19:07:19.367037] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-08. [2025-11-04T19:07:19.580275] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-11-04T19:07:19.865707] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T19:07:20.167081] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-02. [2025-11-04T19:07:20.440481] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T19:07:20.967054] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-04. [2025-11-04T19:07:21.251985] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T19:07:21.767055] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-09. [2025-11-04T19:07:22.374228] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-11-04T19:07:22.537955] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-11-04T19:07:22.567051] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-10. [2025-11-04T19:07:23.336898] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-11-04T19:07:23.367026] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-11. [2025-11-04T19:07:24.102803] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T19:07:24.104167] [INFO ] [REMOTE ] GEL: Full fetch complete (47734 records). [2025-11-04T19:07:24.104237] [DEBUG] [VALIDATOR ] GEL: Validating primary data. [2025-11-04T19:07:24.172256] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-03. [2025-11-04T19:07:24.514393] [DEBUG] [VALIDATOR ] GEL: All expected trading days are present. [2025-11-04T19:07:24.514472] [DEBUG] [VALIDATOR ] GEL: Primary data validation passed. [2025-11-04T19:07:24.515337] [INFO ] [MAIN ] GEL: Fetched primary data is valid. [2025-11-04T19:07:24.515402] [DEBUG] [LOCAL ] GEL: Saving primary data. [2025-11-04T19:07:24.516391] [DEBUG] [LOCAL ] GEL: Trimmed 47734 records to 30746 (keeping 200 days of data) [2025-11-04T19:07:24.701173] [DEBUG] [LOCAL ] GEL: Primary data saved (30746 records). [2025-11-04T19:07:24.701296] [INFO ] [MAIN ] GEL: Full history saved. [2025-11-04T19:07:24.701375] [INFO ] [MAIN ] Update Progress: 912/2367 symbols completed. GEL took 28.6s. Estimated remaining: 3h 23m 11s. Rate: 7.2 symbols/min [2025-11-04T19:07:24.868344] [INFO ] [MAIN ] GES: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:07:24.900249] [DEBUG] [LOCAL ] GES: Loaded 36961 existing records from primary storage. [2025-11-04T19:07:24.900560] [INFO ] [MAIN ] GES: Loaded 36961 existing records. [2025-11-04T19:07:24.974463] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-08. [2025-11-04T19:07:25.080073] [WARN ] [MAIN ] GES: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:07:25.080271] [INFO ] [MAIN ] GES: Wiping existing primary data before full refresh. [2025-11-04T19:07:25.080317] [INFO ] [LOCAL ] GES: Starting deletion of primary data. [2025-11-04T19:07:25.081056] [DEBUG] [LOCAL ] GES: Deleted primary data file. [2025-11-04T19:07:25.081114] [INFO ] [MAIN ] GES: Starting full primary refresh. [2025-11-04T19:07:25.430694] [INFO ] [REMOTE ] GES: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:07:25.566338] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T19:07:25.767072] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-01. [2025-11-04T19:07:26.538607] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-11-04T19:07:26.567008] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-09. [2025-11-04T19:07:26.665292] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T19:07:27.367078] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-04. [2025-11-04T19:07:28.160538] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T19:07:28.171103] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-05. [2025-11-04T19:07:28.645688] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T19:07:28.967090] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-05. [2025-11-04T19:07:29.298199] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T19:07:29.767051] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-02. [2025-11-04T19:07:30.229243] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T19:07:30.567039] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-06. [2025-11-04T19:07:30.734131] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T19:07:31.367013] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-06. [2025-11-04T19:07:32.133575] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-11-04T19:07:32.167040] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-07. [2025-11-04T19:07:32.762001] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T19:07:32.967030] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-10. [2025-11-04T19:07:33.569032] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T19:07:33.767073] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-03. [2025-11-04T19:07:34.567061] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-07. [2025-11-04T19:07:34.735463] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-11-04T19:07:34.971166] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T19:07:35.367098] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-04. [2025-11-04T19:07:35.872445] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T19:07:36.167024] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-08. [2025-11-04T19:07:36.653237] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T19:07:36.967099] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-11. [2025-11-04T19:07:37.436574] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T19:07:37.434701] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T19:07:37.450635] [INFO ] [REMOTE ] GE: Full fetch complete (77852 records). [2025-11-04T19:07:37.450856] [DEBUG] [VALIDATOR ] GE: Validating primary data. [2025-11-04T19:07:37.772147] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-09. [2025-11-04T19:07:37.860310] [DEBUG] [VALIDATOR ] GE: All expected trading days are present. [2025-11-04T19:07:37.860416] [DEBUG] [VALIDATOR ] GE: Primary data validation passed. [2025-11-04T19:07:37.861840] [INFO ] [MAIN ] GE: Fetched primary data is valid. [2025-11-04T19:07:37.861913] [DEBUG] [LOCAL ] GE: Saving primary data. [2025-11-04T19:07:37.864019] [DEBUG] [LOCAL ] GE: Trimmed 77852 records to 53683 (keeping 200 days of data) [2025-11-04T19:07:38.223325] [DEBUG] [LOCAL ] GE: Primary data saved (53683 records). [2025-11-04T19:07:38.223524] [INFO ] [MAIN ] GE: Full history saved. [2025-11-04T19:07:38.223634] [INFO ] [MAIN ] Update Progress: 913/2367 symbols completed. GE took 1m 3s. Estimated remaining: 3h 23m 11s. Rate: 7.2 symbols/min [2025-11-04T19:07:38.400750] [INFO ] [MAIN ] GETY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:07:38.437863] [DEBUG] [LOCAL ] GETY: Loaded 35039 existing records from primary storage. [2025-11-04T19:07:38.438307] [INFO ] [MAIN ] GETY: Loaded 35039 existing records. [2025-11-04T19:07:38.572294] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-08. [2025-11-04T19:07:38.630118] [WARN ] [MAIN ] GETY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:07:38.630370] [INFO ] [MAIN ] GETY: Wiping existing primary data before full refresh. [2025-11-04T19:07:38.630435] [INFO ] [LOCAL ] GETY: Starting deletion of primary data. [2025-11-04T19:07:38.631096] [DEBUG] [LOCAL ] GETY: Deleted primary data file. [2025-11-04T19:07:38.631162] [INFO ] [MAIN ] GETY: Starting full primary refresh. [2025-11-04T19:07:38.994891] [INFO ] [REMOTE ] GETY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:07:39.282453] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-11-04T19:07:39.367065] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-10. [2025-11-04T19:07:40.072954] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-11-04T19:07:40.167054] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-05. [2025-11-04T19:07:40.907219] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-11-04T19:07:40.967096] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-09. [2025-11-04T19:07:41.149026] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-11-04T19:07:41.767102] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-01. [2025-11-04T19:07:42.172918] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T19:07:42.567115] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-11. [2025-11-04T19:07:43.323374] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-11-04T19:07:43.366994] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-02. [2025-11-04T19:07:43.718183] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T19:07:43.721026] [INFO ] [REMOTE ] GENI: Full fetch complete (76102 records). [2025-11-04T19:07:43.721190] [DEBUG] [VALIDATOR ] GENI: Validating primary data. [2025-11-04T19:07:44.149878] [DEBUG] [VALIDATOR ] GENI: All expected trading days are present. [2025-11-04T19:07:44.149994] [DEBUG] [VALIDATOR ] GENI: Primary data validation passed. [2025-11-04T19:07:44.151350] [INFO ] [MAIN ] GENI: Fetched primary data is valid. [2025-11-04T19:07:44.151420] [DEBUG] [LOCAL ] GENI: Saving primary data. [2025-11-04T19:07:44.153627] [DEBUG] [LOCAL ] GENI: Trimmed 76102 records to 52863 (keeping 200 days of data) [2025-11-04T19:07:44.202953] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-06. [2025-11-04T19:07:44.503754] [DEBUG] [LOCAL ] GENI: Primary data saved (52863 records). [2025-11-04T19:07:44.504116] [INFO ] [MAIN ] GENI: Full history saved. [2025-11-04T19:07:44.504268] [INFO ] [MAIN ] Update Progress: 914/2367 symbols completed. GENI took 35.1s. Estimated remaining: 3h 22m 59s. Rate: 7.2 symbols/min [2025-11-04T19:07:44.649530] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T19:07:44.729613] [INFO ] [MAIN ] GEV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:07:44.783057] [DEBUG] [LOCAL ] GEV: Loaded 52980 existing records from primary storage. [2025-11-04T19:07:44.783475] [INFO ] [MAIN ] GEV: Loaded 52980 existing records. [2025-11-04T19:07:44.972099] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-10. [2025-11-04T19:07:44.971918] [WARN ] [MAIN ] GEV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:07:44.973686] [INFO ] [MAIN ] GEV: Wiping existing primary data before full refresh. [2025-11-04T19:07:44.973753] [INFO ] [LOCAL ] GEV: Starting deletion of primary data. [2025-11-04T19:07:44.975100] [DEBUG] [LOCAL ] GEV: Deleted primary data file. [2025-11-04T19:07:44.975180] [INFO ] [MAIN ] GEV: Starting full primary refresh. [2025-11-04T19:07:45.411214] [INFO ] [REMOTE ] GEV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:07:45.436814] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-11-04T19:07:45.767114] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-03. [2025-11-04T19:07:46.567078] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-01. [2025-11-04T19:07:46.670253] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-11-04T19:07:46.995889] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T19:07:47.367072] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-11. [2025-11-04T19:07:48.167025] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-07. [2025-11-04T19:07:48.327671] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-11-04T19:07:48.637577] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-11 (1.27 seconds). [2025-11-04T19:07:48.639612] [INFO ] [REMOTE ] GEO: Full fetch complete (75317 records). [2025-11-04T19:07:48.639763] [DEBUG] [VALIDATOR ] GEO: Validating primary data. [2025-11-04T19:07:48.977929] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-04. [2025-11-04T19:07:49.037016] [DEBUG] [VALIDATOR ] GEO: All expected trading days are present. [2025-11-04T19:07:49.037092] [DEBUG] [VALIDATOR ] GEO: Primary data validation passed. [2025-11-04T19:07:49.038054] [INFO ] [MAIN ] GEO: Fetched primary data is valid. [2025-11-04T19:07:49.038124] [DEBUG] [LOCAL ] GEO: Saving primary data. [2025-11-04T19:07:49.039517] [DEBUG] [LOCAL ] GEO: Trimmed 75317 records to 51514 (keeping 200 days of data) [2025-11-04T19:07:49.342638] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T19:07:49.369594] [DEBUG] [LOCAL ] GEO: Primary data saved (51514 records). [2025-11-04T19:07:49.369764] [INFO ] [MAIN ] GEO: Full history saved. [2025-11-04T19:07:49.369858] [INFO ] [MAIN ] Update Progress: 915/2367 symbols completed. GEO took 36.7s. Estimated remaining: 3h 22m 45s. Rate: 7.2 symbols/min [2025-11-04T19:07:49.536053] [INFO ] [MAIN ] GF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:07:49.539969] [DEBUG] [LOCAL ] GF: Loaded 2500 existing records from primary storage. [2025-11-04T19:07:49.540108] [INFO ] [MAIN ] GF: Loaded 2500 existing records. [2025-11-04T19:07:49.717187] [WARN ] [MAIN ] GF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:07:49.717425] [INFO ] [MAIN ] GF: Wiping existing primary data before full refresh. [2025-11-04T19:07:49.717478] [INFO ] [LOCAL ] GF: Starting deletion of primary data. [2025-11-04T19:07:49.717638] [DEBUG] [LOCAL ] GF: Deleted primary data file. [2025-11-04T19:07:49.717724] [INFO ] [MAIN ] GF: Starting full primary refresh. [2025-11-04T19:07:49.772306] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-02. [2025-11-04T19:07:50.080795] [INFO ] [REMOTE ] GF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:07:50.111264] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T19:07:50.567101] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-08. [2025-11-04T19:07:51.367021] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-05. [2025-11-04T19:07:51.482878] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-11-04T19:07:51.727616] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T19:07:52.167108] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-09. [2025-11-04T19:07:52.282684] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T19:07:52.967060] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-01. [2025-11-04T19:07:53.115518] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-11-04T19:07:53.280006] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-01 (0.31 seconds). [2025-11-04T19:07:53.767046] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-02. [2025-11-04T19:07:54.095021] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-11-04T19:07:54.567066] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-10. [2025-11-04T19:07:55.367045] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-03. [2025-11-04T19:07:55.591010] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T19:07:55.756636] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-11-04T19:07:56.167066] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-11. [2025-11-04T19:07:56.967111] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-04. [2025-11-04T19:07:56.971940] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-11 (0.80 seconds). [2025-11-04T19:07:56.973400] [INFO ] [REMOTE ] GES: Full fetch complete (57577 records). [2025-11-04T19:07:56.973484] [DEBUG] [VALIDATOR ] GES: Validating primary data. [2025-11-04T19:07:57.370106] [DEBUG] [VALIDATOR ] GES: All expected trading days are present. [2025-11-04T19:07:57.370908] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-11-04T19:07:57.370984] [DEBUG] [VALIDATOR ] GES: Primary data validation passed. [2025-11-04T19:07:57.373735] [INFO ] [MAIN ] GES: Fetched primary data is valid. [2025-11-04T19:07:57.373813] [DEBUG] [LOCAL ] GES: Saving primary data. [2025-11-04T19:07:57.375091] [DEBUG] [LOCAL ] GES: Trimmed 57577 records to 36775 (keeping 200 days of data) [2025-11-04T19:07:57.592775] [DEBUG] [LOCAL ] GES: Primary data saved (36775 records). [2025-11-04T19:07:57.592912] [INFO ] [MAIN ] GES: Full history saved. [2025-11-04T19:07:57.592990] [INFO ] [MAIN ] Update Progress: 916/2367 symbols completed. GES took 32.9s. Estimated remaining: 3h 22m 37s. Rate: 7.2 symbols/min [2025-11-04T19:07:57.763707] [INFO ] [MAIN ] GFF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:07:57.767088] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-05. [2025-11-04T19:07:57.791834] [DEBUG] [LOCAL ] GFF: Loaded 28512 existing records from primary storage. [2025-11-04T19:07:57.792139] [INFO ] [MAIN ] GFF: Loaded 28512 existing records. [2025-11-04T19:07:57.984553] [WARN ] [MAIN ] GFF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:07:57.984826] [INFO ] [MAIN ] GFF: Wiping existing primary data before full refresh. [2025-11-04T19:07:57.984877] [INFO ] [LOCAL ] GFF: Starting deletion of primary data. [2025-11-04T19:07:57.985446] [DEBUG] [LOCAL ] GFF: Deleted primary data file. [2025-11-04T19:07:57.985517] [INFO ] [MAIN ] GFF: Starting full primary refresh. [2025-11-04T19:07:58.268016] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-11-04T19:07:58.346466] [INFO ] [REMOTE ] GFF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:07:58.567045] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-06. [2025-11-04T19:07:58.976727] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-11-04T19:07:59.367069] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-01. [2025-11-04T19:08:00.167076] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-07. [2025-11-04T19:08:00.372032] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T19:08:00.427582] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-07 (0.26 seconds). [2025-11-04T19:08:00.967083] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-06. [2025-11-04T19:08:01.767042] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-08. [2025-11-04T19:08:02.037124] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T19:08:02.132857] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-11-04T19:08:02.567052] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-09. [2025-11-04T19:08:02.887885] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-11-04T19:08:03.367019] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-10. [2025-11-04T19:08:03.696968] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-10 (0.33 seconds). [2025-11-04T19:08:04.167030] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-03. [2025-11-04T19:08:04.967081] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-02. [2025-11-04T19:08:05.767034] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-11. [2025-11-04T19:08:05.858446] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T19:08:06.005809] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-11-04T19:08:06.533691] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T19:08:06.534724] [INFO ] [REMOTE ] GF: Full fetch complete (4790 records). [2025-11-04T19:08:06.535033] [DEBUG] [VALIDATOR ] GF: Validating primary data. [2025-11-04T19:08:06.572214] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-07. [2025-11-04T19:08:06.915401] [DEBUG] [VALIDATOR ] GF: All expected trading days are present. [2025-11-04T19:08:06.915520] [DEBUG] [VALIDATOR ] GF: Primary data validation passed. [2025-11-04T19:08:06.915813] [INFO ] [MAIN ] GF: Fetched primary data is valid. [2025-11-04T19:08:06.915878] [DEBUG] [LOCAL ] GF: Saving primary data. [2025-11-04T19:08:06.916283] [DEBUG] [LOCAL ] GF: Trimmed 4790 records to 2500 (keeping 200 days of data) [2025-11-04T19:08:06.931828] [DEBUG] [LOCAL ] GF: Primary data saved (2500 records). [2025-11-04T19:08:06.931903] [INFO ] [MAIN ] GF: Full history saved. [2025-11-04T19:08:06.931987] [INFO ] [MAIN ] Update Progress: 917/2367 symbols completed. GF took 17.6s. Estimated remaining: 3h 22m 30s. Rate: 7.2 symbols/min [2025-11-04T19:08:07.367051] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-04. [2025-11-04T19:08:07.580581] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T19:08:08.167131] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-08. [2025-11-04T19:08:08.967025] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-03. [2025-11-04T19:08:09.117678] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T19:08:09.188917] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-11-04T19:08:09.767002] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-09. [2025-11-04T19:08:09.821284] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-11-04T19:08:10.567891] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-04. [2025-11-04T19:08:10.576960] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-11-04T19:08:11.367065] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-10. [2025-11-04T19:08:11.613188] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-11-04T19:08:12.167118] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-05. [2025-11-04T19:08:12.715971] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T19:08:12.967078] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-11. [2025-11-04T19:08:13.161379] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T19:08:13.767145] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-06. [2025-11-04T19:08:14.280466] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-11 (1.31 seconds). [2025-11-04T19:08:14.283159] [INFO ] [REMOTE ] GETY: Full fetch complete (54971 records). [2025-11-04T19:08:14.283286] [DEBUG] [VALIDATOR ] GETY: Validating primary data. [2025-11-04T19:08:14.568557] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-05. [2025-11-04T19:08:14.683877] [DEBUG] [VALIDATOR ] GETY: All expected trading days are present. [2025-11-04T19:08:14.683985] [DEBUG] [VALIDATOR ] GETY: Primary data validation passed. [2025-11-04T19:08:14.685374] [INFO ] [MAIN ] GETY: Fetched primary data is valid. [2025-11-04T19:08:14.685456] [DEBUG] [LOCAL ] GETY: Saving primary data. [2025-11-04T19:08:14.686950] [DEBUG] [LOCAL ] GETY: Trimmed 54971 records to 35146 (keeping 200 days of data) [2025-11-04T19:08:14.754423] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-11-04T19:08:14.914578] [DEBUG] [LOCAL ] GETY: Primary data saved (35146 records). [2025-11-04T19:08:14.914755] [INFO ] [MAIN ] GETY: Full history saved. [2025-11-04T19:08:14.914848] [INFO ] [MAIN ] Update Progress: 918/2367 symbols completed. GETY took 36.7s. Estimated remaining: 3h 22m 21s. Rate: 7.2 symbols/min [2025-11-04T19:08:15.367119] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-07. [2025-11-04T19:08:16.187670] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T19:08:16.193826] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-08. [2025-11-04T19:08:16.312977] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-11-04T19:08:16.967088] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-06. [2025-11-04T19:08:17.170959] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-11-04T19:08:17.767103] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-09. [2025-11-04T19:08:18.579324] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-11-04T19:08:18.585381] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-10. [2025-11-04T19:08:18.624304] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-11-04T19:08:19.367075] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-07. [2025-11-04T19:08:19.393225] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-11-04T19:08:20.167105] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-11. [2025-11-04T19:08:20.900028] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T19:08:20.901379] [INFO ] [REMOTE ] GFF: Full fetch complete (43233 records). [2025-11-04T19:08:20.901456] [DEBUG] [VALIDATOR ] GFF: Validating primary data. [2025-11-04T19:08:21.282776] [DEBUG] [VALIDATOR ] GFF: All expected trading days are present. [2025-11-04T19:08:21.282892] [DEBUG] [VALIDATOR ] GFF: Primary data validation passed. [2025-11-04T19:08:21.283567] [INFO ] [MAIN ] GFF: Fetched primary data is valid. [2025-11-04T19:08:21.283710] [DEBUG] [LOCAL ] GFF: Saving primary data. [2025-11-04T19:08:21.284862] [DEBUG] [LOCAL ] GFF: Trimmed 43233 records to 28455 (keeping 200 days of data) [2025-11-04T19:08:21.481088] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-11-04T19:08:21.506014] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-08. [2025-11-04T19:08:21.505614] [DEBUG] [LOCAL ] GFF: Primary data saved (28455 records). [2025-11-04T19:08:21.507023] [INFO ] [MAIN ] GFF: Full history saved. [2025-11-04T19:08:21.507111] [INFO ] [MAIN ] Update Progress: 919/2367 symbols completed. GFF took 23.9s. Estimated remaining: 3h 22m 10s. Rate: 7.2 symbols/min [2025-11-04T19:08:23.353313] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-11-04T19:08:23.364701] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-09. [2025-11-04T19:08:25.243592] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-11-04T19:08:25.255161] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-10. [2025-11-04T19:08:27.267264] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-11-04T19:08:27.279725] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-11. [2025-11-04T19:08:29.127476] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-11 (1.85 seconds). [2025-11-04T19:08:29.129552] [INFO ] [REMOTE ] GEV: Full fetch complete (77062 records). [2025-11-04T19:08:29.129664] [DEBUG] [VALIDATOR ] GEV: Validating primary data. [2025-11-04T19:08:29.520434] [DEBUG] [VALIDATOR ] GEV: All expected trading days are present. [2025-11-04T19:08:29.520510] [DEBUG] [VALIDATOR ] GEV: Primary data validation passed. [2025-11-04T19:08:29.521383] [INFO ] [MAIN ] GEV: Fetched primary data is valid. [2025-11-04T19:08:29.521446] [DEBUG] [LOCAL ] GEV: Saving primary data. [2025-11-04T19:08:29.522823] [DEBUG] [LOCAL ] GEV: Trimmed 77062 records to 52982 (keeping 200 days of data) [2025-11-04T19:08:29.844107] [DEBUG] [LOCAL ] GEV: Primary data saved (52982 records). [2025-11-04T19:08:29.844231] [INFO ] [MAIN ] GEV: Full history saved. [2025-11-04T19:08:29.845107] [INFO ] [MAIN ] Update Progress: 920/2367 symbols completed. GEV took 45.3s. Estimated remaining: 3h 22m 1s. Rate: 7.2 symbols/min [2025-11-04T19:08:30.527400] [INFO ] [MAIN ] GFI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:08:30.541487] [INFO ] [MAIN ] GGB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:08:30.545751] [INFO ] [MAIN ] GFL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:08:30.571133] [INFO ] [MAIN ] GFR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:08:30.583046] [DEBUG] [LOCAL ] GFR: Loaded 6787 existing records from primary storage. [2025-11-04T19:08:30.584227] [INFO ] [MAIN ] GFR: Loaded 6787 existing records. [2025-11-04T19:08:30.807871] [WARN ] [MAIN ] GFR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:08:30.808163] [INFO ] [MAIN ] GFR: Wiping existing primary data before full refresh. [2025-11-04T19:08:30.808214] [INFO ] [LOCAL ] GFR: Starting deletion of primary data. [2025-11-04T19:08:30.808402] [DEBUG] [LOCAL ] GFR: Deleted primary data file. [2025-11-04T19:08:30.808462] [INFO ] [MAIN ] GFR: Starting full primary refresh. [2025-11-04T19:08:30.894742] [DEBUG] [LOCAL ] GGB: Loaded 51603 existing records from primary storage. [2025-11-04T19:08:30.946869] [INFO ] [MAIN ] GGB: Loaded 51603 existing records. [2025-11-04T19:08:30.987688] [DEBUG] [LOCAL ] GFI: Loaded 53114 existing records from primary storage. [2025-11-04T19:08:30.987030] [DEBUG] [LOCAL ] GFL: Loaded 47252 existing records from primary storage. [2025-11-04T19:08:31.003619] [INFO ] [MAIN ] GFI: Loaded 53114 existing records. [2025-11-04T19:08:31.003765] [INFO ] [MAIN ] GFL: Loaded 47252 existing records. [2025-11-04T19:08:31.487327] [WARN ] [MAIN ] GGB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:08:31.573298] [INFO ] [MAIN ] GGB: Wiping existing primary data before full refresh. [2025-11-04T19:08:31.609828] [INFO ] [LOCAL ] GGB: Starting deletion of primary data. [2025-11-04T19:08:31.634386] [DEBUG] [LOCAL ] GGB: Deleted primary data file. [2025-11-04T19:08:31.645206] [WARN ] [MAIN ] GFI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:08:31.655604] [INFO ] [MAIN ] GGB: Starting full primary refresh. [2025-11-04T19:08:31.764095] [INFO ] [MAIN ] GFI: Wiping existing primary data before full refresh. [2025-11-04T19:08:31.767346] [WARN ] [MAIN ] GFL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:08:31.794246] [INFO ] [LOCAL ] GFI: Starting deletion of primary data. [2025-11-04T19:08:31.806825] [DEBUG] [LOCAL ] GFI: Deleted primary data file. [2025-11-04T19:08:31.812111] [INFO ] [MAIN ] GFL: Wiping existing primary data before full refresh. [2025-11-04T19:08:31.797527] [INFO ] [REMOTE ] GFR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:08:31.817731] [INFO ] [MAIN ] GFI: Starting full primary refresh. [2025-11-04T19:08:31.818073] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-01. [2025-11-04T19:08:31.823338] [INFO ] [LOCAL ] GFL: Starting deletion of primary data. [2025-11-04T19:08:31.836909] [DEBUG] [LOCAL ] GFL: Deleted primary data file. [2025-11-04T19:08:31.861451] [INFO ] [MAIN ] GFL: Starting full primary refresh. [2025-11-04T19:08:32.673744] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T19:08:32.775582] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-02. [2025-11-04T19:08:32.822366] [INFO ] [REMOTE ] GFI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:08:32.843739] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-01. [2025-11-04T19:08:32.888183] [INFO ] [REMOTE ] GGB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:08:32.900611] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-01. [2025-11-04T19:08:32.934425] [INFO ] [REMOTE ] GFL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:08:32.934977] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-01. [2025-11-04T19:08:33.302209] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-11-04T19:08:33.304554] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-03. [2025-11-04T19:08:33.958475] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-11-04T19:08:33.961548] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-04. [2025-11-04T19:08:34.029679] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-11-04T19:08:34.039879] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-02. [2025-11-04T19:08:34.134724] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T19:08:34.145951] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-02. [2025-11-04T19:08:34.281762] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T19:08:34.292243] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-02. [2025-11-04T19:08:34.451990] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-11-04T19:08:34.455747] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-05. [2025-11-04T19:08:35.002368] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T19:08:35.005485] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-06. [2025-11-04T19:08:35.110739] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T19:08:35.119979] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-03. [2025-11-04T19:08:35.275458] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-11-04T19:08:35.285263] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-03. [2025-11-04T19:08:35.434792] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T19:08:35.444600] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-03. [2025-11-04T19:08:35.464956] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-11-04T19:08:36.167058] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-07. [2025-11-04T19:08:36.374029] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-11-04T19:08:36.569782] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T19:08:36.568429] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-11-04T19:08:36.859371] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T19:08:36.967011] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-04. [2025-11-04T19:08:37.767030] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-08. [2025-11-04T19:08:38.136073] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-11-04T19:08:38.567081] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-09. [2025-11-04T19:08:38.716985] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-11-04T19:08:39.002720] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-11-04T19:08:39.367036] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-05. [2025-11-04T19:08:40.167058] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-10. [2025-11-04T19:08:40.575607] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-11-04T19:08:40.848333] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T19:08:40.967034] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-11. [2025-11-04T19:08:41.604462] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T19:08:41.605760] [INFO ] [REMOTE ] GFR: Full fetch complete (11825 records). [2025-11-04T19:08:41.605835] [DEBUG] [VALIDATOR ] GFR: Validating primary data. [2025-11-04T19:08:41.780338] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-04. [2025-11-04T19:08:42.029273] [DEBUG] [VALIDATOR ] GFR: All expected trading days are present. [2025-11-04T19:08:42.029379] [DEBUG] [VALIDATOR ] GFR: Primary data validation passed. [2025-11-04T19:08:42.029868] [INFO ] [MAIN ] GFR: Fetched primary data is valid. [2025-11-04T19:08:42.029936] [DEBUG] [LOCAL ] GFR: Saving primary data. [2025-11-04T19:08:42.030500] [DEBUG] [LOCAL ] GFR: Trimmed 11825 records to 6909 (keeping 200 days of data) [2025-11-04T19:08:42.071669] [DEBUG] [LOCAL ] GFR: Primary data saved (6909 records). [2025-11-04T19:08:42.071864] [INFO ] [MAIN ] GFR: Full history saved. [2025-11-04T19:08:42.071977] [INFO ] [MAIN ] Update Progress: 921/2367 symbols completed. GFR took 12.2s. Estimated remaining: 3h 21m 59s. Rate: 7.2 symbols/min [2025-11-04T19:08:42.246511] [INFO ] [MAIN ] GGG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:08:42.289066] [DEBUG] [LOCAL ] GGG: Loaded 40329 existing records from primary storage. [2025-11-04T19:08:42.289514] [INFO ] [MAIN ] GGG: Loaded 40329 existing records. [2025-11-04T19:08:42.490775] [WARN ] [MAIN ] GGG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:08:42.491120] [INFO ] [MAIN ] GGG: Wiping existing primary data before full refresh. [2025-11-04T19:08:42.491178] [INFO ] [LOCAL ] GGG: Starting deletion of primary data. [2025-11-04T19:08:42.491486] [DEBUG] [LOCAL ] GGG: Deleted primary data file. [2025-11-04T19:08:42.491541] [INFO ] [MAIN ] GGG: Starting full primary refresh. [2025-11-04T19:08:42.574383] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-04. [2025-11-04T19:08:42.887243] [INFO ] [REMOTE ] GGG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:08:43.230044] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T19:08:43.367092] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-05. [2025-11-04T19:08:43.841949] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T19:08:44.167099] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-06. [2025-11-04T19:08:44.681291] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T19:08:44.967037] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-06. [2025-11-04T19:08:45.699271] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-11-04T19:08:45.767043] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-05. [2025-11-04T19:08:46.096965] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T19:08:46.567048] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-07. [2025-11-04T19:08:47.029324] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T19:08:47.367060] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-07. [2025-11-04T19:08:48.167073] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-06. [2025-11-04T19:08:48.259356] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-11-04T19:08:48.757544] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-11-04T19:08:48.967015] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-08. [2025-11-04T19:08:49.211675] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T19:08:49.767036] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-08. [2025-11-04T19:08:50.501016] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T19:08:50.567033] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-09. [2025-11-04T19:08:51.367049] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-01. [2025-11-04T19:08:51.433180] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-11-04T19:08:51.771996] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T19:08:52.167106] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-10. [2025-11-04T19:08:52.552522] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T19:08:52.967102] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-07. [2025-11-04T19:08:53.600805] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T19:08:53.767018] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-09. [2025-11-04T19:08:54.080893] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T19:08:54.567046] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-02. [2025-11-04T19:08:55.367097] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-11. [2025-11-04T19:08:55.634294] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-11-04T19:08:55.672724] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T19:08:56.167069] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-10. [2025-11-04T19:08:56.492455] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:08:56.494513] [INFO ] [REMOTE ] GGB: Full fetch complete (74365 records). [2025-11-04T19:08:56.494692] [DEBUG] [VALIDATOR ] GGB: Validating primary data. [2025-11-04T19:08:56.900572] [DEBUG] [VALIDATOR ] GGB: All expected trading days are present. [2025-11-04T19:08:56.900673] [DEBUG] [VALIDATOR ] GGB: Primary data validation passed. [2025-11-04T19:08:56.901618] [INFO ] [MAIN ] GGB: Fetched primary data is valid. [2025-11-04T19:08:56.901702] [DEBUG] [LOCAL ] GGB: Saving primary data. [2025-11-04T19:08:56.903077] [DEBUG] [LOCAL ] GGB: Trimmed 74365 records to 51612 (keeping 200 days of data) [2025-11-04T19:08:56.988762] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-08. [2025-11-04T19:08:57.218196] [DEBUG] [LOCAL ] GGB: Primary data saved (51612 records). [2025-11-04T19:08:57.218360] [INFO ] [MAIN ] GGB: Full history saved. [2025-11-04T19:08:57.218470] [INFO ] [MAIN ] Update Progress: 922/2367 symbols completed. GGB took 27.4s. Estimated remaining: 3h 22m 1s. Rate: 7.2 symbols/min [2025-11-04T19:08:57.385544] [INFO ] [MAIN ] GGT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:08:57.399228] [DEBUG] [LOCAL ] GGT: Loaded 14579 existing records from primary storage. [2025-11-04T19:08:57.399452] [INFO ] [MAIN ] GGT: Loaded 14579 existing records. [2025-11-04T19:08:57.575797] [WARN ] [MAIN ] GGT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:08:57.576021] [INFO ] [MAIN ] GGT: Wiping existing primary data before full refresh. [2025-11-04T19:08:57.576068] [INFO ] [LOCAL ] GGT: Starting deletion of primary data. [2025-11-04T19:08:57.576417] [DEBUG] [LOCAL ] GGT: Deleted primary data file. [2025-11-04T19:08:57.576474] [INFO ] [MAIN ] GGT: Starting full primary refresh. [2025-11-04T19:08:57.782607] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-03. [2025-11-04T19:08:57.935941] [INFO ] [REMOTE ] GGT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:08:58.059927] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-11-04T19:08:58.100732] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T19:08:58.567049] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-09. [2025-11-04T19:08:58.924189] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T19:08:59.367048] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-04. [2025-11-04T19:08:59.676512] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T19:09:00.167041] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-11. [2025-11-04T19:09:00.547886] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T19:09:00.967031] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-01. [2025-11-04T19:09:01.367936] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-01 (0.40 seconds). [2025-11-04T19:09:01.656614] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-11 (1.49 seconds). [2025-11-04T19:09:01.658777] [INFO ] [REMOTE ] GFI: Full fetch complete (77089 records). [2025-11-04T19:09:01.658904] [DEBUG] [VALIDATOR ] GFI: Validating primary data. [2025-11-04T19:09:01.772154] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-02. [2025-11-04T19:09:02.080496] [DEBUG] [VALIDATOR ] GFI: All expected trading days are present. [2025-11-04T19:09:02.080585] [DEBUG] [VALIDATOR ] GFI: Primary data validation passed. [2025-11-04T19:09:02.081574] [INFO ] [MAIN ] GFI: Fetched primary data is valid. [2025-11-04T19:09:02.081669] [DEBUG] [LOCAL ] GFI: Saving primary data. [2025-11-04T19:09:02.083233] [DEBUG] [LOCAL ] GFI: Trimmed 77089 records to 53114 (keeping 200 days of data) [2025-11-04T19:09:02.415977] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T19:09:02.424031] [DEBUG] [LOCAL ] GFI: Primary data saved (53114 records). [2025-11-04T19:09:02.424111] [INFO ] [MAIN ] GFI: Full history saved. [2025-11-04T19:09:02.424179] [INFO ] [MAIN ] Update Progress: 923/2367 symbols completed. GFI took 32.5s. Estimated remaining: 3h 21m 48s. Rate: 7.2 symbols/min [2025-11-04T19:09:02.575796] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-05. [2025-11-04T19:09:02.606134] [INFO ] [MAIN ] GGZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:09:02.610242] [DEBUG] [LOCAL ] GGZ: Loaded 2240 existing records from primary storage. [2025-11-04T19:09:02.610545] [INFO ] [MAIN ] GGZ: Loaded 2240 existing records. [2025-11-04T19:09:02.823075] [WARN ] [MAIN ] GGZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:09:02.823282] [INFO ] [MAIN ] GGZ: Wiping existing primary data before full refresh. [2025-11-04T19:09:02.823328] [INFO ] [LOCAL ] GGZ: Starting deletion of primary data. [2025-11-04T19:09:02.823511] [DEBUG] [LOCAL ] GGZ: Deleted primary data file. [2025-11-04T19:09:02.823567] [INFO ] [MAIN ] GGZ: Starting full primary refresh. [2025-11-04T19:09:03.192247] [INFO ] [REMOTE ] GGZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:09:03.367027] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-01. [2025-11-04T19:09:03.737893] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T19:09:03.769851] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-01 (0.40 seconds). [2025-11-04T19:09:04.167042] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-02. [2025-11-04T19:09:04.395760] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-02 (0.23 seconds). [2025-11-04T19:09:04.967024] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-03. [2025-11-04T19:09:05.284982] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-11-04T19:09:05.767112] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-03. [2025-11-04T19:09:06.357889] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-11-04T19:09:06.567036] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-04. [2025-11-04T19:09:07.095676] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T19:09:07.367055] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-06. [2025-11-04T19:09:08.167067] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-04. [2025-11-04T19:09:08.390093] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T19:09:08.420144] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-04 (0.25 seconds). [2025-11-04T19:09:08.967053] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-05. [2025-11-04T19:09:09.630068] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T19:09:09.767086] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-06. [2025-11-04T19:09:10.403005] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-11-04T19:09:10.567077] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-10. [2025-11-04T19:09:11.367039] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-07. [2025-11-04T19:09:11.806458] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-11-04T19:09:12.167056] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-07. [2025-11-04T19:09:12.448031] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T19:09:12.893525] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-11-04T19:09:12.967080] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-08. [2025-11-04T19:09:13.541945] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-11-04T19:09:13.767067] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-09. [2025-11-04T19:09:14.362983] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-11-04T19:09:14.567053] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-05. [2025-11-04T19:09:15.000818] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T19:09:15.367052] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-06. [2025-11-04T19:09:15.638534] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-11-04T19:09:16.167033] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-08. [2025-11-04T19:09:16.967085] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-10. [2025-11-04T19:09:17.231242] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-11-04T19:09:17.463065] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-11-04T19:09:17.767064] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-09. [2025-11-04T19:09:18.567054] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-11. [2025-11-04T19:09:18.764618] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T19:09:19.367081] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-10. [2025-11-04T19:09:19.594806] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T19:09:19.596766] [INFO ] [REMOTE ] GFL: Full fetch complete (69675 records). [2025-11-04T19:09:19.596891] [DEBUG] [VALIDATOR ] GFL: Validating primary data. [2025-11-04T19:09:19.999103] [DEBUG] [VALIDATOR ] GFL: All expected trading days are present. [2025-11-04T19:09:19.999223] [DEBUG] [VALIDATOR ] GFL: Primary data validation passed. [2025-11-04T19:09:20.000087] [INFO ] [MAIN ] GFL: Fetched primary data is valid. [2025-11-04T19:09:20.000165] [DEBUG] [LOCAL ] GFL: Saving primary data. [2025-11-04T19:09:20.001559] [DEBUG] [LOCAL ] GFL: Trimmed 69675 records to 47243 (keeping 200 days of data) [2025-11-04T19:09:20.173602] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-07. [2025-11-04T19:09:20.288930] [DEBUG] [LOCAL ] GFL: Primary data saved (47243 records). [2025-11-04T19:09:20.289078] [INFO ] [MAIN ] GFL: Full history saved. [2025-11-04T19:09:20.289163] [INFO ] [MAIN ] Update Progress: 924/2367 symbols completed. GFL took 50.4s. Estimated remaining: 3h 21m 54s. Rate: 7.1 symbols/min [2025-11-04T19:09:20.463707] [INFO ] [MAIN ] GHC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:09:20.468489] [DEBUG] [LOCAL ] GHC: Loaded 3152 existing records from primary storage. [2025-11-04T19:09:20.468833] [INFO ] [MAIN ] GHC: Loaded 3152 existing records. [2025-11-04T19:09:20.529616] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-11-04T19:09:20.618987] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T19:09:20.712720] [WARN ] [MAIN ] GHC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:09:20.712992] [INFO ] [MAIN ] GHC: Wiping existing primary data before full refresh. [2025-11-04T19:09:20.713043] [INFO ] [LOCAL ] GHC: Starting deletion of primary data. [2025-11-04T19:09:20.713293] [DEBUG] [LOCAL ] GHC: Deleted primary data file. [2025-11-04T19:09:20.713352] [INFO ] [MAIN ] GHC: Starting full primary refresh. [2025-11-04T19:09:20.972204] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-11. [2025-11-04T19:09:21.093238] [INFO ] [REMOTE ] GHC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:09:21.767047] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-11. [2025-11-04T19:09:22.060180] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T19:09:22.062385] [INFO ] [REMOTE ] GGG: Full fetch complete (59793 records). [2025-11-04T19:09:22.062479] [DEBUG] [VALIDATOR ] GGG: Validating primary data. [2025-11-04T19:09:22.378449] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T19:09:22.380194] [INFO ] [REMOTE ] GGT: Full fetch complete (19327 records). [2025-11-04T19:09:22.380336] [DEBUG] [VALIDATOR ] GGT: Validating primary data. [2025-11-04T19:09:22.569547] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-08. [2025-11-04T19:09:22.602771] [DEBUG] [VALIDATOR ] GGG: All expected trading days are present. [2025-11-04T19:09:22.630973] [DEBUG] [VALIDATOR ] GGG: Primary data validation passed. [2025-11-04T19:09:22.637914] [INFO ] [MAIN ] GGG: Fetched primary data is valid. [2025-11-04T19:09:22.640907] [DEBUG] [LOCAL ] GGG: Saving primary data. [2025-11-04T19:09:22.742862] [DEBUG] [LOCAL ] GGG: Trimmed 59793 records to 40335 (keeping 200 days of data) [2025-11-04T19:09:22.958445] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-11-04T19:09:22.963702] [DEBUG] [VALIDATOR ] GGT: All expected trading days are present. [2025-11-04T19:09:22.968135] [DEBUG] [VALIDATOR ] GGT: Primary data validation passed. [2025-11-04T19:09:22.970705] [INFO ] [MAIN ] GGT: Fetched primary data is valid. [2025-11-04T19:09:22.972697] [DEBUG] [LOCAL ] GGT: Saving primary data. [2025-11-04T19:09:22.998903] [DEBUG] [LOCAL ] GGT: Trimmed 19327 records to 14632 (keeping 200 days of data) [2025-11-04T19:09:23.198351] [DEBUG] [LOCAL ] GGT: Primary data saved (14632 records). [2025-11-04T19:09:23.200595] [INFO ] [MAIN ] GGT: Full history saved. [2025-11-04T19:09:23.202632] [INFO ] [MAIN ] Update Progress: 925/2367 symbols completed. GGT took 26.0s. Estimated remaining: 3h 21m 37s. Rate: 7.2 symbols/min [2025-11-04T19:09:23.324048] [DEBUG] [LOCAL ] GGG: Primary data saved (40335 records). [2025-11-04T19:09:23.335283] [INFO ] [MAIN ] GGG: Full history saved. [2025-11-04T19:09:23.340617] [INFO ] [MAIN ] Update Progress: 926/2367 symbols completed. GGG took 41.3s. Estimated remaining: 3h 21m 16s. Rate: 7.2 symbols/min [2025-11-04T19:09:23.375204] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-09. [2025-11-04T19:09:23.502002] [INFO ] [MAIN ] GHG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:09:23.554037] [DEBUG] [LOCAL ] GHG: Loaded 3287 existing records from primary storage. [2025-11-04T19:09:23.590585] [INFO ] [MAIN ] GHG: Loaded 3287 existing records. [2025-11-04T19:09:23.590360] [INFO ] [MAIN ] GHI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:09:23.734096] [DEBUG] [LOCAL ] GHI: Loaded 8373 existing records from primary storage. [2025-11-04T19:09:23.736762] [INFO ] [MAIN ] GHI: Loaded 8373 existing records. [2025-11-04T19:09:23.818531] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-11-04T19:09:23.960903] [WARN ] [MAIN ] GHG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:09:23.993586] [INFO ] [MAIN ] GHG: Wiping existing primary data before full refresh. [2025-11-04T19:09:24.001435] [INFO ] [LOCAL ] GHG: Starting deletion of primary data. [2025-11-04T19:09:24.018302] [DEBUG] [LOCAL ] GHG: Deleted primary data file. [2025-11-04T19:09:24.023141] [INFO ] [MAIN ] GHG: Starting full primary refresh. [2025-11-04T19:09:24.023085] [WARN ] [MAIN ] GHI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:09:24.065500] [INFO ] [MAIN ] GHI: Wiping existing primary data before full refresh. [2025-11-04T19:09:24.086018] [INFO ] [LOCAL ] GHI: Starting deletion of primary data. [2025-11-04T19:09:24.097228] [DEBUG] [LOCAL ] GHI: Deleted primary data file. [2025-11-04T19:09:24.112858] [INFO ] [MAIN ] GHI: Starting full primary refresh. [2025-11-04T19:09:24.174700] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-01. [2025-11-04T19:09:24.670026] [INFO ] [REMOTE ] GHG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:09:24.714721] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-11-04T19:09:24.734287] [INFO ] [REMOTE ] GHI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:09:24.967042] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-01. [2025-11-04T19:09:25.502228] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-11-04T19:09:25.767131] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-02. [2025-11-04T19:09:26.066870] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-02 (0.30 seconds). [2025-11-04T19:09:26.567091] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-03. [2025-11-04T19:09:26.905678] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-11-04T19:09:27.367071] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-01. [2025-11-04T19:09:27.653484] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-01 (0.29 seconds). [2025-11-04T19:09:28.167050] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-02. [2025-11-04T19:09:28.423182] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-02 (0.26 seconds). [2025-11-04T19:09:28.967126] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-02. [2025-11-04T19:09:29.338174] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-11-04T19:09:29.767128] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-03. [2025-11-04T19:09:30.075140] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-11-04T19:09:30.567118] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-10. [2025-11-04T19:09:30.869993] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-10 (0.30 seconds). [2025-11-04T19:09:31.367038] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-03. [2025-11-04T19:09:31.835461] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-11-04T19:09:32.167084] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-04. [2025-11-04T19:09:32.543323] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T19:09:32.967077] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-05. [2025-11-04T19:09:33.329057] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-11-04T19:09:33.767039] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-04. [2025-11-04T19:09:34.175842] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-11-04T19:09:34.567033] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-05. [2025-11-04T19:09:34.878143] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-11-04T19:09:35.367023] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-06. [2025-11-04T19:09:35.647853] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-06 (0.28 seconds). [2025-11-04T19:09:36.167034] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-07. [2025-11-04T19:09:36.567148] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-11-04T19:09:36.967071] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-06. [2025-11-04T19:09:37.365915] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-11-04T19:09:37.767047] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-08. [2025-11-04T19:09:38.081184] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-08 (0.31 seconds). [2025-11-04T19:09:38.567045] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-04. [2025-11-04T19:09:39.092470] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T19:09:39.367110] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-05. [2025-11-04T19:09:39.798185] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T19:09:40.167073] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-07. [2025-11-04T19:09:40.446637] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-07 (0.28 seconds). [2025-11-04T19:09:40.967066] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-11. [2025-11-04T19:09:41.341022] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-11 (0.37 seconds). [2025-11-04T19:09:41.341685] [INFO ] [REMOTE ] GGZ: Full fetch complete (3638 records). [2025-11-04T19:09:41.341754] [DEBUG] [VALIDATOR ] GGZ: Validating primary data. [2025-11-04T19:09:41.706355] [DEBUG] [VALIDATOR ] GGZ: All expected trading days are present. [2025-11-04T19:09:41.706434] [DEBUG] [VALIDATOR ] GGZ: Primary data validation passed. [2025-11-04T19:09:41.706555] [INFO ] [MAIN ] GGZ: Fetched primary data is valid. [2025-11-04T19:09:41.706593] [DEBUG] [LOCAL ] GGZ: Saving primary data. [2025-11-04T19:09:41.707048] [DEBUG] [LOCAL ] GGZ: Trimmed 3638 records to 2235 (keeping 200 days of data) [2025-11-04T19:09:41.721363] [DEBUG] [LOCAL ] GGZ: Primary data saved (2235 records). [2025-11-04T19:09:41.721499] [INFO ] [MAIN ] GGZ: Full history saved. [2025-11-04T19:09:41.721579] [INFO ] [MAIN ] Update Progress: 927/2367 symbols completed. GGZ took 39.3s. Estimated remaining: 3h 21m 23s. Rate: 7.2 symbols/min [2025-11-04T19:09:41.772143] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-09. [2025-11-04T19:09:41.901529] [INFO ] [MAIN ] GHLD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:09:41.906794] [DEBUG] [LOCAL ] GHLD: Loaded 4427 existing records from primary storage. [2025-11-04T19:09:41.906941] [INFO ] [MAIN ] GHLD: Loaded 4427 existing records. [2025-11-04T19:09:42.092114] [WARN ] [MAIN ] GHLD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:09:42.092305] [INFO ] [MAIN ] GHLD: Wiping existing primary data before full refresh. [2025-11-04T19:09:42.092351] [INFO ] [LOCAL ] GHLD: Starting deletion of primary data. [2025-11-04T19:09:42.092559] [DEBUG] [LOCAL ] GHLD: Deleted primary data file. [2025-11-04T19:09:42.092616] [INFO ] [MAIN ] GHLD: Starting full primary refresh. [2025-11-04T19:09:42.311419] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T19:09:42.455487] [INFO ] [REMOTE ] GHLD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:09:42.567004] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-01. [2025-11-04T19:09:42.814601] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-01 (0.25 seconds). [2025-11-04T19:09:43.367035] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-10. [2025-11-04T19:09:43.726144] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-11-04T19:09:44.167081] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-11. [2025-11-04T19:09:44.544245] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T19:09:44.544943] [INFO ] [REMOTE ] GHG: Full fetch complete (4281 records). [2025-11-04T19:09:44.545028] [DEBUG] [VALIDATOR ] GHG: Validating primary data. [2025-11-04T19:09:44.904708] [DEBUG] [VALIDATOR ] GHG: All expected trading days are present. [2025-11-04T19:09:44.904789] [DEBUG] [VALIDATOR ] GHG: Primary data validation passed. [2025-11-04T19:09:44.904919] [INFO ] [MAIN ] GHG: Fetched primary data is valid. [2025-11-04T19:09:44.904956] [DEBUG] [LOCAL ] GHG: Saving primary data. [2025-11-04T19:09:44.905304] [DEBUG] [LOCAL ] GHG: Trimmed 4281 records to 3240 (keeping 200 days of data) [2025-11-04T19:09:44.923766] [DEBUG] [LOCAL ] GHG: Primary data saved (3240 records). [2025-11-04T19:09:44.923830] [INFO ] [MAIN ] GHG: Full history saved. [2025-11-04T19:09:44.923894] [INFO ] [MAIN ] Update Progress: 928/2367 symbols completed. GHG took 21.7s. Estimated remaining: 3h 21m 7s. Rate: 7.2 symbols/min [2025-11-04T19:09:44.972265] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-02. [2025-11-04T19:09:45.096950] [INFO ] [MAIN ] GHM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:09:45.110896] [DEBUG] [LOCAL ] GHM: Loaded 14465 existing records from primary storage. [2025-11-04T19:09:45.111113] [INFO ] [MAIN ] GHM: Loaded 14465 existing records. [2025-11-04T19:09:45.292516] [WARN ] [MAIN ] GHM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:09:45.295698] [INFO ] [MAIN ] GHM: Wiping existing primary data before full refresh. [2025-11-04T19:09:45.295588] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T19:09:45.295890] [INFO ] [LOCAL ] GHM: Starting deletion of primary data. [2025-11-04T19:09:45.297374] [DEBUG] [LOCAL ] GHM: Deleted primary data file. [2025-11-04T19:09:45.297432] [INFO ] [MAIN ] GHM: Starting full primary refresh. [2025-11-04T19:09:45.646225] [INFO ] [REMOTE ] GHM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:09:45.767015] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-01. [2025-11-04T19:09:46.358986] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-11-04T19:09:46.567048] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-06. [2025-11-04T19:09:47.033491] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-11-04T19:09:47.367030] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-03. [2025-11-04T19:09:47.663307] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-03 (0.30 seconds). [2025-11-04T19:09:48.167113] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-07. [2025-11-04T19:09:48.702973] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-11-04T19:09:48.967066] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-02. [2025-11-04T19:09:49.689305] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T19:09:49.767083] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-03. [2025-11-04T19:09:50.489483] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-11-04T19:09:50.567057] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-04. [2025-11-04T19:09:50.878749] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-11-04T19:09:51.367057] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-08. [2025-11-04T19:09:51.857965] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-11-04T19:09:52.167033] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-09. [2025-11-04T19:09:52.665001] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-11-04T19:09:52.967055] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-08. [2025-11-04T19:09:53.310919] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-11-04T19:09:53.767070] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-10. [2025-11-04T19:09:54.345052] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-11-04T19:09:54.567039] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-11. [2025-11-04T19:09:55.249846] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T19:09:55.250820] [INFO ] [REMOTE ] GHI: Full fetch complete (11662 records). [2025-11-04T19:09:55.250888] [DEBUG] [VALIDATOR ] GHI: Validating primary data. [2025-11-04T19:09:55.372149] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-09. [2025-11-04T19:09:55.653958] [DEBUG] [VALIDATOR ] GHI: All expected trading days are present. [2025-11-04T19:09:55.654064] [DEBUG] [VALIDATOR ] GHI: Primary data validation passed. [2025-11-04T19:09:55.654617] [INFO ] [MAIN ] GHI: Fetched primary data is valid. [2025-11-04T19:09:55.654727] [DEBUG] [LOCAL ] GHI: Saving primary data. [2025-11-04T19:09:55.655351] [DEBUG] [LOCAL ] GHI: Trimmed 11662 records to 8431 (keeping 200 days of data) [2025-11-04T19:09:55.713099] [DEBUG] [LOCAL ] GHI: Primary data saved (8431 records). [2025-11-04T19:09:55.713239] [INFO ] [MAIN ] GHI: Full history saved. [2025-11-04T19:09:55.713332] [INFO ] [MAIN ] Update Progress: 929/2367 symbols completed. GHI took 32.4s. Estimated remaining: 3h 21m 2s. Rate: 7.2 symbols/min [2025-11-04T19:09:55.856333] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-11-04T19:09:55.926176] [INFO ] [MAIN ] GHY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:09:55.946396] [DEBUG] [LOCAL ] GHY: Loaded 20391 existing records from primary storage. [2025-11-04T19:09:55.946689] [INFO ] [MAIN ] GHY: Loaded 20391 existing records. [2025-11-04T19:09:56.126881] [WARN ] [MAIN ] GHY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:09:56.127130] [INFO ] [MAIN ] GHY: Wiping existing primary data before full refresh. [2025-11-04T19:09:56.127180] [INFO ] [LOCAL ] GHY: Starting deletion of primary data. [2025-11-04T19:09:56.127634] [DEBUG] [LOCAL ] GHY: Deleted primary data file. [2025-11-04T19:09:56.127734] [INFO ] [MAIN ] GHY: Starting full primary refresh. [2025-11-04T19:09:56.172196] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-04. [2025-11-04T19:09:56.497482] [INFO ] [REMOTE ] GHY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:09:56.812630] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-11-04T19:09:56.967093] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-05. [2025-11-04T19:09:57.608176] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T19:09:57.767034] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-05. [2025-11-04T19:09:58.061857] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-05 (0.29 seconds). [2025-11-04T19:09:58.567070] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-01. [2025-11-04T19:09:59.367008] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-06. [2025-11-04T19:09:59.448055] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T19:09:59.658253] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-06 (0.29 seconds). [2025-11-04T19:10:00.167048] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-06. [2025-11-04T19:10:00.919702] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-11-04T19:10:00.967042] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-07. [2025-11-04T19:10:01.767047] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-10. [2025-11-04T19:10:01.794995] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T19:10:02.260070] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-11-04T19:10:02.567045] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-08. [2025-11-04T19:10:03.324759] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-11-04T19:10:03.367072] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-09. [2025-11-04T19:10:03.943904] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-11-04T19:10:04.167045] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-02. [2025-11-04T19:10:04.843926] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-11-04T19:10:04.967048] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-03. [2025-11-04T19:10:05.728952] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T19:10:05.767018] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-04. [2025-11-04T19:10:06.555509] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-11-04T19:10:06.567000] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-05. [2025-11-04T19:10:07.322815] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-11-04T19:10:07.367051] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-07. [2025-11-04T19:10:07.781410] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-11-04T19:10:08.167040] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-08. [2025-11-04T19:10:08.640302] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-11-04T19:10:08.967098] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-10. [2025-11-04T19:10:09.723441] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-11-04T19:10:09.767035] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-11. [2025-11-04T19:10:10.508087] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T19:10:10.509458] [INFO ] [REMOTE ] GHM: Full fetch complete (22077 records). [2025-11-04T19:10:10.509548] [DEBUG] [VALIDATOR ] GHM: Validating primary data. [2025-11-04T19:10:10.575314] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-11. [2025-11-04T19:10:10.913982] [DEBUG] [VALIDATOR ] GHM: All expected trading days are present. [2025-11-04T19:10:10.914104] [DEBUG] [VALIDATOR ] GHM: Primary data validation passed. [2025-11-04T19:10:10.915218] [INFO ] [MAIN ] GHM: Fetched primary data is valid. [2025-11-04T19:10:10.915293] [DEBUG] [LOCAL ] GHM: Saving primary data. [2025-11-04T19:10:10.916417] [DEBUG] [LOCAL ] GHM: Trimmed 22077 records to 14536 (keeping 200 days of data) [2025-11-04T19:10:11.010390] [DEBUG] [LOCAL ] GHM: Primary data saved (14536 records). [2025-11-04T19:10:11.010608] [INFO ] [MAIN ] GHM: Full history saved. [2025-11-04T19:10:11.010750] [INFO ] [MAIN ] Update Progress: 930/2367 symbols completed. GHM took 26.1s. Estimated remaining: 3h 21m 4s. Rate: 7.1 symbols/min [2025-11-04T19:10:11.200332] [INFO ] [MAIN ] GIB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:10:11.236242] [DEBUG] [LOCAL ] GIB: Loaded 30852 existing records from primary storage. [2025-11-04T19:10:11.236609] [INFO ] [MAIN ] GIB: Loaded 30852 existing records. [2025-11-04T19:10:11.358878] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T19:10:11.368086] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-06. [2025-11-04T19:10:11.382965] [INFO ] [REMOTE ] GHC: Full fetch complete (4084 records). [2025-11-04T19:10:11.407478] [DEBUG] [VALIDATOR ] GHC: Validating primary data. [2025-11-04T19:10:11.521414] [WARN ] [MAIN ] GIB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:10:11.543050] [INFO ] [MAIN ] GIB: Wiping existing primary data before full refresh. [2025-11-04T19:10:11.548258] [INFO ] [LOCAL ] GIB: Starting deletion of primary data. [2025-11-04T19:10:11.575614] [DEBUG] [LOCAL ] GIB: Deleted primary data file. [2025-11-04T19:10:11.585974] [INFO ] [MAIN ] GIB: Starting full primary refresh. [2025-11-04T19:10:12.088775] [DEBUG] [VALIDATOR ] GHC: All expected trading days are present. [2025-11-04T19:10:12.114793] [DEBUG] [VALIDATOR ] GHC: Primary data validation passed. [2025-11-04T19:10:12.138115] [INFO ] [MAIN ] GHC: Fetched primary data is valid. [2025-11-04T19:10:12.156356] [DEBUG] [LOCAL ] GHC: Saving primary data. [2025-11-04T19:10:12.173316] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-09. [2025-11-04T19:10:12.161810] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T19:10:12.184472] [DEBUG] [LOCAL ] GHC: Trimmed 4084 records to 3294 (keeping 200 days of data) [2025-11-04T19:10:12.265444] [INFO ] [REMOTE ] GIB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:10:12.279135] [DEBUG] [LOCAL ] GHC: Primary data saved (3294 records). [2025-11-04T19:10:12.279328] [INFO ] [MAIN ] GHC: Full history saved. [2025-11-04T19:10:12.279432] [INFO ] [MAIN ] Update Progress: 931/2367 symbols completed. GHC took 52.0s. Estimated remaining: 3h 20m 45s. Rate: 7.2 symbols/min [2025-11-04T19:10:12.460838] [INFO ] [MAIN ] GIC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:10:12.476338] [DEBUG] [LOCAL ] GIC: Loaded 13224 existing records from primary storage. [2025-11-04T19:10:12.476690] [INFO ] [MAIN ] GIC: Loaded 13224 existing records. [2025-11-04T19:10:12.671174] [WARN ] [MAIN ] GIC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:10:12.676751] [INFO ] [MAIN ] GIC: Wiping existing primary data before full refresh. [2025-11-04T19:10:12.676948] [INFO ] [LOCAL ] GIC: Starting deletion of primary data. [2025-11-04T19:10:12.677957] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-11-04T19:10:12.678011] [DEBUG] [LOCAL ] GIC: Deleted primary data file. [2025-11-04T19:10:12.679813] [INFO ] [MAIN ] GIC: Starting full primary refresh. [2025-11-04T19:10:12.972285] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-01. [2025-11-04T19:10:13.066522] [INFO ] [REMOTE ] GIC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:10:13.767045] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-10. [2025-11-04T19:10:13.920080] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T19:10:14.088961] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-10 (0.32 seconds). [2025-11-04T19:10:14.567083] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-11. [2025-11-04T19:10:14.958403] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-11 (0.39 seconds). [2025-11-04T19:10:14.959464] [INFO ] [REMOTE ] GHLD: Full fetch complete (5284 records). [2025-11-04T19:10:14.959549] [DEBUG] [VALIDATOR ] GHLD: Validating primary data. [2025-11-04T19:10:15.367118] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-02. [2025-11-04T19:10:15.395620] [DEBUG] [VALIDATOR ] GHLD: All expected trading days are present. [2025-11-04T19:10:15.395752] [DEBUG] [VALIDATOR ] GHLD: Primary data validation passed. [2025-11-04T19:10:15.396003] [INFO ] [MAIN ] GHLD: Fetched primary data is valid. [2025-11-04T19:10:15.396073] [DEBUG] [LOCAL ] GHLD: Saving primary data. [2025-11-04T19:10:15.396590] [DEBUG] [LOCAL ] GHLD: Trimmed 5284 records to 4425 (keeping 200 days of data) [2025-11-04T19:10:15.427476] [DEBUG] [LOCAL ] GHLD: Primary data saved (4425 records). [2025-11-04T19:10:15.427611] [INFO ] [MAIN ] GHLD: Full history saved. [2025-11-04T19:10:15.427735] [INFO ] [MAIN ] Update Progress: 932/2367 symbols completed. GHLD took 33.7s. Estimated remaining: 3h 20m 28s. Rate: 7.2 symbols/min [2025-11-04T19:10:15.619943] [INFO ] [MAIN ] GIL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:10:15.661578] [DEBUG] [LOCAL ] GIL: Loaded 43941 existing records from primary storage. [2025-11-04T19:10:15.661962] [INFO ] [MAIN ] GIL: Loaded 43941 existing records. [2025-11-04T19:10:15.880584] [WARN ] [MAIN ] GIL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:10:15.880872] [INFO ] [MAIN ] GIL: Wiping existing primary data before full refresh. [2025-11-04T19:10:15.880923] [INFO ] [LOCAL ] GIL: Starting deletion of primary data. [2025-11-04T19:10:15.882183] [DEBUG] [LOCAL ] GIL: Deleted primary data file. [2025-11-04T19:10:15.882256] [INFO ] [MAIN ] GIL: Starting full primary refresh. [2025-11-04T19:10:16.171612] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-01. [2025-11-04T19:10:16.318205] [INFO ] [REMOTE ] GIL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:10:16.338879] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T19:10:16.967047] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-01. [2025-11-04T19:10:17.037534] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T19:10:17.767075] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-02. [2025-11-04T19:10:17.960427] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T19:10:18.290545] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-11-04T19:10:18.567056] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-02. [2025-11-04T19:10:19.367032] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-03. [2025-11-04T19:10:19.674421] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T19:10:19.929817] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-11-04T19:10:20.167082] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-03. [2025-11-04T19:10:20.967088] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-04. [2025-11-04T19:10:21.304702] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T19:10:21.550167] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-11-04T19:10:21.767087] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-04. [2025-11-04T19:10:22.567072] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-07. [2025-11-04T19:10:22.850887] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T19:10:23.321469] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-11-04T19:10:23.367008] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-05. [2025-11-04T19:10:24.006328] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T19:10:24.167093] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-05. [2025-11-04T19:10:24.967021] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-06. [2025-11-04T19:10:25.109854] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T19:10:25.496005] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-11-04T19:10:25.767102] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-08. [2025-11-04T19:10:26.466463] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T19:10:26.567032] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-09. [2025-11-04T19:10:27.296534] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-11-04T19:10:27.367052] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-07. [2025-11-04T19:10:27.845171] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-11-04T19:10:28.167058] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-03. [2025-11-04T19:10:28.967055] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-08. [2025-11-04T19:10:29.353571] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T19:10:29.671880] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T19:10:29.767044] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-04. [2025-11-04T19:10:30.567039] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-09. [2025-11-04T19:10:30.963038] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T19:10:31.197099] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:10:31.367041] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-05. [2025-11-04T19:10:32.167091] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-10. [2025-11-04T19:10:32.519843] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T19:10:32.810605] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-11-04T19:10:32.967122] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-06. [2025-11-04T19:10:33.751519] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T19:10:33.766998] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-07. [2025-11-04T19:10:34.567095] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-10. [2025-11-04T19:10:34.719750] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-11-04T19:10:35.367028] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-08. [2025-11-04T19:10:35.393993] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-11-04T19:10:36.167024] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-11. [2025-11-04T19:10:36.268729] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-11-04T19:10:36.851352] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T19:10:36.852989] [INFO ] [REMOTE ] GHY: Full fetch complete (31582 records). [2025-11-04T19:10:36.853081] [DEBUG] [VALIDATOR ] GHY: Validating primary data. [2025-11-04T19:10:36.968378] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-06. [2025-11-04T19:10:37.250812] [DEBUG] [VALIDATOR ] GHY: All expected trading days are present. [2025-11-04T19:10:37.250935] [DEBUG] [VALIDATOR ] GHY: Primary data validation passed. [2025-11-04T19:10:37.251744] [INFO ] [MAIN ] GHY: Fetched primary data is valid. [2025-11-04T19:10:37.251809] [DEBUG] [LOCAL ] GHY: Saving primary data. [2025-11-04T19:10:37.252825] [DEBUG] [LOCAL ] GHY: Trimmed 31582 records to 20298 (keeping 200 days of data) [2025-11-04T19:10:37.384547] [DEBUG] [LOCAL ] GHY: Primary data saved (20298 records). [2025-11-04T19:10:37.384773] [INFO ] [MAIN ] GHY: Full history saved. [2025-11-04T19:10:37.384893] [INFO ] [MAIN ] Update Progress: 933/2367 symbols completed. GHY took 41.7s. Estimated remaining: 3h 20m 41s. Rate: 7.1 symbols/min [2025-11-04T19:10:37.572058] [INFO ] [MAIN ] GIS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:10:37.625267] [DEBUG] [LOCAL ] GIS: Loaded 53677 existing records from primary storage. [2025-11-04T19:10:37.625741] [INFO ] [MAIN ] GIS: Loaded 53677 existing records. [2025-11-04T19:10:37.772280] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-09. [2025-11-04T19:10:37.853071] [WARN ] [MAIN ] GIS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:10:37.853447] [INFO ] [MAIN ] GIS: Wiping existing primary data before full refresh. [2025-11-04T19:10:37.853505] [INFO ] [LOCAL ] GIS: Starting deletion of primary data. [2025-11-04T19:10:37.854810] [DEBUG] [LOCAL ] GIS: Deleted primary data file. [2025-11-04T19:10:37.854885] [INFO ] [MAIN ] GIS: Starting full primary refresh. [2025-11-04T19:10:38.248845] [INFO ] [REMOTE ] GIS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:10:38.272050] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T19:10:38.567050] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-07. [2025-11-04T19:10:38.629174] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-11-04T19:10:39.367086] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-10. [2025-11-04T19:10:39.649712] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T19:10:40.167099] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-08. [2025-11-04T19:10:40.329994] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-11-04T19:10:40.967075] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-11. [2025-11-04T19:10:41.490946] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-11-04T19:10:41.767042] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-11. [2025-11-04T19:10:41.833154] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T19:10:41.834634] [INFO ] [REMOTE ] GIB: Full fetch complete (46781 records). [2025-11-04T19:10:41.834739] [DEBUG] [VALIDATOR ] GIB: Validating primary data. [2025-11-04T19:10:42.211681] [DEBUG] [VALIDATOR ] GIB: All expected trading days are present. [2025-11-04T19:10:42.211780] [DEBUG] [VALIDATOR ] GIB: Primary data validation passed. [2025-11-04T19:10:42.212432] [INFO ] [MAIN ] GIB: Fetched primary data is valid. [2025-11-04T19:10:42.212475] [DEBUG] [LOCAL ] GIB: Saving primary data. [2025-11-04T19:10:42.213573] [DEBUG] [LOCAL ] GIB: Trimmed 46781 records to 30825 (keeping 200 days of data) [2025-11-04T19:10:42.400345] [DEBUG] [LOCAL ] GIB: Primary data saved (30825 records). [2025-11-04T19:10:42.403768] [INFO ] [MAIN ] GIB: Full history saved. [2025-11-04T19:10:42.403988] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T19:10:42.404213] [INFO ] [MAIN ] Update Progress: 934/2367 symbols completed. GIB took 31.4s. Estimated remaining: 3h 20m 27s. Rate: 7.1 symbols/min [2025-11-04T19:10:42.410885] [INFO ] [REMOTE ] GIC: Full fetch complete (19950 records). [2025-11-04T19:10:42.476692] [DEBUG] [VALIDATOR ] GIC: Validating primary data. [2025-11-04T19:10:42.577588] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-09. [2025-11-04T19:10:42.698806] [INFO ] [MAIN ] GKOS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:10:43.049405] [DEBUG] [VALIDATOR ] GIC: All expected trading days are present. [2025-11-04T19:10:43.049271] [DEBUG] [LOCAL ] GKOS: Loaded 42499 existing records from primary storage. [2025-11-04T19:10:43.049692] [DEBUG] [VALIDATOR ] GIC: Primary data validation passed. [2025-11-04T19:10:43.050562] [INFO ] [MAIN ] GIC: Fetched primary data is valid. [2025-11-04T19:10:43.050783] [INFO ] [MAIN ] GKOS: Loaded 42499 existing records. [2025-11-04T19:10:43.050861] [DEBUG] [LOCAL ] GIC: Saving primary data. [2025-11-04T19:10:43.157680] [DEBUG] [LOCAL ] GIC: Trimmed 19950 records to 13138 (keeping 200 days of data) [2025-11-04T19:10:43.311236] [WARN ] [MAIN ] GKOS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:10:43.319139] [INFO ] [MAIN ] GKOS: Wiping existing primary data before full refresh. [2025-11-04T19:10:43.320792] [INFO ] [LOCAL ] GKOS: Starting deletion of primary data. [2025-11-04T19:10:43.328122] [DEBUG] [LOCAL ] GKOS: Deleted primary data file. [2025-11-04T19:10:43.329409] [INFO ] [MAIN ] GKOS: Starting full primary refresh. [2025-11-04T19:10:43.332685] [DEBUG] [LOCAL ] GIC: Primary data saved (13138 records). [2025-11-04T19:10:43.375000] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-01. [2025-11-04T19:10:43.396680] [INFO ] [MAIN ] GIC: Full history saved. [2025-11-04T19:10:43.408381] [INFO ] [MAIN ] Update Progress: 935/2367 symbols completed. GIC took 31.1s. Estimated remaining: 3h 20m 8s. Rate: 7.2 symbols/min [2025-11-04T19:10:43.724978] [INFO ] [MAIN ] GL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:10:43.887493] [INFO ] [REMOTE ] GKOS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:10:43.944931] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-11-04T19:10:43.958605] [DEBUG] [LOCAL ] GL: Loaded 37187 existing records from primary storage. [2025-11-04T19:10:43.959940] [INFO ] [MAIN ] GL: Loaded 37187 existing records. [2025-11-04T19:10:44.151966] [WARN ] [MAIN ] GL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:10:44.152202] [INFO ] [MAIN ] GL: Wiping existing primary data before full refresh. [2025-11-04T19:10:44.152250] [INFO ] [LOCAL ] GL: Starting deletion of primary data. [2025-11-04T19:10:44.153047] [DEBUG] [LOCAL ] GL: Deleted primary data file. [2025-11-04T19:10:44.153104] [INFO ] [MAIN ] GL: Starting full primary refresh. [2025-11-04T19:10:44.173321] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-10. [2025-11-04T19:10:44.503976] [INFO ] [REMOTE ] GL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:10:44.889071] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-11-04T19:10:44.967083] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-01. [2025-11-04T19:10:45.463400] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T19:10:45.767072] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-11. [2025-11-04T19:10:45.937736] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-11-04T19:10:46.567040] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-02. [2025-11-04T19:10:46.924338] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-11 (1.16 seconds). [2025-11-04T19:10:46.926197] [INFO ] [REMOTE ] GIL: Full fetch complete (63013 records). [2025-11-04T19:10:46.926278] [DEBUG] [VALIDATOR ] GIL: Validating primary data. [2025-11-04T19:10:47.313136] [DEBUG] [VALIDATOR ] GIL: All expected trading days are present. [2025-11-04T19:10:47.313212] [DEBUG] [VALIDATOR ] GIL: Primary data validation passed. [2025-11-04T19:10:47.314016] [INFO ] [MAIN ] GIL: Fetched primary data is valid. [2025-11-04T19:10:47.314077] [DEBUG] [LOCAL ] GIL: Saving primary data. [2025-11-04T19:10:47.315252] [DEBUG] [LOCAL ] GIL: Trimmed 63013 records to 43992 (keeping 200 days of data) [2025-11-04T19:10:47.374366] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-01. [2025-11-04T19:10:47.569526] [DEBUG] [LOCAL ] GIL: Primary data saved (43992 records). [2025-11-04T19:10:47.569671] [INFO ] [MAIN ] GIL: Full history saved. [2025-11-04T19:10:47.569764] [INFO ] [MAIN ] Update Progress: 936/2367 symbols completed. GIL took 32.1s. Estimated remaining: 3h 19m 53s. Rate: 7.2 symbols/min [2025-11-04T19:10:47.740874] [INFO ] [MAIN ] GLOB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:10:47.781355] [DEBUG] [LOCAL ] GLOB: Loaded 48191 existing records from primary storage. [2025-11-04T19:10:47.781759] [INFO ] [MAIN ] GLOB: Loaded 48191 existing records. [2025-11-04T19:10:47.962567] [WARN ] [MAIN ] GLOB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:10:47.962872] [INFO ] [MAIN ] GLOB: Wiping existing primary data before full refresh. [2025-11-04T19:10:47.962928] [INFO ] [LOCAL ] GLOB: Starting deletion of primary data. [2025-11-04T19:10:47.963828] [DEBUG] [LOCAL ] GLOB: Deleted primary data file. [2025-11-04T19:10:47.963893] [INFO ] [MAIN ] GLOB: Starting full primary refresh. [2025-11-04T19:10:48.168979] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-02. [2025-11-04T19:10:48.179210] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-11-04T19:10:48.369099] [INFO ] [REMOTE ] GLOB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:10:48.396759] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T19:10:48.967096] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-02. [2025-11-04T19:10:49.290731] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-11-04T19:10:49.767092] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-03. [2025-11-04T19:10:50.032245] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T19:10:50.567032] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-03. [2025-11-04T19:10:50.852447] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T19:10:51.367041] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-04. [2025-11-04T19:10:51.847621] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T19:10:52.167056] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-04. [2025-11-04T19:10:52.704328] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T19:10:52.967031] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-01. [2025-11-04T19:10:53.395080] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-11-04T19:10:53.767033] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-05. [2025-11-04T19:10:53.885927] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-11-04T19:10:54.567033] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-05. [2025-11-04T19:10:54.823029] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T19:10:55.367035] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-06. [2025-11-04T19:10:55.710716] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T19:10:56.167082] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-06. [2025-11-04T19:10:56.292527] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T19:10:56.967066] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-02. [2025-11-04T19:10:57.287208] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T19:10:57.767065] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-03. [2025-11-04T19:10:58.224033] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T19:10:58.567084] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-07. [2025-11-04T19:10:59.087464] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T19:10:59.367105] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-04. [2025-11-04T19:10:59.587557] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T19:11:00.167067] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-07. [2025-11-04T19:11:00.967169] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-08. [2025-11-04T19:11:01.098822] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-11-04T19:11:01.296634] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T19:11:01.767081] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-08. [2025-11-04T19:11:02.058871] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-11-04T19:11:02.567046] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-09. [2025-11-04T19:11:02.899986] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T19:11:03.367084] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-09. [2025-11-04T19:11:03.560056] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T19:11:04.167042] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-03. [2025-11-04T19:11:04.324292] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T19:11:04.967037] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-10. [2025-11-04T19:11:05.529406] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T19:11:05.767047] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-05. [2025-11-04T19:11:06.176113] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-11-04T19:11:06.567039] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-11. [2025-11-04T19:11:07.311181] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T19:11:07.366999] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-06. [2025-11-04T19:11:07.558880] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T19:11:07.560727] [INFO ] [REMOTE ] GL: Full fetch complete (54881 records). [2025-11-04T19:11:07.560829] [DEBUG] [VALIDATOR ] GL: Validating primary data. [2025-11-04T19:11:08.029360] [DEBUG] [VALIDATOR ] GL: All expected trading days are present. [2025-11-04T19:11:08.029457] [DEBUG] [VALIDATOR ] GL: Primary data validation passed. [2025-11-04T19:11:08.030321] [INFO ] [MAIN ] GL: Fetched primary data is valid. [2025-11-04T19:11:08.030385] [DEBUG] [LOCAL ] GL: Saving primary data. [2025-11-04T19:11:08.031691] [DEBUG] [LOCAL ] GL: Trimmed 54881 records to 37154 (keeping 200 days of data) [2025-11-04T19:11:08.167319] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-04. [2025-11-04T19:11:08.289523] [DEBUG] [LOCAL ] GL: Primary data saved (37154 records). [2025-11-04T19:11:08.289701] [INFO ] [MAIN ] GL: Full history saved. [2025-11-04T19:11:08.289807] [INFO ] [MAIN ] Update Progress: 937/2367 symbols completed. GL took 24.9s. Estimated remaining: 3h 20m 3s. Rate: 7.1 symbols/min [2025-11-04T19:11:08.487445] [INFO ] [MAIN ] GLP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:11:08.498635] [DEBUG] [LOCAL ] GLP: Loaded 9064 existing records from primary storage. [2025-11-04T19:11:08.498891] [INFO ] [MAIN ] GLP: Loaded 9064 existing records. [2025-11-04T19:11:08.708066] [WARN ] [MAIN ] GLP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:11:08.708390] [INFO ] [MAIN ] GLP: Wiping existing primary data before full refresh. [2025-11-04T19:11:08.708560] [INFO ] [LOCAL ] GLP: Starting deletion of primary data. [2025-11-04T19:11:08.708847] [DEBUG] [LOCAL ] GLP: Deleted primary data file. [2025-11-04T19:11:08.708908] [INFO ] [MAIN ] GLP: Starting full primary refresh. [2025-11-04T19:11:08.976028] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-10. [2025-11-04T19:11:09.094705] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-11-04T19:11:09.099279] [INFO ] [REMOTE ] GLP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:11:09.393222] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-11-04T19:11:09.767034] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-01. [2025-11-04T19:11:10.328117] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T19:11:10.498199] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-11-04T19:11:10.567051] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-07. [2025-11-04T19:11:11.367064] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-11. [2025-11-04T19:11:12.173727] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-05. [2025-11-04T19:11:12.179737] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-11-04T19:11:12.514595] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T19:11:12.516450] [INFO ] [REMOTE ] GKOS: Full fetch complete (61062 records). [2025-11-04T19:11:12.516538] [DEBUG] [VALIDATOR ] GKOS: Validating primary data. [2025-11-04T19:11:12.984347] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-08. [2025-11-04T19:11:12.988802] [DEBUG] [VALIDATOR ] GKOS: All expected trading days are present. [2025-11-04T19:11:12.988883] [DEBUG] [VALIDATOR ] GKOS: Primary data validation passed. [2025-11-04T19:11:12.989809] [INFO ] [MAIN ] GKOS: Fetched primary data is valid. [2025-11-04T19:11:12.989875] [DEBUG] [LOCAL ] GKOS: Saving primary data. [2025-11-04T19:11:12.991277] [DEBUG] [LOCAL ] GKOS: Trimmed 61062 records to 42523 (keeping 200 days of data) [2025-11-04T19:11:13.271950] [DEBUG] [LOCAL ] GKOS: Primary data saved (42523 records). [2025-11-04T19:11:13.272120] [INFO ] [MAIN ] GKOS: Full history saved. [2025-11-04T19:11:13.272210] [INFO ] [MAIN ] Update Progress: 938/2367 symbols completed. GKOS took 30.9s. Estimated remaining: 3h 19m 50s. Rate: 7.2 symbols/min [2025-11-04T19:11:13.481793] [INFO ] [MAIN ] GLW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:11:13.534025] [DEBUG] [LOCAL ] GLW: Loaded 53636 existing records from primary storage. [2025-11-04T19:11:13.534342] [INFO ] [MAIN ] GLW: Loaded 53636 existing records. [2025-11-04T19:11:13.762007] [WARN ] [MAIN ] GLW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:11:13.762235] [INFO ] [MAIN ] GLW: Wiping existing primary data before full refresh. [2025-11-04T19:11:13.762418] [INFO ] [LOCAL ] GLW: Starting deletion of primary data. [2025-11-04T19:11:13.763372] [DEBUG] [LOCAL ] GLW: Deleted primary data file. [2025-11-04T19:11:13.763433] [INFO ] [MAIN ] GLW: Starting full primary refresh. [2025-11-04T19:11:13.769047] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-02. [2025-11-04T19:11:13.995455] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-11-04T19:11:14.183915] [INFO ] [REMOTE ] GLW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:11:14.455846] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-11-04T19:11:14.567014] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-06. [2025-11-04T19:11:14.620877] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-11-04T19:11:15.367052] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-01. [2025-11-04T19:11:15.847209] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T19:11:16.167047] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-07. [2025-11-04T19:11:16.900115] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-11-04T19:11:16.967047] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-02. [2025-11-04T19:11:17.537512] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T19:11:17.767050] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-08. [2025-11-04T19:11:18.357016] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-11-04T19:11:18.567032] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-09. [2025-11-04T19:11:19.318337] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-11-04T19:11:19.367027] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-03. [2025-11-04T19:11:20.073727] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T19:11:20.167020] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-03. [2025-11-04T19:11:20.334028] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-11-04T19:11:20.967026] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-09. [2025-11-04T19:11:21.757292] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-11-04T19:11:21.768914] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-10. [2025-11-04T19:11:22.384239] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T19:11:22.567010] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-04. [2025-11-04T19:11:23.367006] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-04. [2025-11-04T19:11:23.533006] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-11-04T19:11:23.863340] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-11-04T19:11:24.093502] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-11-04T19:11:24.167014] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-11. [2025-11-04T19:11:24.505292] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T19:11:24.507374] [INFO ] [REMOTE ] GIS: Full fetch complete (77844 records). [2025-11-04T19:11:24.507490] [DEBUG] [VALIDATOR ] GIS: Validating primary data. [2025-11-04T19:11:24.904927] [DEBUG] [VALIDATOR ] GIS: All expected trading days are present. [2025-11-04T19:11:24.905002] [DEBUG] [VALIDATOR ] GIS: Primary data validation passed. [2025-11-04T19:11:24.905885] [INFO ] [MAIN ] GIS: Fetched primary data is valid. [2025-11-04T19:11:24.905947] [DEBUG] [LOCAL ] GIS: Saving primary data. [2025-11-04T19:11:24.907374] [DEBUG] [LOCAL ] GIS: Trimmed 77844 records to 53677 (keeping 200 days of data) [2025-11-04T19:11:24.970728] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-05. [2025-11-04T19:11:25.236508] [DEBUG] [LOCAL ] GIS: Primary data saved (53677 records). [2025-11-04T19:11:25.236633] [INFO ] [MAIN ] GIS: Full history saved. [2025-11-04T19:11:25.236784] [INFO ] [MAIN ] Update Progress: 939/2367 symbols completed. GIS took 47.9s. Estimated remaining: 3h 19m 47s. Rate: 7.1 symbols/min [2025-11-04T19:11:25.404745] [INFO ] [MAIN ] GM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:11:25.452738] [DEBUG] [LOCAL ] GM: Loaded 53729 existing records from primary storage. [2025-11-04T19:11:25.453044] [INFO ] [MAIN ] GM: Loaded 53729 existing records. [2025-11-04T19:11:25.641381] [WARN ] [MAIN ] GM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:11:25.642226] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T19:11:25.642397] [INFO ] [MAIN ] GM: Wiping existing primary data before full refresh. [2025-11-04T19:11:25.645211] [INFO ] [LOCAL ] GM: Starting deletion of primary data. [2025-11-04T19:11:25.645687] [DEBUG] [LOCAL ] GM: Deleted primary data file. [2025-11-04T19:11:25.645744] [INFO ] [MAIN ] GM: Starting full primary refresh. [2025-11-04T19:11:25.772116] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-05. [2025-11-04T19:11:26.016664] [INFO ] [REMOTE ] GM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:11:26.567034] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-01. [2025-11-04T19:11:27.201171] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T19:11:27.367009] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-06. [2025-11-04T19:11:28.167006] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-06. [2025-11-04T19:11:28.410168] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-11-04T19:11:28.689055] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T19:11:28.720695] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-11-04T19:11:28.967025] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-10. [2025-11-04T19:11:29.767057] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-02. [2025-11-04T19:11:30.419290] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T19:11:30.567024] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-11. [2025-11-04T19:11:31.368221] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-07. [2025-11-04T19:11:31.368128] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-11-04T19:11:31.814045] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T19:11:31.815915] [INFO ] [REMOTE ] GLOB: Full fetch complete (67213 records). [2025-11-04T19:11:31.816028] [DEBUG] [VALIDATOR ] GLOB: Validating primary data. [2025-11-04T19:11:32.174014] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-03. [2025-11-04T19:11:32.213450] [DEBUG] [VALIDATOR ] GLOB: All expected trading days are present. [2025-11-04T19:11:32.214165] [DEBUG] [VALIDATOR ] GLOB: Primary data validation passed. [2025-11-04T19:11:32.215102] [INFO ] [MAIN ] GLOB: Fetched primary data is valid. [2025-11-04T19:11:32.215286] [DEBUG] [LOCAL ] GLOB: Saving primary data. [2025-11-04T19:11:32.216683] [DEBUG] [LOCAL ] GLOB: Trimmed 67213 records to 48285 (keeping 200 days of data) [2025-11-04T19:11:32.492457] [DEBUG] [LOCAL ] GLOB: Primary data saved (48285 records). [2025-11-04T19:11:32.492579] [INFO ] [MAIN ] GLOB: Full history saved. [2025-11-04T19:11:32.492680] [INFO ] [MAIN ] Update Progress: 940/2367 symbols completed. GLOB took 44.9s. Estimated remaining: 3h 19m 36s. Rate: 7.1 symbols/min [2025-11-04T19:11:32.680768] [INFO ] [MAIN ] GME: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:11:32.730982] [DEBUG] [LOCAL ] GME: Loaded 53774 existing records from primary storage. [2025-11-04T19:11:32.731388] [INFO ] [MAIN ] GME: Loaded 53774 existing records. [2025-11-04T19:11:32.908981] [WARN ] [MAIN ] GME: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:11:32.909178] [INFO ] [MAIN ] GME: Wiping existing primary data before full refresh. [2025-11-04T19:11:32.909225] [INFO ] [LOCAL ] GME: Starting deletion of primary data. [2025-11-04T19:11:32.910043] [DEBUG] [LOCAL ] GME: Deleted primary data file. [2025-11-04T19:11:32.910099] [INFO ] [MAIN ] GME: Starting full primary refresh. [2025-11-04T19:11:32.972112] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-07. [2025-11-04T19:11:33.305482] [INFO ] [REMOTE ] GME: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:11:33.337431] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-11-04T19:11:33.560924] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-11-04T19:11:33.767054] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-08. [2025-11-04T19:11:34.078399] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-11-04T19:11:34.340174] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-11-04T19:11:34.567084] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-08. [2025-11-04T19:11:35.367027] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-04. [2025-11-04T19:11:36.167059] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-01. [2025-11-04T19:11:36.315257] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-11-04T19:11:36.967080] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-09. [2025-11-04T19:11:37.252760] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-11-04T19:11:37.592271] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:11:37.767094] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-05. [2025-11-04T19:11:38.224398] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-01 (2.06 seconds). [2025-11-04T19:11:38.567044] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-02. [2025-11-04T19:11:39.367031] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-10. [2025-11-04T19:11:39.429588] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-11-04T19:11:39.879405] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-11-04T19:11:40.167076] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-11. [2025-11-04T19:11:40.440633] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-11-04T19:11:40.720745] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T19:11:40.721907] [INFO ] [REMOTE ] GLP: Full fetch complete (14128 records). [2025-11-04T19:11:40.721983] [DEBUG] [VALIDATOR ] GLP: Validating primary data. [2025-11-04T19:11:40.972222] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-09. [2025-11-04T19:11:41.138863] [DEBUG] [VALIDATOR ] GLP: All expected trading days are present. [2025-11-04T19:11:41.138961] [DEBUG] [VALIDATOR ] GLP: Primary data validation passed. [2025-11-04T19:11:41.139532] [INFO ] [MAIN ] GLP: Fetched primary data is valid. [2025-11-04T19:11:41.139587] [DEBUG] [LOCAL ] GLP: Saving primary data. [2025-11-04T19:11:41.140188] [DEBUG] [LOCAL ] GLP: Trimmed 14128 records to 9076 (keeping 200 days of data) [2025-11-04T19:11:41.204215] [DEBUG] [LOCAL ] GLP: Primary data saved (9076 records). [2025-11-04T19:11:41.204388] [INFO ] [MAIN ] GLP: Full history saved. [2025-11-04T19:11:41.204481] [INFO ] [MAIN ] Update Progress: 941/2367 symbols completed. GLP took 32.9s. Estimated remaining: 3h 19m 29s. Rate: 7.1 symbols/min [2025-11-04T19:11:41.374014] [INFO ] [MAIN ] GMED: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:11:41.415255] [DEBUG] [LOCAL ] GMED: Loaded 48051 existing records from primary storage. [2025-11-04T19:11:41.415579] [INFO ] [MAIN ] GMED: Loaded 48051 existing records. [2025-11-04T19:11:41.595706] [WARN ] [MAIN ] GMED: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:11:41.595926] [INFO ] [MAIN ] GMED: Wiping existing primary data before full refresh. [2025-11-04T19:11:41.595975] [INFO ] [LOCAL ] GMED: Starting deletion of primary data. [2025-11-04T19:11:41.596914] [DEBUG] [LOCAL ] GMED: Deleted primary data file. [2025-11-04T19:11:41.596976] [INFO ] [MAIN ] GMED: Starting full primary refresh. [2025-11-04T19:11:41.780349] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-06. [2025-11-04T19:11:41.969464] [INFO ] [REMOTE ] GMED: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:11:42.567033] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-01. [2025-11-04T19:11:42.814373] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-11-04T19:11:43.367347] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-03. [2025-11-04T19:11:43.376941] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-11-04T19:11:43.907400] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T19:11:44.167040] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-02. [2025-11-04T19:11:44.967037] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-10. [2025-11-04T19:11:45.587659] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-11-04T19:11:45.668257] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-03 (2.30 seconds). [2025-11-04T19:11:45.767072] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-04. [2025-11-04T19:11:46.567088] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-07. [2025-11-04T19:11:46.977849] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-11-04T19:11:47.367042] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-11. [2025-11-04T19:11:47.685498] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-11 (0.32 seconds). [2025-11-04T19:11:47.687684] [INFO ] [REMOTE ] GLW: Full fetch complete (77802 records). [2025-11-04T19:11:47.687798] [DEBUG] [VALIDATOR ] GLW: Validating primary data. [2025-11-04T19:11:48.081777] [DEBUG] [VALIDATOR ] GLW: All expected trading days are present. [2025-11-04T19:11:48.081949] [DEBUG] [VALIDATOR ] GLW: Primary data validation passed. [2025-11-04T19:11:48.083351] [INFO ] [MAIN ] GLW: Fetched primary data is valid. [2025-11-04T19:11:48.083448] [DEBUG] [LOCAL ] GLW: Saving primary data. [2025-11-04T19:11:48.096135] [DEBUG] [LOCAL ] GLW: Trimmed 77802 records to 53637 (keeping 200 days of data) [2025-11-04T19:11:48.120948] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-04 (2.35 seconds). [2025-11-04T19:11:48.172308] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-03. [2025-11-04T19:11:48.495369] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-11-04T19:11:48.510380] [DEBUG] [LOCAL ] GLW: Primary data saved (53637 records). [2025-11-04T19:11:48.510495] [INFO ] [MAIN ] GLW: Full history saved. [2025-11-04T19:11:48.510574] [INFO ] [MAIN ] Update Progress: 942/2367 symbols completed. GLW took 35.2s. Estimated remaining: 3h 19m 19s. Rate: 7.1 symbols/min [2025-11-04T19:11:48.689378] [INFO ] [MAIN ] GMRE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:11:48.719841] [DEBUG] [LOCAL ] GMRE: Loaded 34870 existing records from primary storage. [2025-11-04T19:11:48.720132] [INFO ] [MAIN ] GMRE: Loaded 34870 existing records. [2025-11-04T19:11:48.904184] [WARN ] [MAIN ] GMRE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:11:48.904452] [INFO ] [MAIN ] GMRE: Wiping existing primary data before full refresh. [2025-11-04T19:11:48.904502] [INFO ] [LOCAL ] GMRE: Starting deletion of primary data. [2025-11-04T19:11:48.905144] [DEBUG] [LOCAL ] GMRE: Deleted primary data file. [2025-11-04T19:11:48.905203] [INFO ] [MAIN ] GMRE: Starting full primary refresh. [2025-11-04T19:11:48.972131] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-08. [2025-11-04T19:11:49.282154] [INFO ] [REMOTE ] GMRE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:11:49.496716] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T19:11:49.767103] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-04. [2025-11-04T19:11:50.567027] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-01. [2025-11-04T19:11:50.664672] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-11-04T19:11:50.966015] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T19:11:51.367063] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-05. [2025-11-04T19:11:51.705143] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T19:11:52.167037] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-05. [2025-11-04T19:11:52.862498] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-11-04T19:11:52.967003] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-06. [2025-11-04T19:11:53.767083] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-09. [2025-11-04T19:11:54.295437] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T19:11:54.370582] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-05 (2.20 seconds). [2025-11-04T19:11:54.567030] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-07. [2025-11-04T19:11:55.329743] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-11-04T19:11:55.367034] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-10. [2025-11-04T19:11:55.939262] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T19:11:56.167047] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-02. [2025-11-04T19:11:56.967022] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-06. [2025-11-04T19:11:57.122453] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T19:11:57.292605] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-11-04T19:11:57.767048] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-03. [2025-11-04T19:11:58.567086] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-11. [2025-11-04T19:11:58.849982] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T19:11:58.913940] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-11 (0.35 seconds). [2025-11-04T19:11:58.916620] [INFO ] [REMOTE ] GM: Full fetch complete (77907 records). [2025-11-04T19:11:58.916831] [DEBUG] [VALIDATOR ] GM: Validating primary data. [2025-11-04T19:11:59.355960] [DEBUG] [VALIDATOR ] GM: All expected trading days are present. [2025-11-04T19:11:59.356190] [DEBUG] [VALIDATOR ] GM: Primary data validation passed. [2025-11-04T19:11:59.357207] [INFO ] [MAIN ] GM: Fetched primary data is valid. [2025-11-04T19:11:59.357290] [DEBUG] [LOCAL ] GM: Saving primary data. [2025-11-04T19:11:59.359013] [DEBUG] [LOCAL ] GM: Trimmed 77907 records to 53729 (keeping 200 days of data) [2025-11-04T19:11:59.370349] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-08. [2025-11-04T19:11:59.493933] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-06 (2.53 seconds). [2025-11-04T19:11:59.734679] [DEBUG] [LOCAL ] GM: Primary data saved (53729 records). [2025-11-04T19:11:59.734830] [INFO ] [MAIN ] GM: Full history saved. [2025-11-04T19:11:59.734910] [INFO ] [MAIN ] Update Progress: 943/2367 symbols completed. GM took 34.5s. Estimated remaining: 3h 19m 14s. Rate: 7.1 symbols/min [2025-11-04T19:11:59.903855] [INFO ] [MAIN ] GNE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:11:59.920633] [DEBUG] [LOCAL ] GNE: Loaded 18286 existing records from primary storage. [2025-11-04T19:11:59.920890] [INFO ] [MAIN ] GNE: Loaded 18286 existing records. [2025-11-04T19:12:00.098196] [WARN ] [MAIN ] GNE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:12:00.098437] [INFO ] [MAIN ] GNE: Wiping existing primary data before full refresh. [2025-11-04T19:12:00.098485] [INFO ] [LOCAL ] GNE: Starting deletion of primary data. [2025-11-04T19:12:00.098893] [DEBUG] [LOCAL ] GNE: Deleted primary data file. [2025-11-04T19:12:00.098952] [INFO ] [MAIN ] GNE: Starting full primary refresh. [2025-11-04T19:12:00.177240] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-04. [2025-11-04T19:12:00.531975] [INFO ] [REMOTE ] GNE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:12:00.923343] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-11-04T19:12:00.967049] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-01. [2025-11-04T19:12:01.398187] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T19:12:01.556389] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-11-04T19:12:01.767043] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-02. [2025-11-04T19:12:02.567047] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-09. [2025-11-04T19:12:02.596631] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T19:12:03.367069] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-05. [2025-11-04T19:12:03.838106] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-11-04T19:12:04.167057] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-10. [2025-11-04T19:12:04.529278] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T19:12:04.967050] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-07. [2025-11-04T19:12:05.490553] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T19:12:05.767042] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-11. [2025-11-04T19:12:06.567096] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-06. [2025-11-04T19:12:06.989362] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T19:12:06.991251] [INFO ] [REMOTE ] GMED: Full fetch complete (70013 records). [2025-11-04T19:12:06.991357] [DEBUG] [VALIDATOR ] GMED: Validating primary data. [2025-11-04T19:12:07.397730] [DEBUG] [VALIDATOR ] GMED: All expected trading days are present. [2025-11-04T19:12:07.380784] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-03. [2025-11-04T19:12:07.397895] [DEBUG] [VALIDATOR ] GMED: Primary data validation passed. [2025-11-04T19:12:07.399487] [INFO ] [MAIN ] GMED: Fetched primary data is valid. [2025-11-04T19:12:07.399821] [DEBUG] [LOCAL ] GMED: Saving primary data. [2025-11-04T19:12:07.432920] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-07 (2.47 seconds). [2025-11-04T19:12:07.431447] [DEBUG] [LOCAL ] GMED: Trimmed 70013 records to 48104 (keeping 200 days of data) [2025-11-04T19:12:07.727419] [DEBUG] [LOCAL ] GMED: Primary data saved (48104 records). [2025-11-04T19:12:07.727586] [INFO ] [MAIN ] GMED: Full history saved. [2025-11-04T19:12:07.727849] [INFO ] [MAIN ] Update Progress: 944/2367 symbols completed. GMED took 26.5s. Estimated remaining: 3h 19m 5s. Rate: 7.1 symbols/min [2025-11-04T19:12:07.865672] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T19:12:07.936406] [INFO ] [MAIN ] GNK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:12:07.944952] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-11-04T19:12:07.980791] [DEBUG] [LOCAL ] GNK: Loaded 38519 existing records from primary storage. [2025-11-04T19:12:07.981133] [INFO ] [MAIN ] GNK: Loaded 38519 existing records. [2025-11-04T19:12:08.164135] [WARN ] [MAIN ] GNK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:12:08.164353] [INFO ] [MAIN ] GNK: Wiping existing primary data before full refresh. [2025-11-04T19:12:08.164402] [INFO ] [LOCAL ] GNK: Starting deletion of primary data. [2025-11-04T19:12:08.165061] [DEBUG] [LOCAL ] GNK: Deleted primary data file. [2025-11-04T19:12:08.165118] [INFO ] [MAIN ] GNK: Starting full primary refresh. [2025-11-04T19:12:08.175938] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-07. [2025-11-04T19:12:08.514277] [INFO ] [REMOTE ] GNK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:12:08.967086] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-01. [2025-11-04T19:12:09.290631] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T19:12:09.767015] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-04. [2025-11-04T19:12:09.825281] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T19:12:10.283699] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-11-04T19:12:10.567106] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-08. [2025-11-04T19:12:11.367027] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-05. [2025-11-04T19:12:11.762341] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T19:12:12.004913] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T19:12:12.167035] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-06. [2025-11-04T19:12:12.882856] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-11-04T19:12:12.967021] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-07. [2025-11-04T19:12:13.708232] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T19:12:13.767045] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-08. [2025-11-04T19:12:14.481544] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T19:12:14.567032] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-02. [2025-11-04T19:12:15.367031] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-09. [2025-11-04T19:12:15.597024] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T19:12:15.997622] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:12:16.167044] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-08. [2025-11-04T19:12:16.967037] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-09. [2025-11-04T19:12:17.767015] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-10. [2025-11-04T19:12:17.951330] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T19:12:17.993637] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-11-04T19:12:18.294009] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-11-04T19:12:18.567043] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-03. [2025-11-04T19:12:19.367091] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-11. [2025-11-04T19:12:19.797757] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T19:12:19.988774] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T19:12:19.990196] [INFO ] [REMOTE ] GNE: Full fetch complete (22789 records). [2025-11-04T19:12:19.990278] [DEBUG] [VALIDATOR ] GNE: Validating primary data. [2025-11-04T19:12:20.167106] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-09. [2025-11-04T19:12:20.367765] [DEBUG] [VALIDATOR ] GNE: All expected trading days are present. [2025-11-04T19:12:20.367848] [DEBUG] [VALIDATOR ] GNE: Primary data validation passed. [2025-11-04T19:12:20.368306] [INFO ] [MAIN ] GNE: Fetched primary data is valid. [2025-11-04T19:12:20.368344] [DEBUG] [LOCAL ] GNE: Saving primary data. [2025-11-04T19:12:20.368982] [DEBUG] [LOCAL ] GNE: Trimmed 22789 records to 18328 (keeping 200 days of data) [2025-11-04T19:12:20.477955] [DEBUG] [LOCAL ] GNE: Primary data saved (18328 records). [2025-11-04T19:12:20.478092] [INFO ] [MAIN ] GNE: Full history saved. [2025-11-04T19:12:20.478167] [INFO ] [MAIN ] Update Progress: 945/2367 symbols completed. GNE took 20.7s. Estimated remaining: 3h 19m 4s. Rate: 7.1 symbols/min [2025-11-04T19:12:20.658704] [INFO ] [MAIN ] GNL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:12:20.697733] [DEBUG] [LOCAL ] GNL: Loaded 45955 existing records from primary storage. [2025-11-04T19:12:20.698026] [INFO ] [MAIN ] GNL: Loaded 45955 existing records. [2025-11-04T19:12:20.879320] [WARN ] [MAIN ] GNL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:12:20.879533] [INFO ] [MAIN ] GNL: Wiping existing primary data before full refresh. [2025-11-04T19:12:20.879601] [INFO ] [LOCAL ] GNL: Starting deletion of primary data. [2025-11-04T19:12:20.879828] [DEBUG] [LOCAL ] GNL: Deleted primary data file. [2025-11-04T19:12:20.879890] [INFO ] [MAIN ] GNL: Starting full primary refresh. [2025-11-04T19:12:20.976061] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-04. [2025-11-04T19:12:21.249088] [INFO ] [REMOTE ] GNL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:12:21.767130] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-10. [2025-11-04T19:12:22.174026] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-11-04T19:12:22.221529] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T19:12:22.562067] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T19:12:22.567361] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-01. [2025-11-04T19:12:23.367023] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-11. [2025-11-04T19:12:23.818304] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T19:12:24.075838] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T19:12:24.077106] [INFO ] [REMOTE ] GMRE: Full fetch complete (50450 records). [2025-11-04T19:12:24.077181] [DEBUG] [VALIDATOR ] GMRE: Validating primary data. [2025-11-04T19:12:24.175176] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-10. [2025-11-04T19:12:24.477587] [DEBUG] [VALIDATOR ] GMRE: All expected trading days are present. [2025-11-04T19:12:24.477702] [DEBUG] [VALIDATOR ] GMRE: Primary data validation passed. [2025-11-04T19:12:24.478392] [INFO ] [MAIN ] GMRE: Fetched primary data is valid. [2025-11-04T19:12:24.478433] [DEBUG] [LOCAL ] GMRE: Saving primary data. [2025-11-04T19:12:24.479558] [DEBUG] [LOCAL ] GMRE: Trimmed 50450 records to 34724 (keeping 200 days of data) [2025-11-04T19:12:24.684743] [DEBUG] [LOCAL ] GMRE: Primary data saved (34724 records). [2025-11-04T19:12:24.684871] [INFO ] [MAIN ] GMRE: Full history saved. [2025-11-04T19:12:24.684945] [INFO ] [MAIN ] Update Progress: 946/2367 symbols completed. GMRE took 36.2s. Estimated remaining: 3h 18m 49s. Rate: 7.1 symbols/min [2025-11-04T19:12:24.868012] [INFO ] [MAIN ] GNRC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:12:24.910287] [DEBUG] [LOCAL ] GNRC: Loaded 45302 existing records from primary storage. [2025-11-04T19:12:24.910561] [INFO ] [MAIN ] GNRC: Loaded 45302 existing records. [2025-11-04T19:12:24.976240] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-02. [2025-11-04T19:12:25.090196] [WARN ] [MAIN ] GNRC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:12:25.090381] [INFO ] [MAIN ] GNRC: Wiping existing primary data before full refresh. [2025-11-04T19:12:25.090427] [INFO ] [LOCAL ] GNRC: Starting deletion of primary data. [2025-11-04T19:12:25.090715] [DEBUG] [LOCAL ] GNRC: Deleted primary data file. [2025-11-04T19:12:25.090773] [INFO ] [MAIN ] GNRC: Starting full primary refresh. [2025-11-04T19:12:25.442893] [INFO ] [REMOTE ] GNRC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:12:25.767071] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-05. [2025-11-04T19:12:26.066447] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T19:12:26.144235] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-11-04T19:12:26.567067] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-03. [2025-11-04T19:12:26.856533] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-11-04T19:12:27.367063] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-01. [2025-11-04T19:12:27.869472] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T19:12:28.167058] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-11. [2025-11-04T19:12:28.544796] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T19:12:28.967040] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-04. [2025-11-04T19:12:29.767098] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-06. [2025-11-04T19:12:30.319347] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-11 (2.15 seconds). [2025-11-04T19:12:30.359578] [INFO ] [REMOTE ] GME: Full fetch complete (77917 records). [2025-11-04T19:12:30.359756] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T19:12:30.360251] [DEBUG] [VALIDATOR ] GME: Validating primary data. [2025-11-04T19:12:30.572204] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-05. [2025-11-04T19:12:30.928429] [DEBUG] [VALIDATOR ] GME: All expected trading days are present. [2025-11-04T19:12:30.928681] [DEBUG] [VALIDATOR ] GME: Primary data validation passed. [2025-11-04T19:12:30.929897] [INFO ] [MAIN ] GME: Fetched primary data is valid. [2025-11-04T19:12:30.929988] [DEBUG] [LOCAL ] GME: Saving primary data. [2025-11-04T19:12:30.941426] [DEBUG] [LOCAL ] GME: Trimmed 77917 records to 53774 (keeping 200 days of data) [2025-11-04T19:12:30.965819] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T19:12:31.298151] [DEBUG] [LOCAL ] GME: Primary data saved (53774 records). [2025-11-04T19:12:31.298283] [INFO ] [MAIN ] GME: Full history saved. [2025-11-04T19:12:31.298373] [INFO ] [MAIN ] Update Progress: 947/2367 symbols completed. GME took 58.8s. Estimated remaining: 3h 18m 38s. Rate: 7.1 symbols/min [2025-11-04T19:12:31.372173] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-07. [2025-11-04T19:12:31.481142] [INFO ] [MAIN ] GNT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:12:31.489832] [DEBUG] [LOCAL ] GNT: Loaded 8509 existing records from primary storage. [2025-11-04T19:12:31.490000] [INFO ] [MAIN ] GNT: Loaded 8509 existing records. [2025-11-04T19:12:31.681183] [WARN ] [MAIN ] GNT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:12:31.681683] [INFO ] [MAIN ] GNT: Wiping existing primary data before full refresh. [2025-11-04T19:12:31.681844] [INFO ] [LOCAL ] GNT: Starting deletion of primary data. [2025-11-04T19:12:31.682204] [DEBUG] [LOCAL ] GNT: Deleted primary data file. [2025-11-04T19:12:31.682273] [INFO ] [MAIN ] GNT: Starting full primary refresh. [2025-11-04T19:12:32.016822] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-11-04T19:12:32.093631] [INFO ] [REMOTE ] GNT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:12:32.167038] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-01. [2025-11-04T19:12:32.483376] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-01 (0.32 seconds). [2025-11-04T19:12:32.527384] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-11-04T19:12:32.967042] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-02. [2025-11-04T19:12:33.485385] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-11-04T19:12:33.767026] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-03. [2025-11-04T19:12:34.245031] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-11-04T19:12:34.567048] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-04. [2025-11-04T19:12:35.085069] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-11-04T19:12:35.367031] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-02. [2025-11-04T19:12:36.167055] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-06. [2025-11-04T19:12:36.462360] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T19:12:36.967109] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-08. [2025-11-04T19:12:37.440604] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T19:12:37.767111] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-05. [2025-11-04T19:12:38.054424] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-11-04T19:12:38.177529] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-11-04T19:12:38.567051] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-09. [2025-11-04T19:12:39.367053] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-03. [2025-11-04T19:12:39.672226] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T19:12:40.167043] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-10. [2025-11-04T19:12:40.685553] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T19:12:40.967081] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-04. [2025-11-04T19:12:41.203284] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-11-04T19:12:41.767090] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-07. [2025-11-04T19:12:42.295445] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T19:12:42.567038] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-05. [2025-11-04T19:12:43.016045] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T19:12:43.367031] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-08. [2025-11-04T19:12:43.797297] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T19:12:44.167037] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-06. [2025-11-04T19:12:44.534566] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T19:12:44.967046] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-11. [2025-11-04T19:12:45.369512] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T19:12:45.767076] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-07. [2025-11-04T19:12:45.910225] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T19:12:45.912099] [INFO ] [REMOTE ] GNK: Full fetch complete (57031 records). [2025-11-04T19:12:45.912237] [DEBUG] [VALIDATOR ] GNK: Validating primary data. [2025-11-04T19:12:46.315567] [DEBUG] [VALIDATOR ] GNK: All expected trading days are present. [2025-11-04T19:12:46.315689] [DEBUG] [VALIDATOR ] GNK: Primary data validation passed. [2025-11-04T19:12:46.316504] [INFO ] [MAIN ] GNK: Fetched primary data is valid. [2025-11-04T19:12:46.316548] [DEBUG] [LOCAL ] GNK: Saving primary data. [2025-11-04T19:12:46.317839] [DEBUG] [LOCAL ] GNK: Trimmed 57031 records to 38520 (keeping 200 days of data) [2025-11-04T19:12:46.544860] [DEBUG] [LOCAL ] GNK: Primary data saved (38520 records). [2025-11-04T19:12:46.544992] [INFO ] [MAIN ] GNK: Full history saved. [2025-11-04T19:12:46.545070] [INFO ] [MAIN ] Update Progress: 948/2367 symbols completed. GNK took 38.8s. Estimated remaining: 3h 18m 40s. Rate: 7.1 symbols/min [2025-11-04T19:12:46.575426] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-06. [2025-11-04T19:12:46.713437] [INFO ] [MAIN ] GNW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:12:46.756907] [DEBUG] [LOCAL ] GNW: Loaded 51240 existing records from primary storage. [2025-11-04T19:12:46.757201] [INFO ] [MAIN ] GNW: Loaded 51240 existing records. [2025-11-04T19:12:46.935493] [WARN ] [MAIN ] GNW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:12:46.935815] [INFO ] [MAIN ] GNW: Wiping existing primary data before full refresh. [2025-11-04T19:12:46.935865] [INFO ] [LOCAL ] GNW: Starting deletion of primary data. [2025-11-04T19:12:46.936595] [DEBUG] [LOCAL ] GNW: Deleted primary data file. [2025-11-04T19:12:46.936680] [INFO ] [MAIN ] GNW: Starting full primary refresh. [2025-11-04T19:12:47.228519] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-11-04T19:12:47.312443] [INFO ] [REMOTE ] GNW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:12:47.340355] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-11-04T19:12:47.367042] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-09. [2025-11-04T19:12:48.167038] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-07. [2025-11-04T19:12:48.591784] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-11-04T19:12:48.623409] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T19:12:48.967101] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-01. [2025-11-04T19:12:49.767049] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-10. [2025-11-04T19:12:50.138113] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T19:12:50.567088] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-02. [2025-11-04T19:12:50.991503] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T19:12:51.367035] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-08. [2025-11-04T19:12:51.719838] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T19:12:51.810531] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T19:12:52.167040] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-09. [2025-11-04T19:12:52.584720] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-11-04T19:12:52.967086] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-11. [2025-11-04T19:12:53.767064] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-10. [2025-11-04T19:12:54.152554] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-11 (1.19 seconds). [2025-11-04T19:12:54.154374] [INFO ] [REMOTE ] GNL: Full fetch complete (67817 records). [2025-11-04T19:12:54.154453] [DEBUG] [VALIDATOR ] GNL: Validating primary data. [2025-11-04T19:12:54.502144] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-11-04T19:12:54.552534] [DEBUG] [VALIDATOR ] GNL: All expected trading days are present. [2025-11-04T19:12:54.552612] [DEBUG] [VALIDATOR ] GNL: Primary data validation passed. [2025-11-04T19:12:54.553513] [INFO ] [MAIN ] GNL: Fetched primary data is valid. [2025-11-04T19:12:54.553573] [DEBUG] [LOCAL ] GNL: Saving primary data. [2025-11-04T19:12:54.554845] [DEBUG] [LOCAL ] GNL: Trimmed 67817 records to 45891 (keeping 200 days of data) [2025-11-04T19:12:54.601577] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-03. [2025-11-04T19:12:54.810463] [DEBUG] [LOCAL ] GNL: Primary data saved (45891 records). [2025-11-04T19:12:54.810590] [INFO ] [MAIN ] GNL: Full history saved. [2025-11-04T19:12:54.810687] [INFO ] [MAIN ] Update Progress: 949/2367 symbols completed. GNL took 34.3s. Estimated remaining: 3h 18m 31s. Rate: 7.1 symbols/min [2025-11-04T19:12:54.978479] [INFO ] [MAIN ] GOF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:12:55.018495] [DEBUG] [LOCAL ] GOF: Loaded 42832 existing records from primary storage. [2025-11-04T19:12:55.018838] [INFO ] [MAIN ] GOF: Loaded 42832 existing records. [2025-11-04T19:12:55.194401] [WARN ] [MAIN ] GOF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:12:55.194699] [INFO ] [MAIN ] GOF: Wiping existing primary data before full refresh. [2025-11-04T19:12:55.194756] [INFO ] [LOCAL ] GOF: Starting deletion of primary data. [2025-11-04T19:12:55.195502] [DEBUG] [LOCAL ] GOF: Deleted primary data file. [2025-11-04T19:12:55.195560] [INFO ] [MAIN ] GOF: Starting full primary refresh. [2025-11-04T19:12:55.369129] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-08. [2025-11-04T19:12:55.577026] [INFO ] [REMOTE ] GOF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:12:55.942614] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T19:12:56.167034] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-04. [2025-11-04T19:12:56.709624] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T19:12:56.967045] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-09. [2025-11-04T19:12:57.344774] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T19:12:57.767072] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-01. [2025-11-04T19:12:58.329930] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T19:12:58.567069] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-10. [2025-11-04T19:12:58.999864] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T19:12:59.367013] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-11. [2025-11-04T19:12:59.836147] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T19:12:59.895686] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-11 (0.53 seconds). [2025-11-04T19:12:59.896522] [INFO ] [REMOTE ] GNT: Full fetch complete (11606 records). [2025-11-04T19:12:59.896589] [DEBUG] [VALIDATOR ] GNT: Validating primary data. [2025-11-04T19:13:00.170712] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-02. [2025-11-04T19:13:00.249310] [DEBUG] [VALIDATOR ] GNT: All expected trading days are present. [2025-11-04T19:13:00.249382] [DEBUG] [VALIDATOR ] GNT: Primary data validation passed. [2025-11-04T19:13:00.249673] [INFO ] [MAIN ] GNT: Fetched primary data is valid. [2025-11-04T19:13:00.249733] [DEBUG] [LOCAL ] GNT: Saving primary data. [2025-11-04T19:13:00.250175] [DEBUG] [LOCAL ] GNT: Trimmed 11606 records to 8548 (keeping 200 days of data) [2025-11-04T19:13:00.298667] [DEBUG] [LOCAL ] GNT: Primary data saved (8548 records). [2025-11-04T19:13:00.298799] [INFO ] [MAIN ] GNT: Full history saved. [2025-11-04T19:13:00.298877] [INFO ] [MAIN ] Update Progress: 950/2367 symbols completed. GNT took 29.0s. Estimated remaining: 3h 18m 18s. Rate: 7.1 symbols/min [2025-11-04T19:13:00.482779] [INFO ] [MAIN ] GOLF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:13:00.509832] [DEBUG] [LOCAL ] GOLF: Loaded 29269 existing records from primary storage. [2025-11-04T19:13:00.510127] [INFO ] [MAIN ] GOLF: Loaded 29269 existing records. [2025-11-04T19:13:00.692401] [WARN ] [MAIN ] GOLF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:13:00.692632] [INFO ] [MAIN ] GOLF: Wiping existing primary data before full refresh. [2025-11-04T19:13:00.692714] [INFO ] [LOCAL ] GOLF: Starting deletion of primary data. [2025-11-04T19:13:00.693245] [DEBUG] [LOCAL ] GOLF: Deleted primary data file. [2025-11-04T19:13:00.693302] [INFO ] [MAIN ] GOLF: Starting full primary refresh. [2025-11-04T19:13:00.972200] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-05. [2025-11-04T19:13:01.041567] [INFO ] [REMOTE ] GOLF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:13:01.491563] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-11-04T19:13:01.767106] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-01. [2025-11-04T19:13:02.423109] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T19:13:02.567157] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-06. [2025-11-04T19:13:02.639264] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T19:13:03.367074] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-02. [2025-11-04T19:13:03.823295] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-11-04T19:13:04.167070] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-07. [2025-11-04T19:13:04.237080] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T19:13:04.967047] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-11. [2025-11-04T19:13:05.506345] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-11-04T19:13:05.767086] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-03. [2025-11-04T19:13:06.435135] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-11 (1.47 seconds). [2025-11-04T19:13:06.437072] [INFO ] [REMOTE ] GNRC: Full fetch complete (66012 records). [2025-11-04T19:13:06.437154] [DEBUG] [VALIDATOR ] GNRC: Validating primary data. [2025-11-04T19:13:06.581174] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-03. [2025-11-04T19:13:06.828384] [DEBUG] [VALIDATOR ] GNRC: All expected trading days are present. [2025-11-04T19:13:06.828491] [DEBUG] [VALIDATOR ] GNRC: Primary data validation passed. [2025-11-04T19:13:06.829552] [INFO ] [MAIN ] GNRC: Fetched primary data is valid. [2025-11-04T19:13:06.829602] [DEBUG] [LOCAL ] GNRC: Saving primary data. [2025-11-04T19:13:06.830933] [DEBUG] [LOCAL ] GNRC: Trimmed 66012 records to 45328 (keeping 200 days of data) [2025-11-04T19:13:07.098479] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T19:13:07.145536] [DEBUG] [LOCAL ] GNRC: Primary data saved (45328 records). [2025-11-04T19:13:07.145670] [INFO ] [MAIN ] GNRC: Full history saved. [2025-11-04T19:13:07.145762] [INFO ] [MAIN ] Update Progress: 951/2367 symbols completed. GNRC took 42.5s. Estimated remaining: 3h 18m 8s. Rate: 7.1 symbols/min [2025-11-04T19:13:07.312185] [INFO ] [MAIN ] GOOS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:13:07.349310] [DEBUG] [LOCAL ] GOOS: Loaded 43452 existing records from primary storage. [2025-11-04T19:13:07.349588] [INFO ] [MAIN ] GOOS: Loaded 43452 existing records. [2025-11-04T19:13:07.372310] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-04. [2025-11-04T19:13:07.526416] [WARN ] [MAIN ] GOOS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:13:07.526636] [INFO ] [MAIN ] GOOS: Wiping existing primary data before full refresh. [2025-11-04T19:13:07.526733] [INFO ] [LOCAL ] GOOS: Starting deletion of primary data. [2025-11-04T19:13:07.527362] [DEBUG] [LOCAL ] GOOS: Deleted primary data file. [2025-11-04T19:13:07.527418] [INFO ] [MAIN ] GOOS: Starting full primary refresh. [2025-11-04T19:13:07.881439] [INFO ] [REMOTE ] GOOS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:13:07.901885] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T19:13:08.167052] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-04. [2025-11-04T19:13:08.884593] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T19:13:08.967001] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-05. [2025-11-04T19:13:09.295888] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T19:13:09.767046] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-08. [2025-11-04T19:13:10.090242] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-11-04T19:13:10.567103] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-01. [2025-11-04T19:13:11.038498] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T19:13:11.367040] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-05. [2025-11-04T19:13:11.531579] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T19:13:12.167056] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-02. [2025-11-04T19:13:12.349496] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-11-04T19:13:12.967025] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-06. [2025-11-04T19:13:13.402457] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T19:13:13.767076] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-06. [2025-11-04T19:13:14.131953] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T19:13:14.466722] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T19:13:14.567023] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-07. [2025-11-04T19:13:15.367077] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-09. [2025-11-04T19:13:15.733145] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T19:13:16.167070] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-07. [2025-11-04T19:13:16.534576] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T19:13:16.967027] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-03. [2025-11-04T19:13:17.079157] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-11-04T19:13:17.767030] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-08. [2025-11-04T19:13:18.221490] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-11-04T19:13:18.550593] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-11-04T19:13:18.566994] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-09. [2025-11-04T19:13:19.250273] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T19:13:19.367041] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-10. [2025-11-04T19:13:20.167043] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-08. [2025-11-04T19:13:20.948374] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-11-04T19:13:20.967006] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-11. [2025-11-04T19:13:21.575124] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-11-04T19:13:21.767081] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-10. [2025-11-04T19:13:22.183791] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T19:13:22.186609] [INFO ] [REMOTE ] GNW: Full fetch complete (74013 records). [2025-11-04T19:13:22.186866] [DEBUG] [VALIDATOR ] GNW: Validating primary data. [2025-11-04T19:13:22.567789] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-04. [2025-11-04T19:13:22.778083] [DEBUG] [VALIDATOR ] GNW: All expected trading days are present. [2025-11-04T19:13:22.778233] [DEBUG] [VALIDATOR ] GNW: Primary data validation passed. [2025-11-04T19:13:22.779340] [INFO ] [MAIN ] GNW: Fetched primary data is valid. [2025-11-04T19:13:22.779428] [DEBUG] [LOCAL ] GNW: Saving primary data. [2025-11-04T19:13:22.801462] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-11-04T19:13:22.800171] [DEBUG] [LOCAL ] GNW: Trimmed 74013 records to 51240 (keeping 200 days of data) [2025-11-04T19:13:23.105775] [DEBUG] [LOCAL ] GNW: Primary data saved (51240 records). [2025-11-04T19:13:23.105909] [INFO ] [MAIN ] GNW: Full history saved. [2025-11-04T19:13:23.106000] [INFO ] [MAIN ] Update Progress: 952/2367 symbols completed. GNW took 36.6s. Estimated remaining: 3h 18m 10s. Rate: 7.1 symbols/min [2025-11-04T19:13:23.276166] [INFO ] [MAIN ] GOTU: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:13:23.306994] [DEBUG] [LOCAL ] GOTU: Loaded 36484 existing records from primary storage. [2025-11-04T19:13:23.307264] [INFO ] [MAIN ] GOTU: Loaded 36484 existing records. [2025-11-04T19:13:23.372308] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-09. [2025-11-04T19:13:23.489208] [WARN ] [MAIN ] GOTU: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:13:23.489437] [INFO ] [MAIN ] GOTU: Wiping existing primary data before full refresh. [2025-11-04T19:13:23.489487] [INFO ] [LOCAL ] GOTU: Starting deletion of primary data. [2025-11-04T19:13:23.490085] [DEBUG] [LOCAL ] GOTU: Deleted primary data file. [2025-11-04T19:13:23.490140] [INFO ] [MAIN ] GOTU: Starting full primary refresh. [2025-11-04T19:13:23.851688] [INFO ] [REMOTE ] GOTU: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:13:23.879860] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T19:13:24.167100] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-01. [2025-11-04T19:13:24.671512] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T19:13:24.967106] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-05. [2025-11-04T19:13:25.419397] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T19:13:25.767047] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-11. [2025-11-04T19:13:26.003858] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-11-04T19:13:26.550690] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T19:13:26.551934] [INFO ] [REMOTE ] GOLF: Full fetch complete (45294 records). [2025-11-04T19:13:26.552006] [DEBUG] [VALIDATOR ] GOLF: Validating primary data. [2025-11-04T19:13:26.572779] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-10. [2025-11-04T19:13:26.931847] [DEBUG] [VALIDATOR ] GOLF: All expected trading days are present. [2025-11-04T19:13:26.931920] [DEBUG] [VALIDATOR ] GOLF: Primary data validation passed. [2025-11-04T19:13:26.932570] [INFO ] [MAIN ] GOLF: Fetched primary data is valid. [2025-11-04T19:13:26.932610] [DEBUG] [LOCAL ] GOLF: Saving primary data. [2025-11-04T19:13:26.933496] [DEBUG] [LOCAL ] GOLF: Trimmed 45294 records to 29160 (keeping 200 days of data) [2025-11-04T19:13:27.107089] [DEBUG] [LOCAL ] GOLF: Primary data saved (29160 records). [2025-11-04T19:13:27.107226] [INFO ] [MAIN ] GOLF: Full history saved. [2025-11-04T19:13:27.107348] [INFO ] [MAIN ] Update Progress: 953/2367 symbols completed. GOLF took 26.8s. Estimated remaining: 3h 17m 56s. Rate: 7.1 symbols/min [2025-11-04T19:13:27.273547] [INFO ] [MAIN ] GPC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:13:27.313204] [DEBUG] [LOCAL ] GPC: Loaded 45698 existing records from primary storage. [2025-11-04T19:13:27.313481] [INFO ] [MAIN ] GPC: Loaded 45698 existing records. [2025-11-04T19:13:27.376109] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-02. [2025-11-04T19:13:27.491797] [WARN ] [MAIN ] GPC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:13:27.491999] [INFO ] [MAIN ] GPC: Wiping existing primary data before full refresh. [2025-11-04T19:13:27.492084] [INFO ] [LOCAL ] GPC: Starting deletion of primary data. [2025-11-04T19:13:27.492956] [DEBUG] [LOCAL ] GPC: Deleted primary data file. [2025-11-04T19:13:27.493014] [INFO ] [MAIN ] GPC: Starting full primary refresh. [2025-11-04T19:13:27.856412] [INFO ] [REMOTE ] GPC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:13:28.135977] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-11-04T19:13:28.167001] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-01. [2025-11-04T19:13:28.538715] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T19:13:28.967023] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-11. [2025-11-04T19:13:29.528375] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T19:13:29.767026] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-06. [2025-11-04T19:13:30.522556] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-11 (1.56 seconds). [2025-11-04T19:13:30.524436] [INFO ] [REMOTE ] GOF: Full fetch complete (63143 records). [2025-11-04T19:13:30.524528] [DEBUG] [VALIDATOR ] GOF: Validating primary data. [2025-11-04T19:13:30.579010] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-03. [2025-11-04T19:13:30.933669] [DEBUG] [VALIDATOR ] GOF: All expected trading days are present. [2025-11-04T19:13:30.933770] [DEBUG] [VALIDATOR ] GOF: Primary data validation passed. [2025-11-04T19:13:30.934714] [INFO ] [MAIN ] GOF: Fetched primary data is valid. [2025-11-04T19:13:30.934795] [DEBUG] [LOCAL ] GOF: Saving primary data. [2025-11-04T19:13:30.959324] [DEBUG] [LOCAL ] GOF: Trimmed 63143 records to 42900 (keeping 200 days of data) [2025-11-04T19:13:30.959211] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-11-04T19:13:31.229331] [DEBUG] [LOCAL ] GOF: Primary data saved (42900 records). [2025-11-04T19:13:31.229483] [INFO ] [MAIN ] GOF: Full history saved. [2025-11-04T19:13:31.229583] [INFO ] [MAIN ] Update Progress: 954/2367 symbols completed. GOF took 36.4s. Estimated remaining: 3h 17m 41s. Rate: 7.1 symbols/min [2025-11-04T19:13:31.373556] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-07. [2025-11-04T19:13:31.405973] [INFO ] [MAIN ] GPI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:13:31.420724] [DEBUG] [LOCAL ] GPI: Loaded 14963 existing records from primary storage. [2025-11-04T19:13:31.420943] [INFO ] [MAIN ] GPI: Loaded 14963 existing records. [2025-11-04T19:13:31.601120] [WARN ] [MAIN ] GPI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:13:31.601302] [INFO ] [MAIN ] GPI: Wiping existing primary data before full refresh. [2025-11-04T19:13:31.601354] [INFO ] [LOCAL ] GPI: Starting deletion of primary data. [2025-11-04T19:13:31.601741] [DEBUG] [LOCAL ] GPI: Deleted primary data file. [2025-11-04T19:13:31.601797] [INFO ] [MAIN ] GPI: Starting full primary refresh. [2025-11-04T19:13:32.008334] [INFO ] [REMOTE ] GPI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:13:32.035424] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T19:13:32.166999] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-04. [2025-11-04T19:13:32.686055] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T19:13:32.967091] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-08. [2025-11-04T19:13:33.244951] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T19:13:33.767027] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-02. [2025-11-04T19:13:34.124221] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T19:13:34.567045] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-09. [2025-11-04T19:13:35.028233] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T19:13:35.367039] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-03. [2025-11-04T19:13:35.544749] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T19:13:36.167097] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-05. [2025-11-04T19:13:36.548156] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-11-04T19:13:36.967054] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-01. [2025-11-04T19:13:37.237101] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T19:13:37.678005] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T19:13:37.767088] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-02. [2025-11-04T19:13:38.329341] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-11-04T19:13:38.567044] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-06. [2025-11-04T19:13:39.367017] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-10. [2025-11-04T19:13:39.471136] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-11-04T19:13:40.167040] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-04. [2025-11-04T19:13:40.508438] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T19:13:40.967048] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-03. [2025-11-04T19:13:41.350512] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T19:13:41.755497] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T19:13:41.766996] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-04. [2025-11-04T19:13:42.459375] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-11-04T19:13:42.567037] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-05. [2025-11-04T19:13:43.040097] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-11-04T19:13:43.367037] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-06. [2025-11-04T19:13:43.987006] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-11-04T19:13:44.167056] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-11. [2025-11-04T19:13:44.967065] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-07. [2025-11-04T19:13:45.136886] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T19:13:45.138608] [INFO ] [REMOTE ] GOOS: Full fetch complete (64592 records). [2025-11-04T19:13:45.138752] [DEBUG] [VALIDATOR ] GOOS: Validating primary data. [2025-11-04T19:13:45.529247] [DEBUG] [VALIDATOR ] GOOS: All expected trading days are present. [2025-11-04T19:13:45.529330] [DEBUG] [VALIDATOR ] GOOS: Primary data validation passed. [2025-11-04T19:13:45.530129] [INFO ] [MAIN ] GOOS: Fetched primary data is valid. [2025-11-04T19:13:45.530192] [DEBUG] [LOCAL ] GOOS: Saving primary data. [2025-11-04T19:13:45.531395] [DEBUG] [LOCAL ] GOOS: Trimmed 64592 records to 43496 (keeping 200 days of data) [2025-11-04T19:13:45.768608] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-07. [2025-11-04T19:13:45.781339] [DEBUG] [LOCAL ] GOOS: Primary data saved (43496 records). [2025-11-04T19:13:45.781461] [INFO ] [MAIN ] GOOS: Full history saved. [2025-11-04T19:13:45.781542] [INFO ] [MAIN ] Update Progress: 955/2367 symbols completed. GOOS took 38.6s. Estimated remaining: 3h 17m 42s. Rate: 7.1 symbols/min [2025-11-04T19:13:45.964032] [INFO ] [MAIN ] GPJA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:13:45.985980] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T19:13:45.995698] [DEBUG] [LOCAL ] GPJA: Loaded 3995 existing records from primary storage. [2025-11-04T19:13:45.996174] [INFO ] [MAIN ] GPJA: Loaded 3995 existing records. [2025-11-04T19:13:46.179424] [WARN ] [MAIN ] GPJA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:13:46.179708] [INFO ] [MAIN ] GPJA: Wiping existing primary data before full refresh. [2025-11-04T19:13:46.179768] [INFO ] [LOCAL ] GPJA: Starting deletion of primary data. [2025-11-04T19:13:46.179975] [DEBUG] [LOCAL ] GPJA: Deleted primary data file. [2025-11-04T19:13:46.180030] [INFO ] [MAIN ] GPJA: Starting full primary refresh. [2025-11-04T19:13:46.555991] [INFO ] [REMOTE ] GPJA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:13:46.568179] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-11-04T19:13:46.567583] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-05. [2025-11-04T19:13:47.367073] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-08. [2025-11-04T19:13:47.836085] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-11-04T19:13:48.167065] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-08. [2025-11-04T19:13:48.467743] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-11-04T19:13:48.761058] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-11-04T19:13:48.967023] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-09. [2025-11-04T19:13:49.767033] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-06. [2025-11-04T19:13:50.091719] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T19:13:50.567043] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-09. [2025-11-04T19:13:50.889042] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T19:13:51.213706] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-11-04T19:13:51.367011] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-10. [2025-11-04T19:13:52.119344] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-11-04T19:13:52.167006] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-11. [2025-11-04T19:13:52.837354] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-11 (0.67 seconds). [2025-11-04T19:13:52.838837] [INFO ] [REMOTE ] GPI: Full fetch complete (22740 records). [2025-11-04T19:13:52.838909] [DEBUG] [VALIDATOR ] GPI: Validating primary data. [2025-11-04T19:13:52.972246] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-07. [2025-11-04T19:13:53.218439] [DEBUG] [VALIDATOR ] GPI: All expected trading days are present. [2025-11-04T19:13:53.218513] [DEBUG] [VALIDATOR ] GPI: Primary data validation passed. [2025-11-04T19:13:53.218986] [INFO ] [MAIN ] GPI: Fetched primary data is valid. [2025-11-04T19:13:53.219046] [DEBUG] [LOCAL ] GPI: Saving primary data. [2025-11-04T19:13:53.219679] [DEBUG] [LOCAL ] GPI: Trimmed 22740 records to 15064 (keeping 200 days of data) [2025-11-04T19:13:53.311719] [DEBUG] [LOCAL ] GPI: Primary data saved (15064 records). [2025-11-04T19:13:53.311848] [INFO ] [MAIN ] GPI: Full history saved. [2025-11-04T19:13:53.311926] [INFO ] [MAIN ] Update Progress: 956/2367 symbols completed. GPI took 22.1s. Estimated remaining: 3h 17m 32s. Rate: 7.1 symbols/min [2025-11-04T19:13:53.480479] [INFO ] [MAIN ] GPK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:13:53.525465] [DEBUG] [LOCAL ] GPK: Loaded 52825 existing records from primary storage. [2025-11-04T19:13:53.525786] [INFO ] [MAIN ] GPK: Loaded 52825 existing records. [2025-11-04T19:13:53.702521] [WARN ] [MAIN ] GPK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:13:53.702800] [INFO ] [MAIN ] GPK: Wiping existing primary data before full refresh. [2025-11-04T19:13:53.702859] [INFO ] [LOCAL ] GPK: Starting deletion of primary data. [2025-11-04T19:13:53.703763] [DEBUG] [LOCAL ] GPK: Deleted primary data file. [2025-11-04T19:13:53.703820] [INFO ] [MAIN ] GPK: Starting full primary refresh. [2025-11-04T19:13:53.772196] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-01. [2025-11-04T19:13:54.072955] [INFO ] [REMOTE ] GPK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:13:54.239135] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T19:13:54.377844] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-11-04T19:13:54.567043] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-01. [2025-11-04T19:13:55.367037] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-02. [2025-11-04T19:13:55.740187] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-11-04T19:13:55.899515] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T19:13:56.167051] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-03. [2025-11-04T19:13:56.515875] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-11-04T19:13:56.967017] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-08. [2025-11-04T19:13:57.767068] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-10. [2025-11-04T19:13:58.138311] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T19:13:58.567037] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-04. [2025-11-04T19:13:58.838975] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T19:13:58.955990] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-04 (0.39 seconds). [2025-11-04T19:13:59.367068] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-05. [2025-11-04T19:13:59.647017] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-11-04T19:14:00.167068] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-06. [2025-11-04T19:14:00.579192] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-11-04T19:14:00.967016] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-09. [2025-11-04T19:14:01.767101] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-11. [2025-11-04T19:14:02.132604] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T19:14:02.567095] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-07. [2025-11-04T19:14:02.572852] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T19:14:02.574801] [INFO ] [REMOTE ] GOTU: Full fetch complete (57445 records). [2025-11-04T19:14:02.574915] [DEBUG] [VALIDATOR ] GOTU: Validating primary data. [2025-11-04T19:14:02.956217] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-11-04T19:14:02.975903] [DEBUG] [VALIDATOR ] GOTU: All expected trading days are present. [2025-11-04T19:14:02.976230] [DEBUG] [VALIDATOR ] GOTU: Primary data validation passed. [2025-11-04T19:14:02.977012] [INFO ] [MAIN ] GOTU: Fetched primary data is valid. [2025-11-04T19:14:02.977072] [DEBUG] [LOCAL ] GOTU: Saving primary data. [2025-11-04T19:14:02.978056] [DEBUG] [LOCAL ] GOTU: Trimmed 57445 records to 36462 (keeping 200 days of data) [2025-11-04T19:14:03.171639] [DEBUG] [LOCAL ] GOTU: Primary data saved (36462 records). [2025-11-04T19:14:03.171794] [INFO ] [MAIN ] GOTU: Full history saved. [2025-11-04T19:14:03.171870] [INFO ] [MAIN ] Update Progress: 957/2367 symbols completed. GOTU took 40.1s. Estimated remaining: 3h 17m 26s. Rate: 7.1 symbols/min [2025-11-04T19:14:03.367055] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-08. [2025-11-04T19:14:03.733409] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-11-04T19:14:04.167046] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-02. [2025-11-04T19:14:04.967070] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-10. [2025-11-04T19:14:05.438150] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T19:14:05.767043] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-03. [2025-11-04T19:14:06.109112] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T19:14:06.567101] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-11. [2025-11-04T19:14:06.985435] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T19:14:07.367092] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-04. [2025-11-04T19:14:07.733797] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-11 (1.17 seconds). [2025-11-04T19:14:07.736186] [INFO ] [REMOTE ] GPC: Full fetch complete (68448 records). [2025-11-04T19:14:07.736366] [DEBUG] [VALIDATOR ] GPC: Validating primary data. [2025-11-04T19:14:08.137809] [DEBUG] [VALIDATOR ] GPC: All expected trading days are present. [2025-11-04T19:14:08.137886] [DEBUG] [VALIDATOR ] GPC: Primary data validation passed. [2025-11-04T19:14:08.138728] [INFO ] [MAIN ] GPC: Fetched primary data is valid. [2025-11-04T19:14:08.138790] [DEBUG] [LOCAL ] GPC: Saving primary data. [2025-11-04T19:14:08.140074] [DEBUG] [LOCAL ] GPC: Trimmed 68448 records to 45689 (keeping 200 days of data) [2025-11-04T19:14:08.194500] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-09. [2025-11-04T19:14:08.419487] [DEBUG] [LOCAL ] GPC: Primary data saved (45689 records). [2025-11-04T19:14:08.419614] [INFO ] [MAIN ] GPC: Full history saved. [2025-11-04T19:14:08.419718] [INFO ] [MAIN ] Update Progress: 958/2367 symbols completed. GPC took 41.3s. Estimated remaining: 3h 17m 13s. Rate: 7.1 symbols/min [2025-11-04T19:14:08.683985] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-11-04T19:14:08.683779] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T19:14:08.967076] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-10. [2025-11-04T19:14:09.280316] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-10 (0.31 seconds). [2025-11-04T19:14:09.767101] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-11. [2025-11-04T19:14:10.423006] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T19:14:10.423761] [INFO ] [REMOTE ] GPJA: Full fetch complete (6405 records). [2025-11-04T19:14:10.423833] [DEBUG] [VALIDATOR ] GPJA: Validating primary data. [2025-11-04T19:14:10.572128] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-05. [2025-11-04T19:14:10.790596] [DEBUG] [VALIDATOR ] GPJA: All expected trading days are present. [2025-11-04T19:14:10.790695] [DEBUG] [VALIDATOR ] GPJA: Primary data validation passed. [2025-11-04T19:14:10.790877] [INFO ] [MAIN ] GPJA: Fetched primary data is valid. [2025-11-04T19:14:10.790916] [DEBUG] [LOCAL ] GPJA: Saving primary data. [2025-11-04T19:14:10.791273] [DEBUG] [LOCAL ] GPJA: Trimmed 6405 records to 3994 (keeping 200 days of data) [2025-11-04T19:14:10.814508] [DEBUG] [LOCAL ] GPJA: Primary data saved (3994 records). [2025-11-04T19:14:10.814589] [INFO ] [MAIN ] GPJA: Full history saved. [2025-11-04T19:14:10.814681] [INFO ] [MAIN ] Update Progress: 959/2367 symbols completed. GPJA took 25.0s. Estimated remaining: 3h 16m 55s. Rate: 7.2 symbols/min [2025-11-04T19:14:11.841512] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-11-04T19:14:11.852951] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-06. [2025-11-04T19:14:13.066556] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T19:14:13.077016] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-07. [2025-11-04T19:14:14.438456] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T19:14:14.450051] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-08. [2025-11-04T19:14:15.833966] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T19:14:15.845226] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-09. [2025-11-04T19:14:17.099596] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T19:14:17.111041] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-10. [2025-11-04T19:14:18.624440] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T19:14:18.636763] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-11. [2025-11-04T19:14:20.006596] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-11 (1.37 seconds). [2025-11-04T19:14:20.008714] [INFO ] [REMOTE ] GPK: Full fetch complete (76207 records). [2025-11-04T19:14:20.008848] [DEBUG] [VALIDATOR ] GPK: Validating primary data. [2025-11-04T19:14:20.405313] [DEBUG] [VALIDATOR ] GPK: All expected trading days are present. [2025-11-04T19:14:20.405389] [DEBUG] [VALIDATOR ] GPK: Primary data validation passed. [2025-11-04T19:14:20.406253] [INFO ] [MAIN ] GPK: Fetched primary data is valid. [2025-11-04T19:14:20.406314] [DEBUG] [LOCAL ] GPK: Saving primary data. [2025-11-04T19:14:20.407755] [DEBUG] [LOCAL ] GPK: Trimmed 76207 records to 52830 (keeping 200 days of data) [2025-11-04T19:14:20.718782] [DEBUG] [LOCAL ] GPK: Primary data saved (52830 records). [2025-11-04T19:14:20.718909] [INFO ] [MAIN ] GPK: Full history saved. [2025-11-04T19:14:20.718995] [INFO ] [MAIN ] Update Progress: 960/2367 symbols completed. GPK took 27.4s. Estimated remaining: 3h 16m 49s. Rate: 7.1 symbols/min [2025-11-04T19:14:21.425545] [INFO ] [MAIN ] GPRK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:14:21.426526] [INFO ] [MAIN ] GPMT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:14:21.418344] [INFO ] [MAIN ] GPOR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:14:21.426900] [DEBUG] [LOCAL ] GPMT: No existing data file found in primary storage. [2025-11-04T19:14:21.444580] [WARN ] [MAIN ] GPMT: No existing data found, performing full refresh instead. [2025-11-04T19:14:21.448061] [INFO ] [MAIN ] GPMT: Wiping existing primary data before full refresh. [2025-11-04T19:14:21.447625] [INFO ] [MAIN ] GPN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:14:21.448369] [INFO ] [LOCAL ] GPMT: Starting deletion of primary data. [2025-11-04T19:14:21.448510] [DEBUG] [LOCAL ] GPMT: No primary data file found to delete. [2025-11-04T19:14:21.448555] [INFO ] [MAIN ] GPMT: Starting full primary refresh. [2025-11-04T19:14:21.629704] [DEBUG] [LOCAL ] GPOR: Loaded 23393 existing records from primary storage. [2025-11-04T19:14:21.672553] [INFO ] [MAIN ] GPOR: Loaded 23393 existing records. [2025-11-04T19:14:21.726845] [DEBUG] [LOCAL ] GPRK: Loaded 40207 existing records from primary storage. [2025-11-04T19:14:21.762675] [INFO ] [MAIN ] GPRK: Loaded 40207 existing records. [2025-11-04T19:14:22.033285] [DEBUG] [LOCAL ] GPN: Loaded 51748 existing records from primary storage. [2025-11-04T19:14:22.117273] [INFO ] [MAIN ] GPN: Loaded 51748 existing records. [2025-11-04T19:14:22.222902] [INFO ] [REMOTE ] GPMT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:14:22.228409] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-01. [2025-11-04T19:14:22.279186] [WARN ] [MAIN ] GPOR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:14:22.315882] [WARN ] [MAIN ] GPRK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:14:22.321182] [INFO ] [MAIN ] GPOR: Wiping existing primary data before full refresh. [2025-11-04T19:14:22.332037] [INFO ] [LOCAL ] GPOR: Starting deletion of primary data. [2025-11-04T19:14:22.355758] [INFO ] [MAIN ] GPRK: Wiping existing primary data before full refresh. [2025-11-04T19:14:22.360976] [DEBUG] [LOCAL ] GPOR: Deleted primary data file. [2025-11-04T19:14:22.361147] [INFO ] [MAIN ] GPOR: Starting full primary refresh. [2025-11-04T19:14:22.372821] [INFO ] [LOCAL ] GPRK: Starting deletion of primary data. [2025-11-04T19:14:22.412342] [DEBUG] [LOCAL ] GPRK: Deleted primary data file. [2025-11-04T19:14:22.452841] [INFO ] [MAIN ] GPRK: Starting full primary refresh. [2025-11-04T19:14:22.690092] [WARN ] [MAIN ] GPN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:14:22.731336] [INFO ] [MAIN ] GPN: Wiping existing primary data before full refresh. [2025-11-04T19:14:22.736451] [INFO ] [LOCAL ] GPN: Starting deletion of primary data. [2025-11-04T19:14:22.792556] [DEBUG] [LOCAL ] GPN: Deleted primary data file. [2025-11-04T19:14:22.822953] [INFO ] [MAIN ] GPN: Starting full primary refresh. [2025-11-04T19:14:23.401261] [INFO ] [REMOTE ] GPRK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:14:23.388674] [INFO ] [REMOTE ] GPOR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:14:23.444547] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-01. [2025-11-04T19:14:23.445743] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-01. [2025-11-04T19:14:23.497395] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T19:14:23.530229] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-02. [2025-11-04T19:14:23.617561] [INFO ] [REMOTE ] GPN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:14:23.617866] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-01. [2025-11-04T19:14:24.277382] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T19:14:24.282280] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-03. [2025-11-04T19:14:24.407015] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-11-04T19:14:24.412932] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-02. [2025-11-04T19:14:24.485467] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-11-04T19:14:24.494060] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-02. [2025-11-04T19:14:24.940054] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T19:14:24.950863] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-02. [2025-11-04T19:14:25.115619] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-11-04T19:14:25.121153] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-04. [2025-11-04T19:14:25.221764] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-11-04T19:14:25.229051] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-03. [2025-11-04T19:14:25.400323] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T19:14:25.411984] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-03. [2025-11-04T19:14:26.123806] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-11-04T19:14:26.132000] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-05. [2025-11-04T19:14:26.152512] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-11-04T19:14:26.191846] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T19:14:26.509105] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T19:14:26.566996] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-04. [2025-11-04T19:14:27.098599] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-11-04T19:14:27.367077] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-04. [2025-11-04T19:14:27.692402] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T19:14:28.176117] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-03. [2025-11-04T19:14:28.177918] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-11-04T19:14:28.967040] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-05. [2025-11-04T19:14:29.414473] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T19:14:29.767053] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-05. [2025-11-04T19:14:30.029215] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T19:14:30.567031] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-06. [2025-11-04T19:14:30.650876] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-11-04T19:14:31.367043] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-06. [2025-11-04T19:14:31.636718] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T19:14:32.167036] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-07. [2025-11-04T19:14:32.290467] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-11-04T19:14:32.962194] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-11-04T19:14:32.967001] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-06. [2025-11-04T19:14:33.725523] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-11-04T19:14:33.766998] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-07. [2025-11-04T19:14:34.485253] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-11-04T19:14:34.566996] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-08. [2025-11-04T19:14:35.367092] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-08. [2025-11-04T19:14:35.457567] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T19:14:35.985544] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-11-04T19:14:36.167083] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-07. [2025-11-04T19:14:36.967056] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-09. [2025-11-04T19:14:37.095263] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-11-04T19:14:37.607311] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-11-04T19:14:37.767024] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-08. [2025-11-04T19:14:38.567053] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-09. [2025-11-04T19:14:38.850953] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-11-04T19:14:39.367046] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-04. [2025-11-04T19:14:39.576203] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-11-04T19:14:40.167091] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-10. [2025-11-04T19:14:40.768445] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T19:14:40.845041] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-11-04T19:14:40.967065] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-09. [2025-11-04T19:14:41.767065] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-05. [2025-11-04T19:14:42.071221] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T19:14:42.567050] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-10. [2025-11-04T19:14:43.070737] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T19:14:43.367030] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-06. [2025-11-04T19:14:43.686444] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-11-04T19:14:44.167054] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-11. [2025-11-04T19:14:44.676422] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T19:14:44.776396] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T19:14:44.777591] [INFO ] [REMOTE ] GPMT: Full fetch complete (35740 records). [2025-11-04T19:14:44.777716] [DEBUG] [VALIDATOR ] GPMT: Validating primary data. [2025-11-04T19:14:44.969788] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-07. [2025-11-04T19:14:45.165035] [DEBUG] [VALIDATOR ] GPMT: All expected trading days are present. [2025-11-04T19:14:45.165122] [DEBUG] [VALIDATOR ] GPMT: Primary data validation passed. [2025-11-04T19:14:45.165715] [INFO ] [MAIN ] GPMT: Fetched primary data is valid. [2025-11-04T19:14:45.165776] [DEBUG] [LOCAL ] GPMT: Saving primary data. [2025-11-04T19:14:45.166548] [DEBUG] [LOCAL ] GPMT: Trimmed 35740 records to 24715 (keeping 200 days of data) [2025-11-04T19:14:45.307519] [DEBUG] [LOCAL ] GPMT: Primary data saved (24715 records). [2025-11-04T19:14:45.307619] [INFO ] [MAIN ] GPMT: Full history saved. [2025-11-04T19:14:45.307720] [INFO ] [MAIN ] Update Progress: 961/2367 symbols completed. GPMT took 24.6s. Estimated remaining: 3h 17m 4s. Rate: 7.1 symbols/min [2025-11-04T19:14:45.472885] [INFO ] [MAIN ] GRBK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:14:45.496448] [DEBUG] [LOCAL ] GRBK: Loaded 26198 existing records from primary storage. [2025-11-04T19:14:45.496738] [INFO ] [MAIN ] GRBK: Loaded 26198 existing records. [2025-11-04T19:14:45.672147] [WARN ] [MAIN ] GRBK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:14:45.672359] [INFO ] [MAIN ] GRBK: Wiping existing primary data before full refresh. [2025-11-04T19:14:45.672405] [INFO ] [LOCAL ] GRBK: Starting deletion of primary data. [2025-11-04T19:14:45.672863] [DEBUG] [LOCAL ] GRBK: Deleted primary data file. [2025-11-04T19:14:45.672920] [INFO ] [MAIN ] GRBK: Starting full primary refresh. [2025-11-04T19:14:45.770715] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-10. [2025-11-04T19:14:46.017568] [INFO ] [REMOTE ] GRBK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:14:46.353066] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-11-04T19:14:46.567016] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-08. [2025-11-04T19:14:46.708747] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T19:14:47.367067] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-01. [2025-11-04T19:14:48.053980] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T19:14:48.167000] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-09. [2025-11-04T19:14:48.300835] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-11-04T19:14:48.967027] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-11. [2025-11-04T19:14:49.614748] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-11-04T19:14:49.703624] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T19:14:49.704833] [INFO ] [REMOTE ] GPOR: Full fetch complete (34677 records). [2025-11-04T19:14:49.704904] [DEBUG] [VALIDATOR ] GPOR: Validating primary data. [2025-11-04T19:14:49.772235] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-02. [2025-11-04T19:14:50.066610] [DEBUG] [VALIDATOR ] GPOR: All expected trading days are present. [2025-11-04T19:14:50.066726] [DEBUG] [VALIDATOR ] GPOR: Primary data validation passed. [2025-11-04T19:14:50.067211] [INFO ] [MAIN ] GPOR: Fetched primary data is valid. [2025-11-04T19:14:50.067250] [DEBUG] [LOCAL ] GPOR: Saving primary data. [2025-11-04T19:14:50.068009] [DEBUG] [LOCAL ] GPOR: Trimmed 34677 records to 23436 (keeping 200 days of data) [2025-11-04T19:14:50.206709] [DEBUG] [LOCAL ] GPOR: Primary data saved (23436 records). [2025-11-04T19:14:50.206886] [INFO ] [MAIN ] GPOR: Full history saved. [2025-11-04T19:14:50.206969] [INFO ] [MAIN ] Update Progress: 962/2367 symbols completed. GPOR took 29.5s. Estimated remaining: 3h 16m 51s. Rate: 7.1 symbols/min [2025-11-04T19:14:50.372436] [INFO ] [MAIN ] GRC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:14:50.372560] [DEBUG] [LOCAL ] GRC: No existing data file found in primary storage. [2025-11-04T19:14:50.372613] [WARN ] [MAIN ] GRC: No existing data found, performing full refresh instead. [2025-11-04T19:14:50.372760] [INFO ] [MAIN ] GRC: Wiping existing primary data before full refresh. [2025-11-04T19:14:50.372801] [INFO ] [LOCAL ] GRC: Starting deletion of primary data. [2025-11-04T19:14:50.372853] [DEBUG] [LOCAL ] GRC: No primary data file found to delete. [2025-11-04T19:14:50.372885] [INFO ] [MAIN ] GRC: Starting full primary refresh. [2025-11-04T19:14:50.577809] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-11. [2025-11-04T19:14:50.729535] [INFO ] [REMOTE ] GRC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:14:50.745075] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T19:14:51.367016] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-01. [2025-11-04T19:14:51.604824] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T19:14:51.606601] [INFO ] [REMOTE ] GPRK: Full fetch complete (58804 records). [2025-11-04T19:14:51.606724] [DEBUG] [VALIDATOR ] GPRK: Validating primary data. [2025-11-04T19:14:51.972347] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-11-04T19:14:51.990621] [DEBUG] [VALIDATOR ] GPRK: All expected trading days are present. [2025-11-04T19:14:51.990727] [DEBUG] [VALIDATOR ] GPRK: Primary data validation passed. [2025-11-04T19:14:51.993896] [INFO ] [MAIN ] GPRK: Fetched primary data is valid. [2025-11-04T19:14:51.993970] [DEBUG] [LOCAL ] GPRK: Saving primary data. [2025-11-04T19:14:51.995035] [DEBUG] [LOCAL ] GPRK: Trimmed 58804 records to 40253 (keeping 200 days of data) [2025-11-04T19:14:52.186606] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-03. [2025-11-04T19:14:52.220748] [DEBUG] [LOCAL ] GPRK: Primary data saved (40253 records). [2025-11-04T19:14:52.220864] [INFO ] [MAIN ] GPRK: Full history saved. [2025-11-04T19:14:52.220937] [INFO ] [MAIN ] Update Progress: 963/2367 symbols completed. GPRK took 31.5s. Estimated remaining: 3h 16m 33s. Rate: 7.1 symbols/min [2025-11-04T19:14:52.386288] [INFO ] [MAIN ] GRDN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:14:52.405004] [DEBUG] [LOCAL ] GRDN: Loaded 21862 existing records from primary storage. [2025-11-04T19:14:52.405242] [INFO ] [MAIN ] GRDN: Loaded 21862 existing records. [2025-11-04T19:14:52.577803] [WARN ] [MAIN ] GRDN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:14:52.578008] [INFO ] [MAIN ] GRDN: Wiping existing primary data before full refresh. [2025-11-04T19:14:52.578062] [INFO ] [LOCAL ] GRDN: Starting deletion of primary data. [2025-11-04T19:14:52.578502] [DEBUG] [LOCAL ] GRDN: Deleted primary data file. [2025-11-04T19:14:52.578558] [INFO ] [MAIN ] GRDN: Starting full primary refresh. [2025-11-04T19:14:52.923361] [INFO ] [REMOTE ] GRDN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:14:52.966996] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-01. [2025-11-04T19:14:53.172886] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T19:14:53.545412] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-11-04T19:14:53.767019] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-10. [2025-11-04T19:14:54.567108] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-04. [2025-11-04T19:14:55.141192] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T19:14:55.367059] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-11. [2025-11-04T19:14:55.448800] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-11-04T19:14:56.167041] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-02. [2025-11-04T19:14:56.500732] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:14:56.502729] [INFO ] [REMOTE ] GPN: Full fetch complete (75004 records). [2025-11-04T19:14:56.502825] [DEBUG] [VALIDATOR ] GPN: Validating primary data. [2025-11-04T19:14:56.804822] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-11-04T19:14:56.898246] [DEBUG] [VALIDATOR ] GPN: All expected trading days are present. [2025-11-04T19:14:56.898323] [DEBUG] [VALIDATOR ] GPN: Primary data validation passed. [2025-11-04T19:14:56.899203] [INFO ] [MAIN ] GPN: Fetched primary data is valid. [2025-11-04T19:14:56.899265] [DEBUG] [LOCAL ] GPN: Saving primary data. [2025-11-04T19:14:56.900595] [DEBUG] [LOCAL ] GPN: Trimmed 75004 records to 51743 (keeping 200 days of data) [2025-11-04T19:14:56.987824] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-02. [2025-11-04T19:14:57.207060] [DEBUG] [LOCAL ] GPN: Primary data saved (51743 records). [2025-11-04T19:14:57.207190] [INFO ] [MAIN ] GPN: Full history saved. [2025-11-04T19:14:57.207267] [INFO ] [MAIN ] Update Progress: 964/2367 symbols completed. GPN took 36.5s. Estimated remaining: 3h 16m 20s. Rate: 7.1 symbols/min [2025-11-04T19:14:57.371081] [INFO ] [MAIN ] GRMN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:14:57.404750] [DEBUG] [LOCAL ] GRMN: Loaded 39508 existing records from primary storage. [2025-11-04T19:14:57.405057] [INFO ] [MAIN ] GRMN: Loaded 39508 existing records. [2025-11-04T19:14:57.582246] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-11-04T19:14:57.586438] [WARN ] [MAIN ] GRMN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:14:57.589180] [INFO ] [MAIN ] GRMN: Wiping existing primary data before full refresh. [2025-11-04T19:14:57.589235] [INFO ] [LOCAL ] GRMN: Starting deletion of primary data. [2025-11-04T19:14:57.590096] [DEBUG] [LOCAL ] GRMN: Deleted primary data file. [2025-11-04T19:14:57.590154] [INFO ] [MAIN ] GRMN: Starting full primary refresh. [2025-11-04T19:14:57.772342] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-03. [2025-11-04T19:14:57.937979] [INFO ] [REMOTE ] GRMN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:14:58.436567] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-11-04T19:14:58.567065] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-04. [2025-11-04T19:14:59.121771] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-11-04T19:14:59.367044] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-05. [2025-11-04T19:15:00.167062] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-01. [2025-11-04T19:15:00.189100] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T19:15:00.967075] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-06. [2025-11-04T19:15:01.596008] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-11-04T19:15:01.767067] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-05. [2025-11-04T19:15:01.905370] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-11-04T19:15:02.567065] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-02. [2025-11-04T19:15:02.618919] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-11-04T19:15:03.367040] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-03. [2025-11-04T19:15:03.935773] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T19:15:03.983210] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-11-04T19:15:04.167042] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-04. [2025-11-04T19:15:04.709096] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-11-04T19:15:04.967045] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-05. [2025-11-04T19:15:05.555586] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-11-04T19:15:05.767085] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-06. [2025-11-04T19:15:06.426206] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-11-04T19:15:06.567047] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-06. [2025-11-04T19:15:07.027954] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-11-04T19:15:07.367043] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-03. [2025-11-04T19:15:08.167054] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-07. [2025-11-04T19:15:08.631819] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-11-04T19:15:08.847822] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-11-04T19:15:08.967068] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-07. [2025-11-04T19:15:09.767073] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-07. [2025-11-04T19:15:10.064125] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T19:15:10.567024] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-08. [2025-11-04T19:15:10.584945] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T19:15:11.367071] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-08. [2025-11-04T19:15:11.454927] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T19:15:11.850243] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-11-04T19:15:12.167052] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-08. [2025-11-04T19:15:12.932367] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T19:15:12.967002] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-09. [2025-11-04T19:15:13.767011] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-09. [2025-11-04T19:15:13.805623] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-11-04T19:15:14.567016] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-09. [2025-11-04T19:15:14.586988] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-11-04T19:15:15.144277] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-11-04T19:15:15.367033] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-04. [2025-11-04T19:15:16.167036] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-10. [2025-11-04T19:15:16.511057] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T19:15:16.967009] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-10. [2025-11-04T19:15:17.113287] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T19:15:17.767084] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-11. [2025-11-04T19:15:17.888206] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-11-04T19:15:18.539185] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T19:15:18.540444] [INFO ] [REMOTE ] GRBK: Full fetch complete (39830 records). [2025-11-04T19:15:18.540516] [DEBUG] [VALIDATOR ] GRBK: Validating primary data. [2025-11-04T19:15:18.572241] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-05. [2025-11-04T19:15:18.922761] [DEBUG] [VALIDATOR ] GRBK: All expected trading days are present. [2025-11-04T19:15:18.922841] [DEBUG] [VALIDATOR ] GRBK: Primary data validation passed. [2025-11-04T19:15:18.923466] [INFO ] [MAIN ] GRBK: Fetched primary data is valid. [2025-11-04T19:15:18.923515] [DEBUG] [LOCAL ] GRBK: Saving primary data. [2025-11-04T19:15:18.924400] [DEBUG] [LOCAL ] GRBK: Trimmed 39830 records to 26170 (keeping 200 days of data) [2025-11-04T19:15:19.073321] [DEBUG] [LOCAL ] GRBK: Primary data saved (26170 records). [2025-11-04T19:15:19.073446] [INFO ] [MAIN ] GRBK: Full history saved. [2025-11-04T19:15:19.073522] [INFO ] [MAIN ] Update Progress: 965/2367 symbols completed. GRBK took 33.8s. Estimated remaining: 3h 16m 31s. Rate: 7.1 symbols/min [2025-11-04T19:15:19.237357] [INFO ] [MAIN ] GRND: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:15:19.279500] [DEBUG] [LOCAL ] GRND: Loaded 50160 existing records from primary storage. [2025-11-04T19:15:19.279819] [INFO ] [MAIN ] GRND: Loaded 50160 existing records. [2025-11-04T19:15:19.375472] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-11. [2025-11-04T19:15:19.455618] [WARN ] [MAIN ] GRND: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:15:19.455847] [INFO ] [MAIN ] GRND: Wiping existing primary data before full refresh. [2025-11-04T19:15:19.455894] [INFO ] [LOCAL ] GRND: Starting deletion of primary data. [2025-11-04T19:15:19.456609] [DEBUG] [LOCAL ] GRND: Deleted primary data file. [2025-11-04T19:15:19.456716] [INFO ] [MAIN ] GRND: Starting full primary refresh. [2025-11-04T19:15:19.823083] [INFO ] [REMOTE ] GRND: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:15:19.860088] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T19:15:20.137546] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-11 (0.76 seconds). [2025-11-04T19:15:20.138721] [INFO ] [REMOTE ] GRDN: Full fetch complete (27328 records). [2025-11-04T19:15:20.138793] [DEBUG] [VALIDATOR ] GRDN: Validating primary data. [2025-11-04T19:15:20.172194] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-01. [2025-11-04T19:15:20.585527] [DEBUG] [VALIDATOR ] GRDN: All expected trading days are present. [2025-11-04T19:15:20.585624] [DEBUG] [VALIDATOR ] GRDN: Primary data validation passed. [2025-11-04T19:15:20.586482] [INFO ] [MAIN ] GRDN: Fetched primary data is valid. [2025-11-04T19:15:20.586555] [DEBUG] [LOCAL ] GRDN: Saving primary data. [2025-11-04T19:15:20.587444] [DEBUG] [LOCAL ] GRDN: Trimmed 27328 records to 21855 (keeping 200 days of data) [2025-11-04T19:15:20.730027] [DEBUG] [LOCAL ] GRDN: Primary data saved (21855 records). [2025-11-04T19:15:20.730183] [INFO ] [MAIN ] GRDN: Full history saved. [2025-11-04T19:15:20.730282] [INFO ] [MAIN ] Update Progress: 966/2367 symbols completed. GRDN took 28.5s. Estimated remaining: 3h 16m 13s. Rate: 7.1 symbols/min [2025-11-04T19:15:20.940267] [INFO ] [MAIN ] GRNT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:15:20.972187] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-06. [2025-11-04T19:15:20.974581] [DEBUG] [LOCAL ] GRNT: Loaded 33321 existing records from primary storage. [2025-11-04T19:15:20.974852] [INFO ] [MAIN ] GRNT: Loaded 33321 existing records. [2025-11-04T19:15:21.221375] [WARN ] [MAIN ] GRNT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:15:21.221806] [INFO ] [MAIN ] GRNT: Wiping existing primary data before full refresh. [2025-11-04T19:15:21.221882] [INFO ] [LOCAL ] GRNT: Starting deletion of primary data. [2025-11-04T19:15:21.222258] [DEBUG] [LOCAL ] GRNT: Deleted primary data file. [2025-11-04T19:15:21.222318] [INFO ] [MAIN ] GRNT: Starting full primary refresh. [2025-11-04T19:15:21.615808] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T19:15:21.713690] [INFO ] [REMOTE ] GRNT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:15:21.767022] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-10. [2025-11-04T19:15:22.059471] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T19:15:22.367970] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-11-04T19:15:22.567041] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-11. [2025-11-04T19:15:23.186926] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T19:15:23.188042] [INFO ] [REMOTE ] GRC: Full fetch complete (13728 records). [2025-11-04T19:15:23.188115] [DEBUG] [VALIDATOR ] GRC: Validating primary data. [2025-11-04T19:15:23.376768] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-01. [2025-11-04T19:15:23.575785] [DEBUG] [VALIDATOR ] GRC: All expected trading days are present. [2025-11-04T19:15:23.575889] [DEBUG] [VALIDATOR ] GRC: Primary data validation passed. [2025-11-04T19:15:23.576216] [INFO ] [MAIN ] GRC: Fetched primary data is valid. [2025-11-04T19:15:23.576255] [DEBUG] [LOCAL ] GRC: Saving primary data. [2025-11-04T19:15:23.576801] [DEBUG] [LOCAL ] GRC: Trimmed 13728 records to 9373 (keeping 200 days of data) [2025-11-04T19:15:23.634112] [DEBUG] [LOCAL ] GRC: Primary data saved (9373 records). [2025-11-04T19:15:23.634252] [INFO ] [MAIN ] GRC: Full history saved. [2025-11-04T19:15:23.634325] [INFO ] [MAIN ] Update Progress: 967/2367 symbols completed. GRC took 33.4s. Estimated remaining: 3h 15m 56s. Rate: 7.1 symbols/min [2025-11-04T19:15:23.801266] [INFO ] [MAIN ] GROV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:15:23.810690] [DEBUG] [LOCAL ] GROV: Loaded 8835 existing records from primary storage. [2025-11-04T19:15:23.810895] [INFO ] [MAIN ] GROV: Loaded 8835 existing records. [2025-11-04T19:15:23.996802] [WARN ] [MAIN ] GROV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:15:23.997047] [INFO ] [MAIN ] GROV: Wiping existing primary data before full refresh. [2025-11-04T19:15:23.997095] [INFO ] [LOCAL ] GROV: Starting deletion of primary data. [2025-11-04T19:15:23.997392] [DEBUG] [LOCAL ] GROV: Deleted primary data file. [2025-11-04T19:15:23.997452] [INFO ] [MAIN ] GROV: Starting full primary refresh. [2025-11-04T19:15:24.172933] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-07. [2025-11-04T19:15:24.383717] [INFO ] [REMOTE ] GROV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:15:24.508393] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T19:15:24.967053] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-02. [2025-11-04T19:15:25.437020] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-11-04T19:15:25.648506] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-11-04T19:15:25.767073] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-02. [2025-11-04T19:15:26.567067] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-01. [2025-11-04T19:15:26.964565] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T19:15:27.067472] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-11-04T19:15:27.367036] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-03. [2025-11-04T19:15:28.167054] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-02. [2025-11-04T19:15:28.651031] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T19:15:28.654018] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-11-04T19:15:28.967032] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-03. [2025-11-04T19:15:29.508369] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-11-04T19:15:29.767040] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-04. [2025-11-04T19:15:30.268457] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-11-04T19:15:30.567043] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-05. [2025-11-04T19:15:31.097920] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-11-04T19:15:31.367050] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-06. [2025-11-04T19:15:31.880535] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T19:15:32.167062] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-04. [2025-11-04T19:15:32.967051] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-08. [2025-11-04T19:15:33.474544] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T19:15:33.767032] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-05. [2025-11-04T19:15:33.933284] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-11-04T19:15:34.567049] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-09. [2025-11-04T19:15:35.079295] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T19:15:35.367028] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-03. [2025-11-04T19:15:35.755483] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T19:15:36.167052] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-06. [2025-11-04T19:15:36.355307] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T19:15:36.967068] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-07. [2025-11-04T19:15:37.371893] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T19:15:37.575237] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-11-04T19:15:37.767025] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-04. [2025-11-04T19:15:38.567041] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-10. [2025-11-04T19:15:38.794771] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-11-04T19:15:39.367054] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-08. [2025-11-04T19:15:39.790241] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T19:15:39.913739] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-11-04T19:15:40.167019] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-07. [2025-11-04T19:15:40.967133] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-11. [2025-11-04T19:15:41.592728] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-11-04T19:15:41.767039] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-08. [2025-11-04T19:15:42.167420] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T19:15:42.169208] [INFO ] [REMOTE ] GRMN: Full fetch complete (59522 records). [2025-11-04T19:15:42.169306] [DEBUG] [VALIDATOR ] GRMN: Validating primary data. [2025-11-04T19:15:42.546215] [DEBUG] [VALIDATOR ] GRMN: All expected trading days are present. [2025-11-04T19:15:42.546335] [DEBUG] [VALIDATOR ] GRMN: Primary data validation passed. [2025-11-04T19:15:42.547107] [INFO ] [MAIN ] GRMN: Fetched primary data is valid. [2025-11-04T19:15:42.547169] [DEBUG] [LOCAL ] GRMN: Saving primary data. [2025-11-04T19:15:42.548344] [DEBUG] [LOCAL ] GRMN: Trimmed 59522 records to 39506 (keeping 200 days of data) [2025-11-04T19:15:42.601940] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-05. [2025-11-04T19:15:42.795387] [DEBUG] [LOCAL ] GRMN: Primary data saved (39506 records). [2025-11-04T19:15:42.795540] [INFO ] [MAIN ] GRMN: Full history saved. [2025-11-04T19:15:42.795624] [INFO ] [MAIN ] Update Progress: 968/2367 symbols completed. GRMN took 45.6s. Estimated remaining: 3h 16m 4s. Rate: 7.1 symbols/min [2025-11-04T19:15:42.961330] [INFO ] [MAIN ] GRX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:15:42.968022] [DEBUG] [LOCAL ] GRX: Loaded 4809 existing records from primary storage. [2025-11-04T19:15:42.968212] [INFO ] [MAIN ] GRX: Loaded 4809 existing records. [2025-11-04T19:15:43.154383] [WARN ] [MAIN ] GRX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:15:43.154896] [INFO ] [MAIN ] GRX: Wiping existing primary data before full refresh. [2025-11-04T19:15:43.154962] [INFO ] [LOCAL ] GRX: Starting deletion of primary data. [2025-11-04T19:15:43.155266] [DEBUG] [LOCAL ] GRX: Deleted primary data file. [2025-11-04T19:15:43.155378] [INFO ] [MAIN ] GRX: Starting full primary refresh. [2025-11-04T19:15:43.377484] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-09. [2025-11-04T19:15:43.534722] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-11-04T19:15:43.565363] [INFO ] [REMOTE ] GRX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:15:43.593435] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T19:15:43.909359] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-11-04T19:15:44.167048] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-10. [2025-11-04T19:15:44.621660] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-11-04T19:15:44.967089] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-11. [2025-11-04T19:15:45.414727] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-11 (0.45 seconds). [2025-11-04T19:15:45.415521] [INFO ] [REMOTE ] GROV: Full fetch complete (13360 records). [2025-11-04T19:15:45.415588] [DEBUG] [VALIDATOR ] GROV: Validating primary data. [2025-11-04T19:15:45.786834] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-09. [2025-11-04T19:15:45.797369] [DEBUG] [VALIDATOR ] GROV: All expected trading days are present. [2025-11-04T19:15:45.797452] [DEBUG] [VALIDATOR ] GROV: Primary data validation passed. [2025-11-04T19:15:45.797876] [INFO ] [MAIN ] GROV: Fetched primary data is valid. [2025-11-04T19:15:45.797976] [DEBUG] [LOCAL ] GROV: Saving primary data. [2025-11-04T19:15:45.798526] [DEBUG] [LOCAL ] GROV: Trimmed 13360 records to 8899 (keeping 200 days of data) [2025-11-04T19:15:45.848070] [DEBUG] [LOCAL ] GROV: Primary data saved (8899 records). [2025-11-04T19:15:45.848206] [INFO ] [MAIN ] GROV: Full history saved. [2025-11-04T19:15:45.848292] [INFO ] [MAIN ] Update Progress: 969/2367 symbols completed. GROV took 22.2s. Estimated remaining: 3h 15m 47s. Rate: 7.1 symbols/min [2025-11-04T19:15:46.026668] [INFO ] [MAIN ] GS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:15:46.075274] [DEBUG] [LOCAL ] GS: Loaded 51698 existing records from primary storage. [2025-11-04T19:15:46.075622] [INFO ] [MAIN ] GS: Loaded 51698 existing records. [2025-11-04T19:15:46.257188] [WARN ] [MAIN ] GS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:15:46.257414] [INFO ] [MAIN ] GS: Wiping existing primary data before full refresh. [2025-11-04T19:15:46.257461] [INFO ] [LOCAL ] GS: Starting deletion of primary data. [2025-11-04T19:15:46.258234] [DEBUG] [LOCAL ] GS: Deleted primary data file. [2025-11-04T19:15:46.258292] [INFO ] [MAIN ] GS: Starting full primary refresh. [2025-11-04T19:15:46.572190] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-06. [2025-11-04T19:15:46.625715] [INFO ] [REMOTE ] GS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:15:46.992162] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T19:15:47.367075] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-01. [2025-11-04T19:15:47.626334] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T19:15:48.167043] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-07. [2025-11-04T19:15:48.967063] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-10. [2025-11-04T19:15:49.001632] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T19:15:49.202100] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-11-04T19:15:49.767028] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-08. [2025-11-04T19:15:50.402679] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T19:15:50.567019] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-02. [2025-11-04T19:15:50.687346] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-11-04T19:15:51.367021] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-11. [2025-11-04T19:15:52.034191] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-11-04T19:15:52.167009] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-09. [2025-11-04T19:15:52.461131] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T19:15:52.462957] [INFO ] [REMOTE ] GRND: Full fetch complete (72213 records). [2025-11-04T19:15:52.463126] [DEBUG] [VALIDATOR ] GRND: Validating primary data. [2025-11-04T19:15:52.855254] [DEBUG] [VALIDATOR ] GRND: All expected trading days are present. [2025-11-04T19:15:52.855341] [DEBUG] [VALIDATOR ] GRND: Primary data validation passed. [2025-11-04T19:15:52.856215] [INFO ] [MAIN ] GRND: Fetched primary data is valid. [2025-11-04T19:15:52.856281] [DEBUG] [LOCAL ] GRND: Saving primary data. [2025-11-04T19:15:52.857749] [DEBUG] [LOCAL ] GRND: Trimmed 72213 records to 50157 (keeping 200 days of data) [2025-11-04T19:15:52.968688] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-01. [2025-11-04T19:15:53.166111] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T19:15:53.183017] [DEBUG] [LOCAL ] GRND: Primary data saved (50157 records). [2025-11-04T19:15:53.190413] [INFO ] [MAIN ] GRND: Full history saved. [2025-11-04T19:15:53.190532] [INFO ] [MAIN ] Update Progress: 970/2367 symbols completed. GRND took 34.1s. Estimated remaining: 3h 15m 38s. Rate: 7.1 symbols/min [2025-11-04T19:15:53.364513] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-01 (0.40 seconds). [2025-11-04T19:15:53.365972] [INFO ] [MAIN ] GSBD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:15:53.403220] [DEBUG] [LOCAL ] GSBD: Loaded 39371 existing records from primary storage. [2025-11-04T19:15:53.403484] [INFO ] [MAIN ] GSBD: Loaded 39371 existing records. [2025-11-04T19:15:53.582601] [WARN ] [MAIN ] GSBD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:15:53.582808] [INFO ] [MAIN ] GSBD: Wiping existing primary data before full refresh. [2025-11-04T19:15:53.582859] [INFO ] [LOCAL ] GSBD: Starting deletion of primary data. [2025-11-04T19:15:53.583035] [DEBUG] [LOCAL ] GSBD: Deleted primary data file. [2025-11-04T19:15:53.583080] [INFO ] [MAIN ] GSBD: Starting full primary refresh. [2025-11-04T19:15:53.776503] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-03. [2025-11-04T19:15:53.935195] [INFO ] [REMOTE ] GSBD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:15:54.567056] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-10. [2025-11-04T19:15:55.367026] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-02. [2025-11-04T19:15:55.590499] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-11-04T19:15:55.663750] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-02 (0.30 seconds). [2025-11-04T19:15:55.686440] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-11-04T19:15:56.167072] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-04. [2025-11-04T19:15:56.967044] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-01. [2025-11-04T19:15:57.767030] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-03. [2025-11-04T19:15:57.847412] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-11-04T19:15:58.102263] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T19:15:58.115329] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-11-04T19:15:58.567045] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-04. [2025-11-04T19:15:59.008213] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-11-04T19:15:59.367040] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-05. [2025-11-04T19:15:59.737520] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-11-04T19:16:00.167046] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-11. [2025-11-04T19:16:00.967031] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-05. [2025-11-04T19:16:01.006852] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T19:16:01.008430] [INFO ] [REMOTE ] GRNT: Full fetch complete (48115 records). [2025-11-04T19:16:01.008537] [DEBUG] [VALIDATOR ] GRNT: Validating primary data. [2025-11-04T19:16:01.388482] [DEBUG] [VALIDATOR ] GRNT: All expected trading days are present. [2025-11-04T19:16:01.388570] [DEBUG] [VALIDATOR ] GRNT: Primary data validation passed. [2025-11-04T19:16:01.389378] [INFO ] [MAIN ] GRNT: Fetched primary data is valid. [2025-11-04T19:16:01.389457] [DEBUG] [LOCAL ] GRNT: Saving primary data. [2025-11-04T19:16:01.390600] [DEBUG] [LOCAL ] GRNT: Trimmed 48115 records to 33232 (keeping 200 days of data) [2025-11-04T19:16:01.596938] [DEBUG] [LOCAL ] GRNT: Primary data saved (33232 records). [2025-11-04T19:16:01.597087] [INFO ] [MAIN ] GRNT: Full history saved. [2025-11-04T19:16:01.597183] [INFO ] [MAIN ] Update Progress: 971/2367 symbols completed. GRNT took 40.9s. Estimated remaining: 3h 15m 29s. Rate: 7.1 symbols/min [2025-11-04T19:16:01.768624] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-02. [2025-11-04T19:16:01.768521] [INFO ] [MAIN ] GSK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:16:01.818844] [DEBUG] [LOCAL ] GSK: Loaded 53511 existing records from primary storage. [2025-11-04T19:16:01.819188] [INFO ] [MAIN ] GSK: Loaded 53511 existing records. [2025-11-04T19:16:01.996949] [WARN ] [MAIN ] GSK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:16:01.997395] [INFO ] [MAIN ] GSK: Wiping existing primary data before full refresh. [2025-11-04T19:16:01.997443] [INFO ] [LOCAL ] GSK: Starting deletion of primary data. [2025-11-04T19:16:01.998618] [DEBUG] [LOCAL ] GSK: Deleted primary data file. [2025-11-04T19:16:01.998711] [INFO ] [MAIN ] GSK: Starting full primary refresh. [2025-11-04T19:16:02.389925] [INFO ] [REMOTE ] GSK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:16:02.567040] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-01. [2025-11-04T19:16:02.738300] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-11-04T19:16:02.778167] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-11-04T19:16:03.367056] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-06. [2025-11-04T19:16:03.927100] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T19:16:04.167047] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-06. [2025-11-04T19:16:04.566680] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-11-04T19:16:04.835704] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T19:16:04.967121] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-03. [2025-11-04T19:16:05.767105] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-07. [2025-11-04T19:16:06.051534] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T19:16:06.072491] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-07 (0.31 seconds). [2025-11-04T19:16:06.567061] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-07. [2025-11-04T19:16:07.367057] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-08. [2025-11-04T19:16:07.831369] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-11-04T19:16:08.167024] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-04. [2025-11-04T19:16:08.232624] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-11-04T19:16:08.967043] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-08. [2025-11-04T19:16:09.445032] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-11-04T19:16:09.767065] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-09. [2025-11-04T19:16:10.099703] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-09 (0.33 seconds). [2025-11-04T19:16:10.259596] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T19:16:10.567043] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-05. [2025-11-04T19:16:11.367060] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-02. [2025-11-04T19:16:11.681627] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T19:16:12.167096] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-09. [2025-11-04T19:16:12.704944] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-11-04T19:16:12.967054] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-10. [2025-11-04T19:16:13.375821] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-11-04T19:16:13.582399] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T19:16:13.767048] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-11. [2025-11-04T19:16:14.267128] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T19:16:14.268032] [INFO ] [REMOTE ] GRX: Full fetch complete (6533 records). [2025-11-04T19:16:14.268109] [DEBUG] [VALIDATOR ] GRX: Validating primary data. [2025-11-04T19:16:14.572121] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-10. [2025-11-04T19:16:14.631205] [DEBUG] [VALIDATOR ] GRX: All expected trading days are present. [2025-11-04T19:16:14.631311] [DEBUG] [VALIDATOR ] GRX: Primary data validation passed. [2025-11-04T19:16:14.631542] [INFO ] [MAIN ] GRX: Fetched primary data is valid. [2025-11-04T19:16:14.631604] [DEBUG] [LOCAL ] GRX: Saving primary data. [2025-11-04T19:16:14.632141] [DEBUG] [LOCAL ] GRX: Trimmed 6533 records to 4835 (keeping 200 days of data) [2025-11-04T19:16:14.660607] [DEBUG] [LOCAL ] GRX: Primary data saved (4835 records). [2025-11-04T19:16:14.660739] [INFO ] [MAIN ] GRX: Full history saved. [2025-11-04T19:16:14.660822] [INFO ] [MAIN ] Update Progress: 972/2367 symbols completed. GRX took 31.9s. Estimated remaining: 3h 15m 27s. Rate: 7.1 symbols/min [2025-11-04T19:16:14.828659] [INFO ] [MAIN ] GSL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:16:14.856912] [DEBUG] [LOCAL ] GSL: Loaded 32406 existing records from primary storage. [2025-11-04T19:16:14.857185] [INFO ] [MAIN ] GSL: Loaded 32406 existing records. [2025-11-04T19:16:15.031453] [WARN ] [MAIN ] GSL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:16:15.031727] [INFO ] [MAIN ] GSL: Wiping existing primary data before full refresh. [2025-11-04T19:16:15.031808] [INFO ] [LOCAL ] GSL: Starting deletion of primary data. [2025-11-04T19:16:15.032392] [DEBUG] [LOCAL ] GSL: Deleted primary data file. [2025-11-04T19:16:15.032450] [INFO ] [MAIN ] GSL: Starting full primary refresh. [2025-11-04T19:16:15.373092] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-06. [2025-11-04T19:16:15.377616] [INFO ] [REMOTE ] GSL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:16:16.167009] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-03. [2025-11-04T19:16:16.397849] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-11-04T19:16:16.428222] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-11-04T19:16:16.967010] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-07. [2025-11-04T19:16:17.610343] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T19:16:17.767094] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-11. [2025-11-04T19:16:18.057858] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-11 (0.29 seconds). [2025-11-04T19:16:18.067895] [INFO ] [REMOTE ] GS: Full fetch complete (75803 records). [2025-11-04T19:16:18.081130] [DEBUG] [VALIDATOR ] GS: Validating primary data. [2025-11-04T19:16:18.082394] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-11-04T19:16:18.473062] [DEBUG] [VALIDATOR ] GS: All expected trading days are present. [2025-11-04T19:16:18.473138] [DEBUG] [VALIDATOR ] GS: Primary data validation passed. [2025-11-04T19:16:18.473993] [INFO ] [MAIN ] GS: Fetched primary data is valid. [2025-11-04T19:16:18.474056] [DEBUG] [LOCAL ] GS: Saving primary data. [2025-11-04T19:16:18.475397] [DEBUG] [LOCAL ] GS: Trimmed 75803 records to 51698 (keeping 200 days of data) [2025-11-04T19:16:18.568063] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-04. [2025-11-04T19:16:18.795725] [DEBUG] [LOCAL ] GS: Primary data saved (51698 records). [2025-11-04T19:16:18.795848] [INFO ] [MAIN ] GS: Full history saved. [2025-11-04T19:16:18.795922] [INFO ] [MAIN ] Update Progress: 973/2367 symbols completed. GS took 32.9s. Estimated remaining: 3h 15m 13s. Rate: 7.1 symbols/min [2025-11-04T19:16:18.976632] [INFO ] [MAIN ] GTES: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:16:19.022181] [DEBUG] [LOCAL ] GTES: Loaded 50296 existing records from primary storage. [2025-11-04T19:16:19.022679] [INFO ] [MAIN ] GTES: Loaded 50296 existing records. [2025-11-04T19:16:19.201423] [WARN ] [MAIN ] GTES: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:16:19.201620] [INFO ] [MAIN ] GTES: Wiping existing primary data before full refresh. [2025-11-04T19:16:19.201700] [INFO ] [LOCAL ] GTES: Starting deletion of primary data. [2025-11-04T19:16:19.202539] [DEBUG] [LOCAL ] GTES: Deleted primary data file. [2025-11-04T19:16:19.202597] [INFO ] [MAIN ] GTES: Starting full primary refresh. [2025-11-04T19:16:19.376545] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-01. [2025-11-04T19:16:19.566954] [INFO ] [REMOTE ] GTES: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:16:19.993346] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T19:16:20.167030] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-05. [2025-11-04T19:16:20.233527] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T19:16:20.967092] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-08. [2025-11-04T19:16:21.438051] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-11-04T19:16:21.767074] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-02. [2025-11-04T19:16:22.015127] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-11-04T19:16:22.567077] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-09. [2025-11-04T19:16:22.800297] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T19:16:23.367028] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-03. [2025-11-04T19:16:23.615859] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T19:16:24.167009] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-06. [2025-11-04T19:16:24.362899] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-11-04T19:16:24.967019] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-10. [2025-11-04T19:16:25.512393] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-11-04T19:16:25.767079] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-01. [2025-11-04T19:16:26.276425] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T19:16:26.567084] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-07. [2025-11-04T19:16:27.070993] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T19:16:27.367041] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-11. [2025-11-04T19:16:28.117360] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-11-04T19:16:28.167026] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-04. [2025-11-04T19:16:28.373834] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-11 (1.01 seconds). [2025-11-04T19:16:28.375487] [INFO ] [REMOTE ] GSBD: Full fetch complete (58518 records). [2025-11-04T19:16:28.375565] [DEBUG] [VALIDATOR ] GSBD: Validating primary data. [2025-11-04T19:16:28.772892] [DEBUG] [VALIDATOR ] GSBD: All expected trading days are present. [2025-11-04T19:16:28.772966] [DEBUG] [VALIDATOR ] GSBD: Primary data validation passed. [2025-11-04T19:16:28.773688] [INFO ] [MAIN ] GSBD: Fetched primary data is valid. [2025-11-04T19:16:28.773775] [DEBUG] [LOCAL ] GSBD: Saving primary data. [2025-11-04T19:16:28.774863] [DEBUG] [LOCAL ] GSBD: Trimmed 58518 records to 39389 (keeping 200 days of data) [2025-11-04T19:16:28.972860] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-08. [2025-11-04T19:16:29.008017] [DEBUG] [LOCAL ] GSBD: Primary data saved (39389 records). [2025-11-04T19:16:29.008134] [INFO ] [MAIN ] GSBD: Full history saved. [2025-11-04T19:16:29.008212] [INFO ] [MAIN ] Update Progress: 974/2367 symbols completed. GSBD took 35.8s. Estimated remaining: 3h 15m 7s. Rate: 7.1 symbols/min [2025-11-04T19:16:29.186868] [INFO ] [MAIN ] GTLS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:16:29.210286] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-11-04T19:16:29.259791] [DEBUG] [LOCAL ] GTLS: Loaded 42676 existing records from primary storage. [2025-11-04T19:16:29.260050] [INFO ] [MAIN ] GTLS: Loaded 42676 existing records. [2025-11-04T19:16:29.456680] [WARN ] [MAIN ] GTLS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:16:29.456916] [INFO ] [MAIN ] GTLS: Wiping existing primary data before full refresh. [2025-11-04T19:16:29.456961] [INFO ] [LOCAL ] GTLS: Starting deletion of primary data. [2025-11-04T19:16:29.457822] [DEBUG] [LOCAL ] GTLS: Deleted primary data file. [2025-11-04T19:16:29.457881] [INFO ] [MAIN ] GTLS: Starting full primary refresh. [2025-11-04T19:16:29.772182] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-02. [2025-11-04T19:16:29.808348] [INFO ] [REMOTE ] GTLS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:16:30.512853] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-11-04T19:16:30.567026] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-05. [2025-11-04T19:16:31.036336] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T19:16:31.367086] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-09. [2025-11-04T19:16:31.438102] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T19:16:32.167041] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-03. [2025-11-04T19:16:32.967039] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-01. [2025-11-04T19:16:33.006457] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-11-04T19:16:33.386370] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T19:16:33.767058] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-10. [2025-11-04T19:16:34.087965] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T19:16:34.567045] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-04. [2025-11-04T19:16:35.367055] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-06. [2025-11-04T19:16:35.569852] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-11-04T19:16:35.929434] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:16:36.167050] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-05. [2025-11-04T19:16:36.445536] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T19:16:36.967054] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-11. [2025-11-04T19:16:37.393676] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T19:16:37.767077] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-07. [2025-11-04T19:16:38.436166] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-11 (1.47 seconds). [2025-11-04T19:16:38.438575] [INFO ] [REMOTE ] GSK: Full fetch complete (77636 records). [2025-11-04T19:16:38.438799] [DEBUG] [VALIDATOR ] GSK: Validating primary data. [2025-11-04T19:16:38.572270] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-06. [2025-11-04T19:16:38.858702] [DEBUG] [VALIDATOR ] GSK: All expected trading days are present. [2025-11-04T19:16:38.858823] [DEBUG] [VALIDATOR ] GSK: Primary data validation passed. [2025-11-04T19:16:38.860009] [INFO ] [MAIN ] GSK: Fetched primary data is valid. [2025-11-04T19:16:38.860166] [DEBUG] [LOCAL ] GSK: Saving primary data. [2025-11-04T19:16:38.881193] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T19:16:38.880050] [DEBUG] [LOCAL ] GSK: Trimmed 77636 records to 53509 (keeping 200 days of data) [2025-11-04T19:16:39.208046] [DEBUG] [LOCAL ] GSK: Primary data saved (53509 records). [2025-11-04T19:16:39.208175] [INFO ] [MAIN ] GSK: Full history saved. [2025-11-04T19:16:39.208249] [INFO ] [MAIN ] Update Progress: 975/2367 symbols completed. GSK took 37.6s. Estimated remaining: 3h 15m 1s. Rate: 7.1 symbols/min [2025-11-04T19:16:39.372226] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-02. [2025-11-04T19:16:39.374870] [INFO ] [MAIN ] GTN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:16:39.412256] [DEBUG] [LOCAL ] GTN: Loaded 44239 existing records from primary storage. [2025-11-04T19:16:39.412529] [INFO ] [MAIN ] GTN: Loaded 44239 existing records. [2025-11-04T19:16:39.603969] [WARN ] [MAIN ] GTN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:16:39.604184] [INFO ] [MAIN ] GTN: Wiping existing primary data before full refresh. [2025-11-04T19:16:39.604231] [INFO ] [LOCAL ] GTN: Starting deletion of primary data. [2025-11-04T19:16:39.605121] [DEBUG] [LOCAL ] GTN: Deleted primary data file. [2025-11-04T19:16:39.605178] [INFO ] [MAIN ] GTN: Starting full primary refresh. [2025-11-04T19:16:39.961342] [INFO ] [REMOTE ] GTN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:16:39.993452] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-11-04T19:16:40.167066] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-08. [2025-11-04T19:16:40.343422] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T19:16:40.967081] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-07. [2025-11-04T19:16:41.202377] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T19:16:41.767096] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-01. [2025-11-04T19:16:42.119920] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-11-04T19:16:42.567054] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-08. [2025-11-04T19:16:42.912957] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T19:16:43.367056] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-09. [2025-11-04T19:16:43.752218] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T19:16:44.167034] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-02. [2025-11-04T19:16:44.315244] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-11-04T19:16:44.967039] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-09. [2025-11-04T19:16:45.390128] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T19:16:45.767092] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-03. [2025-11-04T19:16:46.504177] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-11-04T19:16:46.567034] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-10. [2025-11-04T19:16:47.148597] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T19:16:47.367019] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-10. [2025-11-04T19:16:47.951816] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T19:16:48.167023] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-11. [2025-11-04T19:16:48.583865] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-11-04T19:16:48.967036] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-11. [2025-11-04T19:16:49.165514] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T19:16:49.167545] [INFO ] [REMOTE ] GTES: Full fetch complete (73893 records). [2025-11-04T19:16:49.167679] [DEBUG] [VALIDATOR ] GTES: Validating primary data. [2025-11-04T19:16:49.594748] [DEBUG] [VALIDATOR ] GTES: All expected trading days are present. [2025-11-04T19:16:49.594849] [DEBUG] [VALIDATOR ] GTES: Primary data validation passed. [2025-11-04T19:16:49.595720] [INFO ] [MAIN ] GTES: Fetched primary data is valid. [2025-11-04T19:16:49.595780] [DEBUG] [LOCAL ] GTES: Saving primary data. [2025-11-04T19:16:49.597202] [DEBUG] [LOCAL ] GTES: Trimmed 73893 records to 50332 (keeping 200 days of data) [2025-11-04T19:16:49.767737] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-03. [2025-11-04T19:16:49.890560] [DEBUG] [LOCAL ] GTES: Primary data saved (50332 records). [2025-11-04T19:16:49.890715] [INFO ] [MAIN ] GTES: Full history saved. [2025-11-04T19:16:49.890803] [INFO ] [MAIN ] Update Progress: 976/2367 symbols completed. GTES took 31.1s. Estimated remaining: 3h 14m 56s. Rate: 7.1 symbols/min [2025-11-04T19:16:50.038567] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T19:16:50.072177] [INFO ] [MAIN ] GTY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:16:50.072316] [INFO ] [REMOTE ] GSL: Full fetch complete (48364 records). [2025-11-04T19:16:50.072901] [DEBUG] [VALIDATOR ] GSL: Validating primary data. [2025-11-04T19:16:50.335890] [DEBUG] [LOCAL ] GTY: Loaded 29596 existing records from primary storage. [2025-11-04T19:16:50.373074] [INFO ] [MAIN ] GTY: Loaded 29596 existing records. [2025-11-04T19:16:50.583864] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-04. [2025-11-04T19:16:50.665229] [DEBUG] [VALIDATOR ] GSL: All expected trading days are present. [2025-11-04T19:16:50.667954] [DEBUG] [VALIDATOR ] GSL: Primary data validation passed. [2025-11-04T19:16:50.691804] [INFO ] [MAIN ] GSL: Fetched primary data is valid. [2025-11-04T19:16:50.694454] [DEBUG] [LOCAL ] GSL: Saving primary data. [2025-11-04T19:16:50.698468] [WARN ] [MAIN ] GTY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:16:50.698930] [INFO ] [MAIN ] GTY: Wiping existing primary data before full refresh. [2025-11-04T19:16:50.699011] [INFO ] [LOCAL ] GTY: Starting deletion of primary data. [2025-11-04T19:16:50.700109] [DEBUG] [LOCAL ] GTY: Deleted primary data file. [2025-11-04T19:16:50.700283] [INFO ] [MAIN ] GTY: Starting full primary refresh. [2025-11-04T19:16:50.764104] [DEBUG] [LOCAL ] GSL: Trimmed 48364 records to 32483 (keeping 200 days of data) [2025-11-04T19:16:51.196374] [INFO ] [REMOTE ] GTY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:16:51.232553] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T19:16:51.341146] [DEBUG] [LOCAL ] GSL: Primary data saved (32483 records). [2025-11-04T19:16:51.341283] [INFO ] [MAIN ] GSL: Full history saved. [2025-11-04T19:16:51.341363] [INFO ] [MAIN ] Update Progress: 977/2367 symbols completed. GSL took 36.7s. Estimated remaining: 3h 14m 38s. Rate: 7.1 symbols/min [2025-11-04T19:16:51.372227] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-01. [2025-11-04T19:16:51.511816] [INFO ] [MAIN ] GUG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:16:51.523364] [DEBUG] [LOCAL ] GUG: Loaded 11856 existing records from primary storage. [2025-11-04T19:16:51.523554] [INFO ] [MAIN ] GUG: Loaded 11856 existing records. [2025-11-04T19:16:51.698183] [WARN ] [MAIN ] GUG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:16:51.698408] [INFO ] [MAIN ] GUG: Wiping existing primary data before full refresh. [2025-11-04T19:16:51.698454] [INFO ] [LOCAL ] GUG: Starting deletion of primary data. [2025-11-04T19:16:51.698879] [DEBUG] [LOCAL ] GUG: Deleted primary data file. [2025-11-04T19:16:51.698936] [INFO ] [MAIN ] GUG: Starting full primary refresh. [2025-11-04T19:16:52.060512] [INFO ] [REMOTE ] GUG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:16:52.137940] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-11-04T19:16:52.167033] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-01. [2025-11-04T19:16:52.391973] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T19:16:52.762973] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-11-04T19:16:52.967060] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-02. [2025-11-04T19:16:53.539747] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-11-04T19:16:53.767066] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-02. [2025-11-04T19:16:54.567017] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-04. [2025-11-04T19:16:54.613072] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T19:16:55.367036] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-03. [2025-11-04T19:16:55.917718] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:16:56.167042] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-05. [2025-11-04T19:16:56.478956] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T19:16:56.967103] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-04. [2025-11-04T19:16:57.334369] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T19:16:57.767073] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-05. [2025-11-04T19:16:58.034542] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T19:16:58.567062] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-03. [2025-11-04T19:16:58.838371] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T19:16:59.080047] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-11-04T19:16:59.367057] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-04. [2025-11-04T19:16:59.968105] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-11-04T19:17:00.167045] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-06. [2025-11-04T19:17:00.967048] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-05. [2025-11-04T19:17:01.278153] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T19:17:01.767092] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-05. [2025-11-04T19:17:01.909094] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T19:17:02.299838] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-11-04T19:17:02.567055] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-06. [2025-11-04T19:17:03.367065] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-06. [2025-11-04T19:17:03.612343] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T19:17:04.167018] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-07. [2025-11-04T19:17:04.223215] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T19:17:04.967071] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-07. [2025-11-04T19:17:05.481978] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T19:17:05.767079] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-08. [2025-11-04T19:17:06.096536] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T19:17:06.567035] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-07. [2025-11-04T19:17:07.053722] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T19:17:07.367027] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-06. [2025-11-04T19:17:07.788202] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-11-04T19:17:07.975398] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T19:17:08.167106] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-08. [2025-11-04T19:17:08.967014] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-09. [2025-11-04T19:17:09.158909] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-11-04T19:17:09.767093] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-09. [2025-11-04T19:17:10.118017] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T19:17:10.567021] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-08. [2025-11-04T19:17:10.683057] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T19:17:11.367064] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-10. [2025-11-04T19:17:12.021207] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T19:17:12.167052] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-09. [2025-11-04T19:17:12.303226] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T19:17:12.967055] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-10. [2025-11-04T19:17:13.354908] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T19:17:13.767034] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-10. [2025-11-04T19:17:14.118688] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-11-04T19:17:14.567021] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-11. [2025-11-04T19:17:14.790727] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T19:17:15.367051] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-07. [2025-11-04T19:17:15.734238] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-11 (1.17 seconds). [2025-11-04T19:17:15.735936] [INFO ] [REMOTE ] GTN: Full fetch complete (65427 records). [2025-11-04T19:17:15.736026] [DEBUG] [VALIDATOR ] GTN: Validating primary data. [2025-11-04T19:17:16.045916] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-11-04T19:17:16.136030] [DEBUG] [VALIDATOR ] GTN: All expected trading days are present. [2025-11-04T19:17:16.136118] [DEBUG] [VALIDATOR ] GTN: Primary data validation passed. [2025-11-04T19:17:16.136973] [INFO ] [MAIN ] GTN: Fetched primary data is valid. [2025-11-04T19:17:16.137063] [DEBUG] [LOCAL ] GTN: Saving primary data. [2025-11-04T19:17:16.138391] [DEBUG] [LOCAL ] GTN: Trimmed 65427 records to 44229 (keeping 200 days of data) [2025-11-04T19:17:16.185415] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-11. [2025-11-04T19:17:16.393757] [DEBUG] [LOCAL ] GTN: Primary data saved (44229 records). [2025-11-04T19:17:16.393882] [INFO ] [MAIN ] GTN: Full history saved. [2025-11-04T19:17:16.393968] [INFO ] [MAIN ] Update Progress: 978/2367 symbols completed. GTN took 37.2s. Estimated remaining: 3h 14m 53s. Rate: 7.1 symbols/min [2025-11-04T19:17:16.559388] [INFO ] [MAIN ] GUT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:17:16.573597] [DEBUG] [LOCAL ] GUT: Loaded 15288 existing records from primary storage. [2025-11-04T19:17:16.573835] [INFO ] [MAIN ] GUT: Loaded 15288 existing records. [2025-11-04T19:17:16.747261] [WARN ] [MAIN ] GUT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:17:16.747507] [INFO ] [MAIN ] GUT: Wiping existing primary data before full refresh. [2025-11-04T19:17:16.747561] [INFO ] [LOCAL ] GUT: Starting deletion of primary data. [2025-11-04T19:17:16.747988] [DEBUG] [LOCAL ] GUT: Deleted primary data file. [2025-11-04T19:17:16.748047] [INFO ] [MAIN ] GUT: Starting full primary refresh. [2025-11-04T19:17:16.975344] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-11. [2025-11-04T19:17:17.096635] [INFO ] [REMOTE ] GUT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:17:17.322678] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T19:17:17.324271] [INFO ] [REMOTE ] GTLS: Full fetch complete (61284 records). [2025-11-04T19:17:17.324349] [DEBUG] [VALIDATOR ] GTLS: Validating primary data. [2025-11-04T19:17:17.711327] [DEBUG] [VALIDATOR ] GTLS: All expected trading days are present. [2025-11-04T19:17:17.711414] [DEBUG] [VALIDATOR ] GTLS: Primary data validation passed. [2025-11-04T19:17:17.712193] [INFO ] [MAIN ] GTLS: Fetched primary data is valid. [2025-11-04T19:17:17.712259] [DEBUG] [LOCAL ] GTLS: Saving primary data. [2025-11-04T19:17:17.713609] [DEBUG] [LOCAL ] GTLS: Trimmed 61284 records to 42715 (keeping 200 days of data) [2025-11-04T19:17:17.771420] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-01. [2025-11-04T19:17:17.892858] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T19:17:17.919845] [INFO ] [REMOTE ] GTY: Full fetch complete (43786 records). [2025-11-04T19:17:17.928822] [DEBUG] [VALIDATOR ] GTY: Validating primary data. [2025-11-04T19:17:18.117739] [DEBUG] [LOCAL ] GTLS: Primary data saved (42715 records). [2025-11-04T19:17:18.131992] [INFO ] [MAIN ] GTLS: Full history saved. [2025-11-04T19:17:18.132412] [INFO ] [MAIN ] Update Progress: 979/2367 symbols completed. GTLS took 49.1s. Estimated remaining: 3h 14m 35s. Rate: 7.1 symbols/min [2025-11-04T19:17:18.452837] [INFO ] [MAIN ] GVA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:17:18.526919] [DEBUG] [VALIDATOR ] GTY: All expected trading days are present. [2025-11-04T19:17:18.527062] [DEBUG] [VALIDATOR ] GTY: Primary data validation passed. [2025-11-04T19:17:18.527882] [INFO ] [MAIN ] GTY: Fetched primary data is valid. [2025-11-04T19:17:18.527997] [DEBUG] [LOCAL ] GTY: Saving primary data. [2025-11-04T19:17:18.541348] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-11-04T19:17:18.543132] [DEBUG] [LOCAL ] GTY: Trimmed 43786 records to 29646 (keeping 200 days of data) [2025-11-04T19:17:18.606023] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-08. [2025-11-04T19:17:18.657049] [DEBUG] [LOCAL ] GVA: Loaded 31800 existing records from primary storage. [2025-11-04T19:17:18.667180] [INFO ] [MAIN ] GVA: Loaded 31800 existing records. [2025-11-04T19:17:18.903177] [WARN ] [MAIN ] GVA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:17:18.924692] [INFO ] [MAIN ] GVA: Wiping existing primary data before full refresh. [2025-11-04T19:17:18.929713] [INFO ] [LOCAL ] GVA: Starting deletion of primary data. [2025-11-04T19:17:18.935374] [DEBUG] [LOCAL ] GVA: Deleted primary data file. [2025-11-04T19:17:18.935482] [INFO ] [MAIN ] GVA: Starting full primary refresh. [2025-11-04T19:17:18.935324] [DEBUG] [LOCAL ] GTY: Primary data saved (29646 records). [2025-11-04T19:17:18.968508] [INFO ] [MAIN ] GTY: Full history saved. [2025-11-04T19:17:19.017146] [INFO ] [MAIN ] Update Progress: 980/2367 symbols completed. GTY took 29.1s. Estimated remaining: 3h 14m 16s. Rate: 7.1 symbols/min [2025-11-04T19:17:19.394381] [INFO ] [MAIN ] GWH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:17:19.390974] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-02. [2025-11-04T19:17:19.421603] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-11-04T19:17:19.510891] [INFO ] [REMOTE ] GVA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:17:19.520301] [DEBUG] [LOCAL ] GWH: Loaded 22068 existing records from primary storage. [2025-11-04T19:17:19.520527] [INFO ] [MAIN ] GWH: Loaded 22068 existing records. [2025-11-04T19:17:19.725982] [WARN ] [MAIN ] GWH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:17:19.726192] [INFO ] [MAIN ] GWH: Wiping existing primary data before full refresh. [2025-11-04T19:17:19.726239] [INFO ] [LOCAL ] GWH: Starting deletion of primary data. [2025-11-04T19:17:19.726746] [DEBUG] [LOCAL ] GWH: Deleted primary data file. [2025-11-04T19:17:19.726804] [INFO ] [MAIN ] GWH: Starting full primary refresh. [2025-11-04T19:17:20.091621] [INFO ] [REMOTE ] GWH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:17:20.100875] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T19:17:20.167001] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-01. [2025-11-04T19:17:20.729183] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T19:17:20.967054] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-01. [2025-11-04T19:17:21.767018] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-09. [2025-11-04T19:17:21.933717] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-11-04T19:17:22.392517] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:17:22.567038] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-10. [2025-11-04T19:17:23.176546] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-11-04T19:17:23.367038] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-11. [2025-11-04T19:17:24.118408] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-11 (0.75 seconds). [2025-11-04T19:17:24.119555] [INFO ] [REMOTE ] GUG: Full fetch complete (18404 records). [2025-11-04T19:17:24.119627] [DEBUG] [VALIDATOR ] GUG: Validating primary data. [2025-11-04T19:17:24.172357] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-02. [2025-11-04T19:17:24.482580] [DEBUG] [VALIDATOR ] GUG: All expected trading days are present. [2025-11-04T19:17:24.482685] [DEBUG] [VALIDATOR ] GUG: Primary data validation passed. [2025-11-04T19:17:24.483085] [INFO ] [MAIN ] GUG: Fetched primary data is valid. [2025-11-04T19:17:24.483125] [DEBUG] [LOCAL ] GUG: Saving primary data. [2025-11-04T19:17:24.483746] [DEBUG] [LOCAL ] GUG: Trimmed 18404 records to 11974 (keeping 200 days of data) [2025-11-04T19:17:24.554694] [DEBUG] [LOCAL ] GUG: Primary data saved (11974 records). [2025-11-04T19:17:24.554832] [INFO ] [MAIN ] GUG: Full history saved. [2025-11-04T19:17:24.554911] [INFO ] [MAIN ] Update Progress: 981/2367 symbols completed. GUG took 33.2s. Estimated remaining: 3h 14m 4s. Rate: 7.1 symbols/min [2025-11-04T19:17:24.721418] [INFO ] [MAIN ] GWRE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:17:24.754940] [DEBUG] [LOCAL ] GWRE: Loaded 39230 existing records from primary storage. [2025-11-04T19:17:24.755211] [INFO ] [MAIN ] GWRE: Loaded 39230 existing records. [2025-11-04T19:17:24.928749] [WARN ] [MAIN ] GWRE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:17:24.928951] [INFO ] [MAIN ] GWRE: Wiping existing primary data before full refresh. [2025-11-04T19:17:24.929026] [INFO ] [LOCAL ] GWRE: Starting deletion of primary data. [2025-11-04T19:17:24.929837] [DEBUG] [LOCAL ] GWRE: Deleted primary data file. [2025-11-04T19:17:24.929902] [INFO ] [MAIN ] GWRE: Starting full primary refresh. [2025-11-04T19:17:24.972198] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-02. [2025-11-04T19:17:25.290754] [INFO ] [REMOTE ] GWRE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:17:25.308116] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T19:17:25.699908] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-11-04T19:17:25.767097] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-03. [2025-11-04T19:17:25.828990] [DEBUG] [REMOTE ] GWH: Error fetching primary data for 2025-03: 'API call returned an empty response.' [2025-11-04T19:17:25.829474] [WARN ] [REMOTE ] GWH: No data returned for month 2025-03 [2025-11-04T19:17:25.829575] [FATAL] [REMOTE ] GWH: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:17:25.829708] [FATAL] [MAIN ] GWH: Failed to fetch primary data. [2025-11-04T19:17:25.829802] [WARN ] [MAIN ] GWH: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:17:25.829871] [INFO ] [LOCAL ] GWH: Starting deletion of primary data. [2025-11-04T19:17:25.829981] [DEBUG] [LOCAL ] GWH: No primary data file found to delete. [2025-11-04T19:17:25.830038] [INFO ] [LOCAL ] GWH: Starting deletion of derivative data. [2025-11-04T19:17:25.830226] [DEBUG] [LOCAL ] GWH: Deleted derivative data file. [2025-11-04T19:17:25.830322] [INFO ] [MAIN ] Update Progress: 982/2367 symbols completed. GWH took 6.8s. Estimated remaining: 3h 13m 45s. Rate: 7.1 symbols/min [2025-11-04T19:17:25.996283] [INFO ] [MAIN ] GWW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:17:26.019424] [DEBUG] [LOCAL ] GWW: Loaded 19161 existing records from primary storage. [2025-11-04T19:17:26.019870] [INFO ] [MAIN ] GWW: Loaded 19161 existing records. [2025-11-04T19:17:26.239231] [WARN ] [MAIN ] GWW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:17:26.239558] [INFO ] [MAIN ] GWW: Wiping existing primary data before full refresh. [2025-11-04T19:17:26.239612] [INFO ] [LOCAL ] GWW: Starting deletion of primary data. [2025-11-04T19:17:26.240239] [DEBUG] [LOCAL ] GWW: Deleted primary data file. [2025-11-04T19:17:26.240298] [INFO ] [MAIN ] GWW: Starting full primary refresh. [2025-11-04T19:17:26.572261] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-03. [2025-11-04T19:17:26.638149] [INFO ] [REMOTE ] GWW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:17:27.367123] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-01. [2025-11-04T19:17:27.766956] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T19:17:28.167078] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-01. [2025-11-04T19:17:28.580419] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T19:17:28.967095] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-04. [2025-11-04T19:17:29.039810] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T19:17:29.767046] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-02. [2025-11-04T19:17:30.038394] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T19:17:30.377016] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-11-04T19:17:30.567050] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-03. [2025-11-04T19:17:31.367038] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-03. [2025-11-04T19:17:31.643933] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T19:17:32.150445] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-11-04T19:17:32.166988] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-04. [2025-11-04T19:17:32.864448] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T19:17:32.967047] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-02. [2025-11-04T19:17:33.767060] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-05. [2025-11-04T19:17:34.117986] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T19:17:34.567036] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-03. [2025-11-04T19:17:34.868470] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-11-04T19:17:35.367064] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-04. [2025-11-04T19:17:35.932261] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T19:17:36.167075] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-04. [2025-11-04T19:17:36.204763] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-11-04T19:17:36.967066] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-06. [2025-11-04T19:17:37.452808] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T19:17:37.767036] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-05. [2025-11-04T19:17:37.865596] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-11-04T19:17:38.567059] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-07. [2025-11-04T19:17:38.831802] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T19:17:39.367014] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-05. [2025-11-04T19:17:39.491031] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T19:17:39.990583] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-11-04T19:17:40.167039] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-05. [2025-11-04T19:17:40.782873] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-11-04T19:17:40.967028] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-08. [2025-11-04T19:17:41.767010] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-06. [2025-11-04T19:17:41.896449] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-11-04T19:17:42.435268] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-11-04T19:17:42.567081] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-06. [2025-11-04T19:17:43.367089] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-09. [2025-11-04T19:17:43.764929] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T19:17:44.167967] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-06. [2025-11-04T19:17:44.397031] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-11-04T19:17:44.921831] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-11-04T19:17:44.967047] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-07. [2025-11-04T19:17:45.767045] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-07. [2025-11-04T19:17:46.250593] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T19:17:46.364883] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T19:17:46.567025] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-07. [2025-11-04T19:17:47.261125] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-11-04T19:17:47.367033] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-10. [2025-11-04T19:17:48.167062] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-08. [2025-11-04T19:17:48.423105] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-11-04T19:17:48.825269] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-11-04T19:17:48.967035] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-11. [2025-11-04T19:17:49.767060] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-08. [2025-11-04T19:17:49.839534] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T19:17:49.841621] [INFO ] [REMOTE ] GVA: Full fetch complete (48516 records). [2025-11-04T19:17:49.841772] [DEBUG] [VALIDATOR ] GVA: Validating primary data. [2025-11-04T19:17:50.233922] [DEBUG] [VALIDATOR ] GVA: All expected trading days are present. [2025-11-04T19:17:50.234037] [DEBUG] [VALIDATOR ] GVA: Primary data validation passed. [2025-11-04T19:17:50.234799] [INFO ] [MAIN ] GVA: Fetched primary data is valid. [2025-11-04T19:17:50.234865] [DEBUG] [LOCAL ] GVA: Saving primary data. [2025-11-04T19:17:50.235978] [DEBUG] [LOCAL ] GVA: Trimmed 48516 records to 31770 (keeping 200 days of data) [2025-11-04T19:17:50.437477] [DEBUG] [LOCAL ] GVA: Primary data saved (31770 records). [2025-11-04T19:17:50.437679] [INFO ] [MAIN ] GVA: Full history saved. [2025-11-04T19:17:50.437784] [INFO ] [MAIN ] Update Progress: 983/2367 symbols completed. GVA took 32.3s. Estimated remaining: 3h 13m 60s. Rate: 7.1 symbols/min [2025-11-04T19:17:50.575344] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-09. [2025-11-04T19:17:50.623707] [INFO ] [MAIN ] GXO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:17:50.664481] [DEBUG] [LOCAL ] GXO: Loaded 45853 existing records from primary storage. [2025-11-04T19:17:50.664999] [INFO ] [MAIN ] GXO: Loaded 45853 existing records. [2025-11-04T19:17:50.790572] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-11-04T19:17:50.871438] [WARN ] [MAIN ] GXO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:17:50.871629] [INFO ] [MAIN ] GXO: Wiping existing primary data before full refresh. [2025-11-04T19:17:50.871711] [INFO ] [LOCAL ] GXO: Starting deletion of primary data. [2025-11-04T19:17:50.872571] [DEBUG] [LOCAL ] GXO: Deleted primary data file. [2025-11-04T19:17:50.872630] [INFO ] [MAIN ] GXO: Starting full primary refresh. [2025-11-04T19:17:51.258478] [INFO ] [REMOTE ] GXO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:17:51.260042] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T19:17:51.367048] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-10. [2025-11-04T19:17:52.134307] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-11-04T19:17:52.166993] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-11. [2025-11-04T19:17:52.722169] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-11 (0.56 seconds). [2025-11-04T19:17:52.723124] [INFO ] [REMOTE ] GUT: Full fetch complete (22262 records). [2025-11-04T19:17:52.723194] [DEBUG] [VALIDATOR ] GUT: Validating primary data. [2025-11-04T19:17:52.972141] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-01. [2025-11-04T19:17:53.082197] [DEBUG] [VALIDATOR ] GUT: All expected trading days are present. [2025-11-04T19:17:53.082273] [DEBUG] [VALIDATOR ] GUT: Primary data validation passed. [2025-11-04T19:17:53.082760] [INFO ] [MAIN ] GUT: Fetched primary data is valid. [2025-11-04T19:17:53.082821] [DEBUG] [LOCAL ] GUT: Saving primary data. [2025-11-04T19:17:53.083381] [DEBUG] [LOCAL ] GUT: Trimmed 22262 records to 15277 (keeping 200 days of data) [2025-11-04T19:17:53.169035] [DEBUG] [LOCAL ] GUT: Primary data saved (15277 records). [2025-11-04T19:17:53.169162] [INFO ] [MAIN ] GUT: Full history saved. [2025-11-04T19:17:53.169238] [INFO ] [MAIN ] Update Progress: 984/2367 symbols completed. GUT took 36.8s. Estimated remaining: 3h 13m 43s. Rate: 7.1 symbols/min [2025-11-04T19:17:53.333973] [INFO ] [MAIN ] GYLD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:17:53.336941] [DEBUG] [LOCAL ] GYLD: Loaded 1525 existing records from primary storage. [2025-11-04T19:17:53.337077] [INFO ] [MAIN ] GYLD: Loaded 1525 existing records. [2025-11-04T19:17:53.512856] [WARN ] [MAIN ] GYLD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:17:53.513125] [INFO ] [MAIN ] GYLD: Wiping existing primary data before full refresh. [2025-11-04T19:17:53.513175] [INFO ] [LOCAL ] GYLD: Starting deletion of primary data. [2025-11-04T19:17:53.513351] [DEBUG] [LOCAL ] GYLD: Deleted primary data file. [2025-11-04T19:17:53.513407] [INFO ] [MAIN ] GYLD: Starting full primary refresh. [2025-11-04T19:17:53.772135] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-09. [2025-11-04T19:17:53.859709] [INFO ] [REMOTE ] GYLD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:17:54.164190] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T19:17:54.567025] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-08. [2025-11-04T19:17:54.674097] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-11-04T19:17:55.367019] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-10. [2025-11-04T19:17:55.558727] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-11-04T19:17:56.156722] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T19:17:56.167000] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-11. [2025-11-04T19:17:56.963982] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-11 (0.80 seconds). [2025-11-04T19:17:56.966165] [INFO ] [REMOTE ] GWW: Full fetch complete (29300 records). [2025-11-04T19:17:56.966445] [DEBUG] [VALIDATOR ] GWW: Validating primary data. [2025-11-04T19:17:56.972245] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-01. [2025-11-04T19:17:57.372526] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-01 (0.40 seconds). [2025-11-04T19:17:57.386520] [DEBUG] [VALIDATOR ] GWW: All expected trading days are present. [2025-11-04T19:17:57.386599] [DEBUG] [VALIDATOR ] GWW: Primary data validation passed. [2025-11-04T19:17:57.387198] [INFO ] [MAIN ] GWW: Fetched primary data is valid. [2025-11-04T19:17:57.387260] [DEBUG] [LOCAL ] GWW: Saving primary data. [2025-11-04T19:17:57.387980] [DEBUG] [LOCAL ] GWW: Trimmed 29300 records to 19292 (keeping 200 days of data) [2025-11-04T19:17:57.511439] [DEBUG] [LOCAL ] GWW: Primary data saved (19292 records). [2025-11-04T19:17:57.511610] [INFO ] [MAIN ] GWW: Full history saved. [2025-11-04T19:17:57.511756] [INFO ] [MAIN ] Update Progress: 985/2367 symbols completed. GWW took 31.7s. Estimated remaining: 3h 13m 29s. Rate: 7.1 symbols/min [2025-11-04T19:17:57.680626] [INFO ] [MAIN ] H: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:17:57.720242] [DEBUG] [LOCAL ] H: Loaded 44268 existing records from primary storage. [2025-11-04T19:17:57.720534] [INFO ] [MAIN ] H: Loaded 44268 existing records. [2025-11-04T19:17:57.773614] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-02. [2025-11-04T19:17:57.900095] [WARN ] [MAIN ] H: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:17:57.900321] [INFO ] [MAIN ] H: Wiping existing primary data before full refresh. [2025-11-04T19:17:57.900375] [INFO ] [LOCAL ] H: Starting deletion of primary data. [2025-11-04T19:17:57.901159] [DEBUG] [LOCAL ] H: Deleted primary data file. [2025-11-04T19:17:57.901215] [INFO ] [MAIN ] H: Starting full primary refresh. [2025-11-04T19:17:58.156141] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T19:17:58.263969] [INFO ] [REMOTE ] H: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:17:58.567040] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-09. [2025-11-04T19:17:59.367070] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-03. [2025-11-04T19:17:59.678171] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-11-04T19:17:59.747474] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T19:18:00.167067] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-04. [2025-11-04T19:18:00.449206] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-11-04T19:18:00.967058] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-02. [2025-11-04T19:18:01.767109] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-05. [2025-11-04T19:18:02.050676] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T19:18:02.062636] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-11-04T19:18:02.567038] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-06. [2025-11-04T19:18:02.937151] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-11-04T19:18:03.367043] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-07. [2025-11-04T19:18:03.743595] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-11-04T19:18:04.167053] [INFO ] [REMOTE ] H: Fetching primary data for 2025-01. [2025-11-04T19:18:04.967062] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-08. [2025-11-04T19:18:05.135966] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-11-04T19:18:05.324328] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-11-04T19:18:05.767074] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-10. [2025-11-04T19:18:06.567080] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-09. [2025-11-04T19:18:06.860384] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-11-04T19:18:06.876395] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-09 (0.31 seconds). [2025-11-04T19:18:07.367045] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-11. [2025-11-04T19:18:08.167009] [INFO ] [REMOTE ] H: Fetching primary data for 2025-02. [2025-11-04T19:18:08.252351] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T19:18:08.254020] [INFO ] [REMOTE ] GWRE: Full fetch complete (59703 records). [2025-11-04T19:18:08.254101] [DEBUG] [VALIDATOR ] GWRE: Validating primary data. [2025-11-04T19:18:08.628794] [DEBUG] [VALIDATOR ] GWRE: All expected trading days are present. [2025-11-04T19:18:08.628879] [DEBUG] [VALIDATOR ] GWRE: Primary data validation passed. [2025-11-04T19:18:08.629619] [INFO ] [MAIN ] GWRE: Fetched primary data is valid. [2025-11-04T19:18:08.629713] [DEBUG] [LOCAL ] GWRE: Saving primary data. [2025-11-04T19:18:08.630944] [DEBUG] [LOCAL ] GWRE: Trimmed 59703 records to 39254 (keeping 200 days of data) [2025-11-04T19:18:08.863531] [DEBUG] [LOCAL ] GWRE: Primary data saved (39254 records). [2025-11-04T19:18:08.863700] [INFO ] [MAIN ] GWRE: Full history saved. [2025-11-04T19:18:08.863791] [INFO ] [MAIN ] Update Progress: 986/2367 symbols completed. GWRE took 44.3s. Estimated remaining: 3h 13m 25s. Rate: 7.1 symbols/min [2025-11-04T19:18:08.968006] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-03. [2025-11-04T19:18:09.037063] [INFO ] [MAIN ] HAE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:18:09.070033] [DEBUG] [LOCAL ] HAE: Loaded 38785 existing records from primary storage. [2025-11-04T19:18:09.070308] [INFO ] [MAIN ] HAE: Loaded 38785 existing records. [2025-11-04T19:18:09.248967] [WARN ] [MAIN ] HAE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:18:09.249183] [INFO ] [MAIN ] HAE: Wiping existing primary data before full refresh. [2025-11-04T19:18:09.249294] [INFO ] [LOCAL ] HAE: Starting deletion of primary data. [2025-11-04T19:18:09.250043] [DEBUG] [LOCAL ] HAE: Deleted primary data file. [2025-11-04T19:18:09.250100] [INFO ] [MAIN ] HAE: Starting full primary refresh. [2025-11-04T19:18:09.632371] [INFO ] [REMOTE ] HAE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:18:09.646930] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-11-04T19:18:09.767080] [INFO ] [REMOTE ] H: Fetching primary data for 2025-03. [2025-11-04T19:18:10.337773] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T19:18:10.567082] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-04. [2025-11-04T19:18:10.845193] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T19:18:11.367046] [INFO ] [REMOTE ] H: Fetching primary data for 2025-04. [2025-11-04T19:18:11.875617] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T19:18:12.167068] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-01. [2025-11-04T19:18:12.540275] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-11-04T19:18:12.967070] [INFO ] [REMOTE ] H: Fetching primary data for 2025-05. [2025-11-04T19:18:13.323037] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T19:18:13.767062] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-02. [2025-11-04T19:18:14.177169] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T19:18:14.567101] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-05. [2025-11-04T19:18:14.929631] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T19:18:15.367100] [INFO ] [REMOTE ] H: Fetching primary data for 2025-06. [2025-11-04T19:18:15.682527] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-11-04T19:18:16.167045] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-06. [2025-11-04T19:18:16.965857] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-11-04T19:18:16.974815] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-03. [2025-11-04T19:18:17.320267] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T19:18:17.767088] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-10. [2025-11-04T19:18:18.149488] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-11-04T19:18:18.185372] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-11-04T19:18:18.567036] [INFO ] [REMOTE ] H: Fetching primary data for 2025-07. [2025-11-04T19:18:19.367072] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-04. [2025-11-04T19:18:19.884084] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T19:18:20.167057] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-07. [2025-11-04T19:18:20.538863] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-11-04T19:18:20.967026] [INFO ] [REMOTE ] H: Fetching primary data for 2025-08. [2025-11-04T19:18:21.157834] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-11-04T19:18:21.767053] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-05. [2025-11-04T19:18:22.332420] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-11-04T19:18:22.567051] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-11. [2025-11-04T19:18:22.834994] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T19:18:22.948467] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T19:18:22.949141] [INFO ] [REMOTE ] GYLD: Full fetch complete (2139 records). [2025-11-04T19:18:22.949207] [DEBUG] [VALIDATOR ] GYLD: Validating primary data. [2025-11-04T19:18:23.323807] [DEBUG] [VALIDATOR ] GYLD: All expected trading days are present. [2025-11-04T19:18:23.323909] [DEBUG] [VALIDATOR ] GYLD: Primary data validation passed. [2025-11-04T19:18:23.324020] [INFO ] [MAIN ] GYLD: Fetched primary data is valid. [2025-11-04T19:18:23.324062] [DEBUG] [LOCAL ] GYLD: Saving primary data. [2025-11-04T19:18:23.324372] [DEBUG] [LOCAL ] GYLD: Trimmed 2139 records to 1518 (keeping 200 days of data) [2025-11-04T19:18:23.333769] [DEBUG] [LOCAL ] GYLD: Primary data saved (1518 records). [2025-11-04T19:18:23.333826] [INFO ] [MAIN ] GYLD: Full history saved. [2025-11-04T19:18:23.333888] [INFO ] [MAIN ] Update Progress: 987/2367 symbols completed. GYLD took 30.2s. Estimated remaining: 3h 13m 25s. Rate: 7.1 symbols/min [2025-11-04T19:18:23.372747] [INFO ] [REMOTE ] H: Fetching primary data for 2025-09. [2025-11-04T19:18:23.510591] [INFO ] [MAIN ] HAFN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:18:23.546397] [DEBUG] [LOCAL ] HAFN: Loaded 42173 existing records from primary storage. [2025-11-04T19:18:23.546740] [INFO ] [MAIN ] HAFN: Loaded 42173 existing records. [2025-11-04T19:18:23.729410] [WARN ] [MAIN ] HAFN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:18:23.729686] [INFO ] [MAIN ] HAFN: Wiping existing primary data before full refresh. [2025-11-04T19:18:23.729749] [INFO ] [LOCAL ] HAFN: Starting deletion of primary data. [2025-11-04T19:18:23.730684] [DEBUG] [LOCAL ] HAFN: Deleted primary data file. [2025-11-04T19:18:23.730776] [INFO ] [MAIN ] HAFN: Starting full primary refresh. [2025-11-04T19:18:24.093842] [INFO ] [REMOTE ] HAFN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:18:24.167031] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-01. [2025-11-04T19:18:24.523178] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T19:18:24.967046] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-08. [2025-11-04T19:18:25.767025] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-06. [2025-11-04T19:18:25.865287] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-11-04T19:18:26.219747] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T19:18:26.567101] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-09. [2025-11-04T19:18:26.812795] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T19:18:27.367070] [INFO ] [REMOTE ] H: Fetching primary data for 2025-10. [2025-11-04T19:18:27.687571] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T19:18:28.167063] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-10. [2025-11-04T19:18:28.658074] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T19:18:28.967076] [INFO ] [REMOTE ] H: Fetching primary data for 2025-11. [2025-11-04T19:18:29.538320] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T19:18:29.767050] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-11. [2025-11-04T19:18:30.060867] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T19:18:30.062813] [INFO ] [REMOTE ] H: Full fetch complete (64131 records). [2025-11-04T19:18:30.062906] [DEBUG] [VALIDATOR ] H: Validating primary data. [2025-11-04T19:18:30.439670] [DEBUG] [VALIDATOR ] H: All expected trading days are present. [2025-11-04T19:18:30.439762] [DEBUG] [VALIDATOR ] H: Primary data validation passed. [2025-11-04T19:18:30.440536] [INFO ] [MAIN ] H: Fetched primary data is valid. [2025-11-04T19:18:30.440576] [DEBUG] [LOCAL ] H: Saving primary data. [2025-11-04T19:18:30.441818] [DEBUG] [LOCAL ] H: Trimmed 64131 records to 44301 (keeping 200 days of data) [2025-11-04T19:18:30.617403] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-07. [2025-11-04T19:18:30.718935] [DEBUG] [LOCAL ] H: Primary data saved (44301 records). [2025-11-04T19:18:30.719068] [INFO ] [MAIN ] H: Full history saved. [2025-11-04T19:18:30.719173] [INFO ] [MAIN ] Update Progress: 988/2367 symbols completed. H took 33.2s. Estimated remaining: 3h 13m 15s. Rate: 7.1 symbols/min [2025-11-04T19:18:30.895697] [INFO ] [MAIN ] HAL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:18:30.894258] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:18:30.898307] [INFO ] [REMOTE ] GXO: Full fetch complete (69059 records). [2025-11-04T19:18:30.898453] [DEBUG] [VALIDATOR ] GXO: Validating primary data. [2025-11-04T19:18:31.245415] [DEBUG] [LOCAL ] HAL: Loaded 53718 existing records from primary storage. [2025-11-04T19:18:31.267335] [INFO ] [MAIN ] HAL: Loaded 53718 existing records. [2025-11-04T19:18:31.390224] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-02. [2025-11-04T19:18:31.424611] [DEBUG] [VALIDATOR ] GXO: All expected trading days are present. [2025-11-04T19:18:31.441435] [DEBUG] [VALIDATOR ] GXO: Primary data validation passed. [2025-11-04T19:18:31.479128] [INFO ] [MAIN ] GXO: Fetched primary data is valid. [2025-11-04T19:18:31.487473] [DEBUG] [LOCAL ] GXO: Saving primary data. [2025-11-04T19:18:31.507189] [WARN ] [MAIN ] HAL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:18:31.507477] [INFO ] [MAIN ] HAL: Wiping existing primary data before full refresh. [2025-11-04T19:18:31.507530] [INFO ] [LOCAL ] HAL: Starting deletion of primary data. [2025-11-04T19:18:31.508602] [DEBUG] [LOCAL ] HAL: Deleted primary data file. [2025-11-04T19:18:31.508689] [INFO ] [MAIN ] HAL: Starting full primary refresh. [2025-11-04T19:18:31.540336] [DEBUG] [LOCAL ] GXO: Trimmed 69059 records to 45846 (keeping 200 days of data) [2025-11-04T19:18:32.128502] [INFO ] [REMOTE ] HAL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:18:32.145298] [DEBUG] [LOCAL ] GXO: Primary data saved (45846 records). [2025-11-04T19:18:32.145950] [INFO ] [MAIN ] GXO: Full history saved. [2025-11-04T19:18:32.146762] [INFO ] [MAIN ] Update Progress: 989/2367 symbols completed. GXO took 41.7s. Estimated remaining: 3h 12m 57s. Rate: 7.1 symbols/min [2025-11-04T19:18:32.186755] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-01. [2025-11-04T19:18:32.252389] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-11-04T19:18:32.359445] [INFO ] [MAIN ] HASI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:18:32.400550] [DEBUG] [LOCAL ] HASI: Loaded 47238 existing records from primary storage. [2025-11-04T19:18:32.400873] [INFO ] [MAIN ] HASI: Loaded 47238 existing records. [2025-11-04T19:18:32.578336] [WARN ] [MAIN ] HASI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:18:32.578610] [INFO ] [MAIN ] HASI: Wiping existing primary data before full refresh. [2025-11-04T19:18:32.578702] [INFO ] [LOCAL ] HASI: Starting deletion of primary data. [2025-11-04T19:18:32.579637] [DEBUG] [LOCAL ] HASI: Deleted primary data file. [2025-11-04T19:18:32.579741] [INFO ] [MAIN ] HASI: Starting full primary refresh. [2025-11-04T19:18:32.968967] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-08. [2025-11-04T19:18:32.968850] [INFO ] [REMOTE ] HASI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:18:32.973234] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-11-04T19:18:33.767027] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-01. [2025-11-04T19:18:33.836128] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-11-04T19:18:34.515567] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-11-04T19:18:34.567059] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-02. [2025-11-04T19:18:34.781557] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T19:18:35.367054] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-09. [2025-11-04T19:18:35.990938] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-11-04T19:18:36.167052] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-03. [2025-11-04T19:18:36.474017] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T19:18:36.967046] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-02. [2025-11-04T19:18:37.767053] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-10. [2025-11-04T19:18:37.848014] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-11-04T19:18:37.885201] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-11-04T19:18:38.567050] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-04. [2025-11-04T19:18:38.963569] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T19:18:39.367035] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-03. [2025-11-04T19:18:40.167008] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-11. [2025-11-04T19:18:40.328312] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-11-04T19:18:40.850585] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-11-04T19:18:40.967027] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-03. [2025-11-04T19:18:41.321455] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T19:18:41.323672] [INFO ] [REMOTE ] HAE: Full fetch complete (59276 records). [2025-11-04T19:18:41.323824] [DEBUG] [VALIDATOR ] HAE: Validating primary data. [2025-11-04T19:18:41.717617] [DEBUG] [VALIDATOR ] HAE: All expected trading days are present. [2025-11-04T19:18:41.717734] [DEBUG] [VALIDATOR ] HAE: Primary data validation passed. [2025-11-04T19:18:41.718509] [INFO ] [MAIN ] HAE: Fetched primary data is valid. [2025-11-04T19:18:41.718551] [DEBUG] [LOCAL ] HAE: Saving primary data. [2025-11-04T19:18:41.719824] [DEBUG] [LOCAL ] HAE: Trimmed 59276 records to 38730 (keeping 200 days of data) [2025-11-04T19:18:41.768290] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-04. [2025-11-04T19:18:41.945074] [DEBUG] [LOCAL ] HAE: Primary data saved (38730 records). [2025-11-04T19:18:41.945231] [INFO ] [MAIN ] HAE: Full history saved. [2025-11-04T19:18:41.945310] [INFO ] [MAIN ] Update Progress: 990/2367 symbols completed. HAE took 33.1s. Estimated remaining: 3h 12m 50s. Rate: 7.1 symbols/min [2025-11-04T19:18:42.122945] [INFO ] [MAIN ] HAYW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:18:42.165370] [DEBUG] [LOCAL ] HAYW: Loaded 48824 existing records from primary storage. [2025-11-04T19:18:42.165694] [INFO ] [MAIN ] HAYW: Loaded 48824 existing records. [2025-11-04T19:18:42.342629] [WARN ] [MAIN ] HAYW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:18:42.353353] [INFO ] [MAIN ] HAYW: Wiping existing primary data before full refresh. [2025-11-04T19:18:42.359294] [INFO ] [LOCAL ] HAYW: Starting deletion of primary data. [2025-11-04T19:18:42.370030] [DEBUG] [LOCAL ] HAYW: Deleted primary data file. [2025-11-04T19:18:42.371155] [INFO ] [MAIN ] HAYW: Starting full primary refresh. [2025-11-04T19:18:42.371071] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T19:18:42.572576] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-05. [2025-11-04T19:18:42.739865] [INFO ] [REMOTE ] HAYW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:18:43.116761] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:18:43.367053] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-05. [2025-11-04T19:18:44.167038] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-04. [2025-11-04T19:18:44.384198] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-11-04T19:18:44.666244] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T19:18:44.967093] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-06. [2025-11-04T19:18:45.530748] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:18:45.767077] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-06. [2025-11-04T19:18:46.150444] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T19:18:46.567067] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-07. [2025-11-04T19:18:47.234388] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T19:18:47.367052] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-07. [2025-11-04T19:18:47.736721] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T19:18:48.167083] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-01. [2025-11-04T19:18:48.967065] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-08. [2025-11-04T19:18:49.136831] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-11-04T19:18:49.303669] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T19:18:49.767081] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-08. [2025-11-04T19:18:49.988021] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-11-04T19:18:50.567059] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-02. [2025-11-04T19:18:51.340296] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-11-04T19:18:51.367020] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-09. [2025-11-04T19:18:51.629975] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T19:18:52.167080] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-05. [2025-11-04T19:18:52.729511] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T19:18:52.967050] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-09. [2025-11-04T19:18:53.659318] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-11-04T19:18:53.767162] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-06. [2025-11-04T19:18:54.139550] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T19:18:54.567064] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-03. [2025-11-04T19:18:54.984829] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-11-04T19:18:55.367042] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-10. [2025-11-04T19:18:55.772597] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-11-04T19:18:56.167137] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-07. [2025-11-04T19:18:56.523610] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T19:18:56.967173] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-10. [2025-11-04T19:18:57.421013] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T19:18:57.767074] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-04. [2025-11-04T19:18:58.424230] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T19:18:58.567016] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-11. [2025-11-04T19:18:58.811176] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-11 (0.24 seconds). [2025-11-04T19:18:58.813285] [INFO ] [REMOTE ] HAL: Full fetch complete (77897 records). [2025-11-04T19:18:58.813513] [DEBUG] [VALIDATOR ] HAL: Validating primary data. [2025-11-04T19:18:59.210760] [DEBUG] [VALIDATOR ] HAL: All expected trading days are present. [2025-11-04T19:18:59.210875] [DEBUG] [VALIDATOR ] HAL: Primary data validation passed. [2025-11-04T19:18:59.211849] [INFO ] [MAIN ] HAL: Fetched primary data is valid. [2025-11-04T19:18:59.211954] [DEBUG] [LOCAL ] HAL: Saving primary data. [2025-11-04T19:18:59.213537] [DEBUG] [LOCAL ] HAL: Trimmed 77897 records to 53718 (keeping 200 days of data) [2025-11-04T19:18:59.325884] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-11-04T19:18:59.367233] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-11. [2025-11-04T19:18:59.593677] [DEBUG] [LOCAL ] HAL: Primary data saved (53718 records). [2025-11-04T19:18:59.593887] [INFO ] [MAIN ] HAL: Full history saved. [2025-11-04T19:18:59.593994] [INFO ] [MAIN ] Update Progress: 991/2367 symbols completed. HAL took 28.9s. Estimated remaining: 3h 12m 55s. Rate: 7.1 symbols/min [2025-11-04T19:18:59.763268] [INFO ] [MAIN ] HBB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:18:59.770492] [DEBUG] [LOCAL ] HBB: Loaded 5745 existing records from primary storage. [2025-11-04T19:18:59.770737] [INFO ] [MAIN ] HBB: Loaded 5745 existing records. [2025-11-04T19:18:59.949842] [WARN ] [MAIN ] HBB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:18:59.950074] [INFO ] [MAIN ] HBB: Wiping existing primary data before full refresh. [2025-11-04T19:18:59.950120] [INFO ] [LOCAL ] HBB: Starting deletion of primary data. [2025-11-04T19:18:59.950444] [DEBUG] [LOCAL ] HBB: Deleted primary data file. [2025-11-04T19:18:59.950500] [INFO ] [MAIN ] HBB: Starting full primary refresh. [2025-11-04T19:19:00.172262] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-08. [2025-11-04T19:19:00.321910] [INFO ] [REMOTE ] HBB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:19:00.669372] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T19:19:00.671034] [INFO ] [REMOTE ] HAFN: Full fetch complete (64142 records). [2025-11-04T19:19:00.671133] [DEBUG] [VALIDATOR ] HAFN: Validating primary data. [2025-11-04T19:19:00.972153] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-05. [2025-11-04T19:19:01.051665] [DEBUG] [VALIDATOR ] HAFN: All expected trading days are present. [2025-11-04T19:19:01.051750] [DEBUG] [VALIDATOR ] HAFN: Primary data validation passed. [2025-11-04T19:19:01.052529] [INFO ] [MAIN ] HAFN: Fetched primary data is valid. [2025-11-04T19:19:01.052572] [DEBUG] [LOCAL ] HAFN: Saving primary data. [2025-11-04T19:19:01.053828] [DEBUG] [LOCAL ] HAFN: Trimmed 64142 records to 42138 (keeping 200 days of data) [2025-11-04T19:19:01.289139] [DEBUG] [LOCAL ] HAFN: Primary data saved (42138 records). [2025-11-04T19:19:01.289255] [INFO ] [MAIN ] HAFN: Full history saved. [2025-11-04T19:19:01.289328] [INFO ] [MAIN ] Update Progress: 992/2367 symbols completed. HAFN took 38.0s. Estimated remaining: 3h 12m 37s. Rate: 7.1 symbols/min [2025-11-04T19:19:01.458147] [INFO ] [MAIN ] HBI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:19:01.503744] [DEBUG] [LOCAL ] HBI: Loaded 52858 existing records from primary storage. [2025-11-04T19:19:01.504232] [INFO ] [MAIN ] HBI: Loaded 52858 existing records. [2025-11-04T19:19:01.665577] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T19:19:01.741419] [WARN ] [MAIN ] HBI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:19:01.741699] [INFO ] [MAIN ] HBI: Wiping existing primary data before full refresh. [2025-11-04T19:19:01.741759] [INFO ] [LOCAL ] HBI: Starting deletion of primary data. [2025-11-04T19:19:01.742687] [DEBUG] [LOCAL ] HBI: Deleted primary data file. [2025-11-04T19:19:01.742746] [INFO ] [MAIN ] HBI: Starting full primary refresh. [2025-11-04T19:19:01.776756] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-09. [2025-11-04T19:19:02.141663] [INFO ] [REMOTE ] HBI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:19:02.167636] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T19:19:02.567148] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-01. [2025-11-04T19:19:03.256711] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-11-04T19:19:03.366995] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-10. [2025-11-04T19:19:04.019133] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T19:19:04.167006] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-01. [2025-11-04T19:19:04.506912] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-11-04T19:19:04.634387] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T19:19:04.967035] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-02. [2025-11-04T19:19:05.767050] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-02. [2025-11-04T19:19:06.145449] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T19:19:06.166152] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-11-04T19:19:06.567029] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-06. [2025-11-04T19:19:07.367055] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-11. [2025-11-04T19:19:07.929424] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T19:19:08.167019] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-07. [2025-11-04T19:19:08.476173] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T19:19:08.478033] [INFO ] [REMOTE ] HASI: Full fetch complete (68300 records). [2025-11-04T19:19:08.478121] [DEBUG] [VALIDATOR ] HASI: Validating primary data. [2025-11-04T19:19:08.874144] [DEBUG] [VALIDATOR ] HASI: All expected trading days are present. [2025-11-04T19:19:08.874258] [DEBUG] [VALIDATOR ] HASI: Primary data validation passed. [2025-11-04T19:19:08.875137] [INFO ] [MAIN ] HASI: Fetched primary data is valid. [2025-11-04T19:19:08.875202] [DEBUG] [LOCAL ] HASI: Saving primary data. [2025-11-04T19:19:08.876558] [DEBUG] [LOCAL ] HASI: Trimmed 68300 records to 47225 (keeping 200 days of data) [2025-11-04T19:19:08.967241] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-03. [2025-11-04T19:19:09.173791] [DEBUG] [LOCAL ] HASI: Primary data saved (47225 records). [2025-11-04T19:19:09.173945] [INFO ] [MAIN ] HASI: Full history saved. [2025-11-04T19:19:09.174041] [INFO ] [MAIN ] Update Progress: 993/2367 symbols completed. HASI took 37.0s. Estimated remaining: 3h 12m 28s. Rate: 7.1 symbols/min [2025-11-04T19:19:09.363474] [INFO ] [MAIN ] HBM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:19:09.398803] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T19:19:09.450814] [DEBUG] [LOCAL ] HBM: Loaded 52987 existing records from primary storage. [2025-11-04T19:19:09.451184] [INFO ] [MAIN ] HBM: Loaded 52987 existing records. [2025-11-04T19:19:09.567550] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-11-04T19:19:09.646446] [WARN ] [MAIN ] HBM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:19:09.646738] [INFO ] [MAIN ] HBM: Wiping existing primary data before full refresh. [2025-11-04T19:19:09.646796] [INFO ] [LOCAL ] HBM: Starting deletion of primary data. [2025-11-04T19:19:09.647627] [DEBUG] [LOCAL ] HBM: Deleted primary data file. [2025-11-04T19:19:09.647716] [INFO ] [MAIN ] HBM: Starting full primary refresh. [2025-11-04T19:19:09.772120] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-04. [2025-11-04T19:19:10.011212] [INFO ] [REMOTE ] HBM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:19:10.398709] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-11-04T19:19:10.567071] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-03. [2025-11-04T19:19:11.367031] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-01. [2025-11-04T19:19:11.976171] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T19:19:12.167050] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-08. [2025-11-04T19:19:12.643554] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T19:19:12.967043] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-02. [2025-11-04T19:19:13.286336] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-11-04T19:19:13.767078] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-04. [2025-11-04T19:19:14.403672] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-11-04T19:19:14.567049] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-03. [2025-11-04T19:19:15.130575] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:19:15.367087] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-09. [2025-11-04T19:19:16.094725] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T19:19:16.167073] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-05. [2025-11-04T19:19:16.689207] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-11-04T19:19:16.967039] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-10. [2025-11-04T19:19:17.589967] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-11-04T19:19:17.767040] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-06. [2025-11-04T19:19:18.031885] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-11-04T19:19:18.567019] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-05. [2025-11-04T19:19:18.997773] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T19:19:19.056607] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-11-04T19:19:19.367046] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-07. [2025-11-04T19:19:20.167077] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-06. [2025-11-04T19:19:20.579826] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-11-04T19:19:20.614161] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T19:19:20.967113] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-07. [2025-11-04T19:19:21.422826] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-11-04T19:19:21.767091] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-08. [2025-11-04T19:19:22.254808] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-11-04T19:19:22.567034] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-04. [2025-11-04T19:19:23.367065] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-09. [2025-11-04T19:19:23.647748] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-09 (0.28 seconds). [2025-11-04T19:19:24.021158] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T19:19:24.167058] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-05. [2025-11-04T19:19:24.967091] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-10. [2025-11-04T19:19:25.271671] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-10 (0.30 seconds). [2025-11-04T19:19:25.550709] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-11-04T19:19:25.767068] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-11. [2025-11-04T19:19:26.567094] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-08. [2025-11-04T19:19:27.007199] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T19:19:27.009220] [INFO ] [REMOTE ] HAYW: Full fetch complete (71002 records). [2025-11-04T19:19:27.009479] [DEBUG] [VALIDATOR ] HAYW: Validating primary data. [2025-11-04T19:19:27.391293] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-11. [2025-11-04T19:19:27.414222] [DEBUG] [VALIDATOR ] HAYW: All expected trading days are present. [2025-11-04T19:19:27.414421] [DEBUG] [VALIDATOR ] HAYW: Primary data validation passed. [2025-11-04T19:19:27.415539] [INFO ] [MAIN ] HAYW: Fetched primary data is valid. [2025-11-04T19:19:27.416215] [DEBUG] [LOCAL ] HAYW: Saving primary data. [2025-11-04T19:19:27.420724] [DEBUG] [LOCAL ] HAYW: Trimmed 71002 records to 48864 (keeping 200 days of data) [2025-11-04T19:19:27.702613] [DEBUG] [LOCAL ] HAYW: Primary data saved (48864 records). [2025-11-04T19:19:27.702768] [INFO ] [MAIN ] HAYW: Full history saved. [2025-11-04T19:19:27.702850] [INFO ] [MAIN ] Update Progress: 994/2367 symbols completed. HAYW took 45.8s. Estimated remaining: 3h 12m 34s. Rate: 7.1 symbols/min [2025-11-04T19:19:27.871582] [INFO ] [MAIN ] HCA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:19:27.897829] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-11 (0.51 seconds). [2025-11-04T19:19:27.898619] [INFO ] [REMOTE ] HBB: Full fetch complete (9002 records). [2025-11-04T19:19:27.898721] [DEBUG] [VALIDATOR ] HBB: Validating primary data. [2025-11-04T19:19:28.015179] [DEBUG] [LOCAL ] HCA: Loaded 46872 existing records from primary storage. [2025-11-04T19:19:28.019602] [INFO ] [MAIN ] HCA: Loaded 46872 existing records. [2025-11-04T19:19:28.174054] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-06. [2025-11-04T19:19:28.395039] [WARN ] [MAIN ] HCA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:19:28.416729] [INFO ] [MAIN ] HCA: Wiping existing primary data before full refresh. [2025-11-04T19:19:28.417249] [INFO ] [LOCAL ] HCA: Starting deletion of primary data. [2025-11-04T19:19:28.427934] [DEBUG] [LOCAL ] HCA: Deleted primary data file. [2025-11-04T19:19:28.457709] [INFO ] [MAIN ] HCA: Starting full primary refresh. [2025-11-04T19:19:28.537895] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-11-04T19:19:28.614017] [DEBUG] [VALIDATOR ] HBB: All expected trading days are present. [2025-11-04T19:19:28.619579] [DEBUG] [VALIDATOR ] HBB: Primary data validation passed. [2025-11-04T19:19:28.625030] [INFO ] [MAIN ] HBB: Fetched primary data is valid. [2025-11-04T19:19:28.631177] [DEBUG] [LOCAL ] HBB: Saving primary data. [2025-11-04T19:19:28.647095] [DEBUG] [LOCAL ] HBB: Trimmed 9002 records to 5680 (keeping 200 days of data) [2025-11-04T19:19:28.858769] [DEBUG] [LOCAL ] HBB: Primary data saved (5680 records). [2025-11-04T19:19:28.861321] [INFO ] [MAIN ] HBB: Full history saved. [2025-11-04T19:19:28.861433] [INFO ] [MAIN ] Update Progress: 995/2367 symbols completed. HBB took 29.3s. Estimated remaining: 3h 12m 15s. Rate: 7.1 symbols/min [2025-11-04T19:19:28.998439] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-09. [2025-11-04T19:19:29.050808] [INFO ] [REMOTE ] HCA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:19:29.120623] [INFO ] [MAIN ] HCC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:19:29.158306] [DEBUG] [LOCAL ] HCC: Loaded 44523 existing records from primary storage. [2025-11-04T19:19:29.158673] [INFO ] [MAIN ] HCC: Loaded 44523 existing records. [2025-11-04T19:19:29.336460] [WARN ] [MAIN ] HCC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:19:29.336722] [INFO ] [MAIN ] HCC: Wiping existing primary data before full refresh. [2025-11-04T19:19:29.336786] [INFO ] [LOCAL ] HCC: Starting deletion of primary data. [2025-11-04T19:19:29.337512] [DEBUG] [LOCAL ] HCC: Deleted primary data file. [2025-11-04T19:19:29.337569] [INFO ] [MAIN ] HCC: Starting full primary refresh. [2025-11-04T19:19:29.698735] [INFO ] [REMOTE ] HCC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:19:29.767102] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-01. [2025-11-04T19:19:29.837325] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-11-04T19:19:30.271666] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-11-04T19:19:30.567093] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-01. [2025-11-04T19:19:30.793776] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-11-04T19:19:31.367071] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-10. [2025-11-04T19:19:32.030861] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-11-04T19:19:32.167065] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-07. [2025-11-04T19:19:32.680412] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T19:19:32.967056] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-02. [2025-11-04T19:19:33.568286] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T19:19:33.767040] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-08. [2025-11-04T19:19:34.119560] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T19:19:34.567037] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-11. [2025-11-04T19:19:35.010581] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T19:19:35.367095] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-03. [2025-11-04T19:19:35.618711] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T19:19:35.620806] [INFO ] [REMOTE ] HBI: Full fetch complete (76912 records). [2025-11-04T19:19:35.620914] [DEBUG] [VALIDATOR ] HBI: Validating primary data. [2025-11-04T19:19:36.003353] [DEBUG] [VALIDATOR ] HBI: All expected trading days are present. [2025-11-04T19:19:36.003432] [DEBUG] [VALIDATOR ] HBI: Primary data validation passed. [2025-11-04T19:19:36.004336] [INFO ] [MAIN ] HBI: Fetched primary data is valid. [2025-11-04T19:19:36.004400] [DEBUG] [LOCAL ] HBI: Saving primary data. [2025-11-04T19:19:36.005867] [DEBUG] [LOCAL ] HBI: Trimmed 76912 records to 52843 (keeping 200 days of data) [2025-11-04T19:19:36.190811] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-09. [2025-11-04T19:19:36.293797] [DEBUG] [LOCAL ] HBI: Primary data saved (52843 records). [2025-11-04T19:19:36.293919] [INFO ] [MAIN ] HBI: Full history saved. [2025-11-04T19:19:36.293993] [INFO ] [MAIN ] Update Progress: 996/2367 symbols completed. HBI took 35.0s. Estimated remaining: 3h 12m 5s. Rate: 7.1 symbols/min [2025-11-04T19:19:36.459993] [INFO ] [MAIN ] HCI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:19:36.473674] [DEBUG] [LOCAL ] HCI: Loaded 13872 existing records from primary storage. [2025-11-04T19:19:36.473881] [INFO ] [MAIN ] HCI: Loaded 13872 existing records. [2025-11-04T19:19:36.653683] [WARN ] [MAIN ] HCI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:19:36.654035] [INFO ] [MAIN ] HCI: Wiping existing primary data before full refresh. [2025-11-04T19:19:36.654113] [INFO ] [LOCAL ] HCI: Starting deletion of primary data. [2025-11-04T19:19:36.654585] [DEBUG] [LOCAL ] HCI: Deleted primary data file. [2025-11-04T19:19:36.654685] [INFO ] [MAIN ] HCI: Starting full primary refresh. [2025-11-04T19:19:36.972517] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-11-04T19:19:36.977727] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-02. [2025-11-04T19:19:37.045483] [INFO ] [REMOTE ] HCI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:19:37.483537] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T19:19:37.767039] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-04. [2025-11-04T19:19:38.220673] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T19:19:38.567037] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-10. [2025-11-04T19:19:39.214130] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T19:19:39.367020] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-05. [2025-11-04T19:19:40.156764] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T19:19:40.168508] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-03. [2025-11-04T19:19:40.780105] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T19:19:40.967125] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-11. [2025-11-04T19:19:41.333374] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-11-04T19:19:41.767088] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-06. [2025-11-04T19:19:42.201637] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-11 (1.23 seconds). [2025-11-04T19:19:42.203800] [INFO ] [REMOTE ] HBM: Full fetch complete (76859 records). [2025-11-04T19:19:42.203895] [DEBUG] [VALIDATOR ] HBM: Validating primary data. [2025-11-04T19:19:42.570169] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-04. [2025-11-04T19:19:42.611030] [DEBUG] [VALIDATOR ] HBM: All expected trading days are present. [2025-11-04T19:19:42.611100] [DEBUG] [VALIDATOR ] HBM: Primary data validation passed. [2025-11-04T19:19:42.612072] [INFO ] [MAIN ] HBM: Fetched primary data is valid. [2025-11-04T19:19:42.612137] [DEBUG] [LOCAL ] HBM: Saving primary data. [2025-11-04T19:19:42.613723] [DEBUG] [LOCAL ] HBM: Trimmed 76859 records to 52990 (keeping 200 days of data) [2025-11-04T19:19:42.931423] [DEBUG] [LOCAL ] HBM: Primary data saved (52990 records). [2025-11-04T19:19:42.931584] [INFO ] [MAIN ] HBM: Full history saved. [2025-11-04T19:19:42.931769] [INFO ] [MAIN ] Update Progress: 997/2367 symbols completed. HBM took 33.8s. Estimated remaining: 3h 11m 55s. Rate: 7.1 symbols/min [2025-11-04T19:19:42.966343] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-11-04T19:19:43.367086] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-07. [2025-11-04T19:19:43.788320] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T19:19:44.167096] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-05. [2025-11-04T19:19:44.650037] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T19:19:44.967069] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-01. [2025-11-04T19:19:45.136985] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-11-04T19:19:45.461836] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-01 (0.49 seconds). [2025-11-04T19:19:45.767036] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-08. [2025-11-04T19:19:46.567051] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-06. [2025-11-04T19:19:46.969924] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T19:19:47.367031] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-02. [2025-11-04T19:19:47.759760] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-11-04T19:19:47.954878] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-11-04T19:19:48.167079] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-07. [2025-11-04T19:19:48.967040] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-09. [2025-11-04T19:19:49.532842] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T19:19:49.767061] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-03. [2025-11-04T19:19:50.201406] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T19:19:50.309357] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-11-04T19:19:50.567037] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-04. [2025-11-04T19:19:51.199988] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-11-04T19:19:51.367033] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-05. [2025-11-04T19:19:52.007589] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T19:19:52.167038] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-08. [2025-11-04T19:19:52.967052] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-10. [2025-11-04T19:19:53.380666] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T19:19:53.767056] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-06. [2025-11-04T19:19:54.312873] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T19:19:54.337335] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-11-04T19:19:54.567060] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-09. [2025-11-04T19:19:55.367039] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-07. [2025-11-04T19:19:55.712744] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T19:19:55.895128] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-11-04T19:19:56.167123] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-08. [2025-11-04T19:19:56.837553] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T19:19:56.967037] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-10. [2025-11-04T19:19:57.767035] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-09. [2025-11-04T19:19:58.141437] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-11-04T19:19:58.374627] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-11-04T19:19:58.567046] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-10. [2025-11-04T19:19:59.244231] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-11-04T19:19:59.367024] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-11. [2025-11-04T19:19:59.980758] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-11 (0.61 seconds). [2025-11-04T19:19:59.981959] [INFO ] [REMOTE ] HCI: Full fetch complete (19704 records). [2025-11-04T19:19:59.982030] [DEBUG] [VALIDATOR ] HCI: Validating primary data. [2025-11-04T19:20:00.190394] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-11. [2025-11-04T19:20:00.394804] [DEBUG] [VALIDATOR ] HCI: All expected trading days are present. [2025-11-04T19:20:00.394905] [DEBUG] [VALIDATOR ] HCI: Primary data validation passed. [2025-11-04T19:20:00.395429] [INFO ] [MAIN ] HCI: Fetched primary data is valid. [2025-11-04T19:20:00.395475] [DEBUG] [LOCAL ] HCI: Saving primary data. [2025-11-04T19:20:00.396091] [DEBUG] [LOCAL ] HCI: Trimmed 19704 records to 13982 (keeping 200 days of data) [2025-11-04T19:20:00.483922] [DEBUG] [LOCAL ] HCI: Primary data saved (13982 records). [2025-11-04T19:20:00.484052] [INFO ] [MAIN ] HCI: Full history saved. [2025-11-04T19:20:00.484126] [INFO ] [MAIN ] Update Progress: 998/2367 symbols completed. HCI took 24.2s. Estimated remaining: 3h 11m 59s. Rate: 7.1 symbols/min [2025-11-04T19:20:00.967086] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-11. [2025-11-04T19:20:01.978303] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-11 (1.79 seconds). [2025-11-04T19:20:01.980301] [INFO ] [REMOTE ] HCC: Full fetch complete (64639 records). [2025-11-04T19:20:01.980422] [DEBUG] [VALIDATOR ] HCC: Validating primary data. [2025-11-04T19:20:02.447732] [DEBUG] [VALIDATOR ] HCC: All expected trading days are present. [2025-11-04T19:20:02.447609] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-11 (1.48 seconds). [2025-11-04T19:20:02.449364] [DEBUG] [VALIDATOR ] HCC: Primary data validation passed. [2025-11-04T19:20:02.450886] [INFO ] [MAIN ] HCC: Fetched primary data is valid. [2025-11-04T19:20:02.451121] [DEBUG] [LOCAL ] HCC: Saving primary data. [2025-11-04T19:20:02.451455] [INFO ] [REMOTE ] HCA: Full fetch complete (68962 records). [2025-11-04T19:20:02.451547] [DEBUG] [VALIDATOR ] HCA: Validating primary data. [2025-11-04T19:20:02.514784] [DEBUG] [LOCAL ] HCC: Trimmed 64639 records to 44469 (keeping 200 days of data) [2025-11-04T19:20:03.101348] [DEBUG] [VALIDATOR ] HCA: All expected trading days are present. [2025-11-04T19:20:03.127489] [DEBUG] [VALIDATOR ] HCA: Primary data validation passed. [2025-11-04T19:20:03.129361] [INFO ] [MAIN ] HCA: Fetched primary data is valid. [2025-11-04T19:20:03.129766] [DEBUG] [LOCAL ] HCA: Saving primary data. [2025-11-04T19:20:03.133810] [DEBUG] [LOCAL ] HCA: Trimmed 68962 records to 46891 (keeping 200 days of data) [2025-11-04T19:20:03.318263] [DEBUG] [LOCAL ] HCC: Primary data saved (44469 records). [2025-11-04T19:20:03.318677] [INFO ] [MAIN ] HCC: Full history saved. [2025-11-04T19:20:03.318795] [INFO ] [MAIN ] Update Progress: 999/2367 symbols completed. HCC took 34.5s. Estimated remaining: 3h 11m 43s. Rate: 7.1 symbols/min [2025-11-04T19:20:03.517225] [DEBUG] [LOCAL ] HCA: Primary data saved (46891 records). [2025-11-04T19:20:03.517357] [INFO ] [MAIN ] HCA: Full history saved. [2025-11-04T19:20:03.517440] [INFO ] [MAIN ] Update Progress: 1000/2367 symbols completed. HCA took 35.8s. Estimated remaining: 3h 11m 23s. Rate: 7.1 symbols/min [2025-11-04T19:20:04.250072] [INFO ] [MAIN ] HDB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:20:04.233322] [INFO ] [MAIN ] HCXY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:20:04.230197] [INFO ] [MAIN ] HD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:20:04.250916] [DEBUG] [LOCAL ] HCXY: No existing data file found in primary storage. [2025-11-04T19:20:04.251164] [WARN ] [MAIN ] HCXY: No existing data found, performing full refresh instead. [2025-11-04T19:20:04.252363] [INFO ] [MAIN ] HCXY: Wiping existing primary data before full refresh. [2025-11-04T19:20:04.252434] [INFO ] [LOCAL ] HCXY: Starting deletion of primary data. [2025-11-04T19:20:04.252482] [DEBUG] [LOCAL ] HCXY: No primary data file found to delete. [2025-11-04T19:20:04.252514] [INFO ] [MAIN ] HCXY: Starting full primary refresh. [2025-11-04T19:20:04.228343] [INFO ] [MAIN ] HE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:20:04.637557] [DEBUG] [LOCAL ] HD: Loaded 53429 existing records from primary storage. [2025-11-04T19:20:04.677219] [DEBUG] [LOCAL ] HDB: Loaded 51874 existing records from primary storage. [2025-11-04T19:20:04.681896] [INFO ] [REMOTE ] HCXY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:20:04.694974] [INFO ] [MAIN ] HDB: Loaded 51874 existing records. [2025-11-04T19:20:04.694568] [INFO ] [MAIN ] HD: Loaded 53429 existing records. [2025-11-04T19:20:04.727482] [DEBUG] [LOCAL ] HE: Loaded 48484 existing records from primary storage. [2025-11-04T19:20:04.717406] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-01. [2025-11-04T19:20:04.756762] [INFO ] [MAIN ] HE: Loaded 48484 existing records. [2025-11-04T19:20:05.057077] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-11-04T19:20:05.202470] [WARN ] [MAIN ] HD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:20:05.195303] [WARN ] [MAIN ] HDB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:20:05.126498] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-02. [2025-11-04T19:20:05.244975] [INFO ] [MAIN ] HDB: Wiping existing primary data before full refresh. [2025-11-04T19:20:05.250129] [INFO ] [LOCAL ] HDB: Starting deletion of primary data. [2025-11-04T19:20:05.242058] [INFO ] [MAIN ] HD: Wiping existing primary data before full refresh. [2025-11-04T19:20:05.260522] [DEBUG] [LOCAL ] HDB: Deleted primary data file. [2025-11-04T19:20:05.253595] [INFO ] [LOCAL ] HD: Starting deletion of primary data. [2025-11-04T19:20:05.270346] [INFO ] [MAIN ] HDB: Starting full primary refresh. [2025-11-04T19:20:05.297025] [DEBUG] [LOCAL ] HD: Deleted primary data file. [2025-11-04T19:20:05.296941] [WARN ] [MAIN ] HE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:20:05.312879] [INFO ] [MAIN ] HD: Starting full primary refresh. [2025-11-04T19:20:05.375003] [INFO ] [MAIN ] HE: Wiping existing primary data before full refresh. [2025-11-04T19:20:05.411141] [INFO ] [LOCAL ] HE: Starting deletion of primary data. [2025-11-04T19:20:05.429770] [DEBUG] [LOCAL ] HE: Deleted primary data file. [2025-11-04T19:20:05.539851] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-11-04T19:20:05.469572] [INFO ] [MAIN ] HE: Starting full primary refresh. [2025-11-04T19:20:05.573141] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-03. [2025-11-04T19:20:06.043765] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-11-04T19:20:06.111291] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-04. [2025-11-04T19:20:06.209570] [INFO ] [REMOTE ] HDB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:20:06.234244] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-01. [2025-11-04T19:20:06.377073] [INFO ] [REMOTE ] HD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:20:06.397260] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-01. [2025-11-04T19:20:06.481437] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-11-04T19:20:06.481328] [INFO ] [REMOTE ] HE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:20:06.567011] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-01. [2025-11-04T19:20:07.367035] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-05. [2025-11-04T19:20:07.579270] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-05 (0.21 seconds). [2025-11-04T19:20:08.052829] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-11-04T19:20:08.125998] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-11-04T19:20:08.166994] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-02. [2025-11-04T19:20:08.664576] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-01 (2.43 seconds). [2025-11-04T19:20:08.967105] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-02. [2025-11-04T19:20:09.767011] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-06. [2025-11-04T19:20:09.879855] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-11-04T19:20:09.951506] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-06 (0.18 seconds). [2025-11-04T19:20:10.165212] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T19:20:10.567087] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-03. [2025-11-04T19:20:11.367097] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-07. [2025-11-04T19:20:11.618194] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-07 (0.25 seconds). [2025-11-04T19:20:12.167065] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-08. [2025-11-04T19:20:12.370082] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-11-04T19:20:12.393530] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-08 (0.23 seconds). [2025-11-04T19:20:12.967035] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-03. [2025-11-04T19:20:13.767061] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-04. [2025-11-04T19:20:14.285850] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T19:20:14.567013] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-02. [2025-11-04T19:20:15.367028] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-09. [2025-11-04T19:20:16.167023] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-04. [2025-11-04T19:20:17.187052] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-11-04T19:20:17.188176] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-10. [2025-11-04T19:20:17.193630] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-04 (3.43 seconds). [2025-11-04T19:20:17.419808] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-10 (0.23 seconds). [2025-11-04T19:20:17.642148] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-11-04T19:20:17.767022] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-11. [2025-11-04T19:20:17.909187] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-02 (3.34 seconds). [2025-11-04T19:20:18.087353] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-11 (0.32 seconds). [2025-11-04T19:20:18.087997] [INFO ] [REMOTE ] HCXY: Full fetch complete (535 records). [2025-11-04T19:20:18.088062] [DEBUG] [VALIDATOR ] HCXY: Validating primary data. [2025-11-04T19:20:18.267973] [WARN ] [VALIDATOR ] HCXY: Validation failed - insufficient historical data. Have 162 unique days, need 200 days [2025-11-04T19:20:18.268196] [FATAL] [MAIN ] HCXY: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:20:18.268244] [WARN ] [MAIN ] HCXY: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:20:18.268287] [INFO ] [LOCAL ] HCXY: Starting deletion of primary data. [2025-11-04T19:20:18.268368] [DEBUG] [LOCAL ] HCXY: No primary data file found to delete. [2025-11-04T19:20:18.268424] [INFO ] [LOCAL ] HCXY: Starting deletion of derivative data. [2025-11-04T19:20:18.268467] [DEBUG] [LOCAL ] HCXY: No derivative data file found to delete. [2025-11-04T19:20:18.268527] [INFO ] [MAIN ] Update Progress: 1001/2367 symbols completed. HCXY took 14.8s. Estimated remaining: 3h 11m 23s. Rate: 7.1 symbols/min [2025-11-04T19:20:18.460582] [INFO ] [MAIN ] HEI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:20:18.489055] [DEBUG] [LOCAL ] HEI: Loaded 29602 existing records from primary storage. [2025-11-04T19:20:18.489339] [INFO ] [MAIN ] HEI: Loaded 29602 existing records. [2025-11-04T19:20:18.572209] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-03. [2025-11-04T19:20:18.682986] [WARN ] [MAIN ] HEI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:20:18.683207] [INFO ] [MAIN ] HEI: Wiping existing primary data before full refresh. [2025-11-04T19:20:18.683256] [INFO ] [LOCAL ] HEI: Starting deletion of primary data. [2025-11-04T19:20:18.683829] [DEBUG] [LOCAL ] HEI: Deleted primary data file. [2025-11-04T19:20:18.683887] [INFO ] [MAIN ] HEI: Starting full primary refresh. [2025-11-04T19:20:19.081550] [INFO ] [REMOTE ] HEI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:20:19.367087] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-05. [2025-11-04T19:20:19.890798] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T19:20:20.167067] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-01. [2025-11-04T19:20:20.955815] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-11-04T19:20:20.967102] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-06. [2025-11-04T19:20:21.429017] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T19:20:21.767063] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-05. [2025-11-04T19:20:22.319173] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-11-04T19:20:22.567073] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-07. [2025-11-04T19:20:23.064698] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T19:20:23.367106] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-02. [2025-11-04T19:20:24.179149] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-04. [2025-11-04T19:20:24.185194] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-11-04T19:20:24.485152] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-11-04T19:20:24.967019] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-08. [2025-11-04T19:20:25.721871] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-11-04T19:20:25.767016] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-06. [2025-11-04T19:20:26.567050] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-03. [2025-11-04T19:20:26.701688] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-11-04T19:20:27.035460] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T19:20:27.367087] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-05. [2025-11-04T19:20:27.532384] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-11-04T19:20:28.167137] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-04. [2025-11-04T19:20:28.765414] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T19:20:28.967064] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-06. [2025-11-04T19:20:29.296553] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T19:20:29.767041] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-05. [2025-11-04T19:20:30.305985] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T19:20:30.567067] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-07. [2025-11-04T19:20:30.841204] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T19:20:31.367043] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-06. [2025-11-04T19:20:31.938028] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T19:20:32.167033] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-08. [2025-11-04T19:20:32.238374] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T19:20:32.967158] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-07. [2025-11-04T19:20:33.453078] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T19:20:33.767090] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-07. [2025-11-04T19:20:34.567083] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-09. [2025-11-04T19:20:34.812940] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-11-04T19:20:35.178041] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T19:20:35.367144] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-08. [2025-11-04T19:20:35.768198] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T19:20:36.167075] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-08. [2025-11-04T19:20:36.257088] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T19:20:36.967064] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-09. [2025-11-04T19:20:37.378812] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T19:20:37.767049] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-10. [2025-11-04T19:20:38.567085] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-09. [2025-11-04T19:20:38.641496] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-11-04T19:20:39.172587] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T19:20:39.323044] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-11-04T19:20:39.367050] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-09. [2025-11-04T19:20:40.167072] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-10. [2025-11-04T19:20:40.764754] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T19:20:40.967040] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-10. [2025-11-04T19:20:41.767073] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-10. [2025-11-04T19:20:42.075031] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-11-04T19:20:42.283940] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T19:20:42.567050] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-11. [2025-11-04T19:20:42.790783] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T19:20:43.367047] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-11. [2025-11-04T19:20:43.687929] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T19:20:43.689970] [INFO ] [REMOTE ] HDB: Full fetch complete (75686 records). [2025-11-04T19:20:43.690114] [DEBUG] [VALIDATOR ] HDB: Validating primary data. [2025-11-04T19:20:43.817823] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-11 (0.45 seconds). [2025-11-04T19:20:43.860205] [INFO ] [REMOTE ] HD: Full fetch complete (77513 records). [2025-11-04T19:20:43.898237] [DEBUG] [VALIDATOR ] HD: Validating primary data. [2025-11-04T19:20:44.167983] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-11. [2025-11-04T19:20:44.273022] [DEBUG] [VALIDATOR ] HDB: All expected trading days are present. [2025-11-04T19:20:44.292753] [DEBUG] [VALIDATOR ] HDB: Primary data validation passed. [2025-11-04T19:20:44.293869] [INFO ] [MAIN ] HDB: Fetched primary data is valid. [2025-11-04T19:20:44.297328] [DEBUG] [LOCAL ] HDB: Saving primary data. [2025-11-04T19:20:44.420479] [DEBUG] [LOCAL ] HDB: Trimmed 75686 records to 51871 (keeping 200 days of data) [2025-11-04T19:20:44.632012] [DEBUG] [VALIDATOR ] HD: All expected trading days are present. [2025-11-04T19:20:44.635805] [DEBUG] [VALIDATOR ] HD: Primary data validation passed. [2025-11-04T19:20:44.642550] [INFO ] [MAIN ] HD: Fetched primary data is valid. [2025-11-04T19:20:44.648181] [DEBUG] [LOCAL ] HD: Saving primary data. [2025-11-04T19:20:44.663372] [DEBUG] [LOCAL ] HD: Trimmed 77513 records to 53433 (keeping 200 days of data) [2025-11-04T19:20:44.970034] [DEBUG] [LOCAL ] HDB: Primary data saved (51871 records). [2025-11-04T19:20:44.970209] [INFO ] [MAIN ] HDB: Full history saved. [2025-11-04T19:20:44.968868] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-11. [2025-11-04T19:20:44.970342] [INFO ] [MAIN ] Update Progress: 1002/2367 symbols completed. HDB took 41.4s. Estimated remaining: 3h 11m 40s. Rate: 7.1 symbols/min [2025-11-04T19:20:45.316830] [INFO ] [MAIN ] HEQ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:20:45.375626] [DEBUG] [LOCAL ] HEQ: Loaded 6659 existing records from primary storage. [2025-11-04T19:20:45.376184] [INFO ] [MAIN ] HEQ: Loaded 6659 existing records. [2025-11-04T19:20:45.375798] [DEBUG] [LOCAL ] HD: Primary data saved (53433 records). [2025-11-04T19:20:45.398284] [INFO ] [MAIN ] HD: Full history saved. [2025-11-04T19:20:45.411800] [INFO ] [MAIN ] Update Progress: 1003/2367 symbols completed. HD took 41.9s. Estimated remaining: 3h 11m 21s. Rate: 7.1 symbols/min [2025-11-04T19:20:45.510616] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-11 (1.34 seconds). [2025-11-04T19:20:45.605879] [INFO ] [REMOTE ] HE: Full fetch complete (71735 records). [2025-11-04T19:20:45.627701] [DEBUG] [VALIDATOR ] HE: Validating primary data. [2025-11-04T19:20:45.730397] [INFO ] [MAIN ] HESM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:20:45.853451] [WARN ] [MAIN ] HEQ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:20:45.871075] [INFO ] [MAIN ] HEQ: Wiping existing primary data before full refresh. [2025-11-04T19:20:45.871284] [INFO ] [LOCAL ] HEQ: Starting deletion of primary data. [2025-11-04T19:20:45.882606] [DEBUG] [LOCAL ] HEQ: Deleted primary data file. [2025-11-04T19:20:45.929685] [INFO ] [MAIN ] HEQ: Starting full primary refresh. [2025-11-04T19:20:46.213839] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T19:20:46.279067] [INFO ] [REMOTE ] HEI: Full fetch complete (46168 records). [2025-11-04T19:20:46.336699] [DEBUG] [VALIDATOR ] HEI: Validating primary data. [2025-11-04T19:20:46.430035] [DEBUG] [LOCAL ] HESM: Loaded 48677 existing records from primary storage. [2025-11-04T19:20:46.445512] [DEBUG] [VALIDATOR ] HE: All expected trading days are present. [2025-11-04T19:20:46.463523] [DEBUG] [VALIDATOR ] HE: Primary data validation passed. [2025-11-04T19:20:46.475246] [INFO ] [MAIN ] HESM: Loaded 48677 existing records. [2025-11-04T19:20:46.476377] [INFO ] [MAIN ] HE: Fetched primary data is valid. [2025-11-04T19:20:46.535313] [DEBUG] [LOCAL ] HE: Saving primary data. [2025-11-04T19:20:46.819501] [DEBUG] [LOCAL ] HE: Trimmed 71735 records to 48471 (keeping 200 days of data) [2025-11-04T19:20:46.852552] [INFO ] [REMOTE ] HEQ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:20:46.884213] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-01. [2025-11-04T19:20:47.054190] [WARN ] [MAIN ] HESM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:20:47.228040] [INFO ] [MAIN ] HESM: Wiping existing primary data before full refresh. [2025-11-04T19:20:47.274048] [INFO ] [LOCAL ] HESM: Starting deletion of primary data. [2025-11-04T19:20:47.372978] [DEBUG] [LOCAL ] HESM: Deleted primary data file. [2025-11-04T19:20:47.431125] [INFO ] [MAIN ] HESM: Starting full primary refresh. [2025-11-04T19:20:47.457877] [DEBUG] [LOCAL ] HE: Primary data saved (48471 records). [2025-11-04T19:20:47.481090] [INFO ] [MAIN ] HE: Full history saved. [2025-11-04T19:20:47.481345] [INFO ] [MAIN ] Update Progress: 1004/2367 symbols completed. HE took 43.9s. Estimated remaining: 3h 11m 4s. Rate: 7.1 symbols/min [2025-11-04T19:20:47.796966] [DEBUG] [VALIDATOR ] HEI: All expected trading days are present. [2025-11-04T19:20:47.805432] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-11-04T19:20:47.805680] [DEBUG] [VALIDATOR ] HEI: Primary data validation passed. [2025-11-04T19:20:47.823729] [INFO ] [MAIN ] HEI: Fetched primary data is valid. [2025-11-04T19:20:47.897881] [DEBUG] [LOCAL ] HEI: Saving primary data. [2025-11-04T19:20:47.944943] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-02. [2025-11-04T19:20:47.963568] [INFO ] [MAIN ] HF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:20:47.982221] [DEBUG] [LOCAL ] HEI: Trimmed 46168 records to 29635 (keeping 200 days of data) [2025-11-04T19:20:48.016572] [DEBUG] [LOCAL ] HF: No existing data file found in primary storage. [2025-11-04T19:20:48.030396] [WARN ] [MAIN ] HF: No existing data found, performing full refresh instead. [2025-11-04T19:20:48.125518] [INFO ] [MAIN ] HF: Wiping existing primary data before full refresh. [2025-11-04T19:20:48.125687] [INFO ] [LOCAL ] HF: Starting deletion of primary data. [2025-11-04T19:20:48.125807] [DEBUG] [LOCAL ] HF: No primary data file found to delete. [2025-11-04T19:20:48.142509] [INFO ] [MAIN ] HF: Starting full primary refresh. [2025-11-04T19:20:48.429711] [DEBUG] [LOCAL ] HEI: Primary data saved (29635 records). [2025-11-04T19:20:48.458618] [INFO ] [MAIN ] HEI: Full history saved. [2025-11-04T19:20:48.458947] [INFO ] [MAIN ] Update Progress: 1005/2367 symbols completed. HEI took 30.2s. Estimated remaining: 3h 10m 45s. Rate: 7.1 symbols/min [2025-11-04T19:20:48.557162] [INFO ] [REMOTE ] HESM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:20:48.589154] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-01. [2025-11-04T19:20:48.654066] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T19:20:48.667273] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-03. [2025-11-04T19:20:48.861551] [INFO ] [MAIN ] HFND: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:20:48.889588] [DEBUG] [LOCAL ] HFND: No existing data file found in primary storage. [2025-11-04T19:20:48.900445] [WARN ] [MAIN ] HFND: No existing data found, performing full refresh instead. [2025-11-04T19:20:48.918142] [INFO ] [MAIN ] HFND: Wiping existing primary data before full refresh. [2025-11-04T19:20:48.934974] [INFO ] [LOCAL ] HFND: Starting deletion of primary data. [2025-11-04T19:20:48.943854] [DEBUG] [LOCAL ] HFND: No primary data file found to delete. [2025-11-04T19:20:48.947282] [INFO ] [MAIN ] HFND: Starting full primary refresh. [2025-11-04T19:20:49.205459] [INFO ] [REMOTE ] HF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:20:49.239173] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-01. [2025-11-04T19:20:49.245515] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-11-04T19:20:49.472571] [INFO ] [REMOTE ] HFND: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:20:49.565174] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-11-04T19:20:49.767002] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-02. [2025-11-04T19:20:49.991567] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-02 (0.22 seconds). [2025-11-04T19:20:49.993624] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T19:20:50.567049] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-04. [2025-11-04T19:20:50.965866] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-11-04T19:20:51.367039] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-05. [2025-11-04T19:20:51.761393] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-11-04T19:20:52.167028] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-01. [2025-11-04T19:20:52.527229] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-01 (0.36 seconds). [2025-11-04T19:20:52.967044] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-03. [2025-11-04T19:20:53.149970] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-03 (0.18 seconds). [2025-11-04T19:20:53.767094] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-02. [2025-11-04T19:20:54.009081] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-02 (0.24 seconds). [2025-11-04T19:20:54.567094] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-03. [2025-11-04T19:20:54.818419] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-03 (0.25 seconds). [2025-11-04T19:20:55.367060] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-02. [2025-11-04T19:20:56.167104] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-04. [2025-11-04T19:20:56.441676] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-04 (0.27 seconds). [2025-11-04T19:20:56.637890] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T19:20:56.967038] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-04. [2025-11-04T19:20:57.166325] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-04 (0.20 seconds). [2025-11-04T19:20:57.767077] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-05. [2025-11-04T19:20:57.969850] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-05 (0.20 seconds). [2025-11-04T19:20:58.567084] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-03. [2025-11-04T19:20:59.367028] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-06. [2025-11-04T19:20:59.761723] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-11-04T19:20:59.907987] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T19:21:00.167069] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-07. [2025-11-04T19:21:00.711193] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-11-04T19:21:00.967057] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-08. [2025-11-04T19:21:01.362721] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-11-04T19:21:01.767074] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-04. [2025-11-04T19:21:02.567069] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-05. [2025-11-04T19:21:02.820055] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-05 (0.25 seconds). [2025-11-04T19:21:02.910694] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T19:21:03.367043] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-06. [2025-11-04T19:21:03.574715] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-06 (0.21 seconds). [2025-11-04T19:21:04.167039] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-09. [2025-11-04T19:21:04.694855] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-11-04T19:21:04.967050] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-07. [2025-11-04T19:21:05.269719] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-07 (0.30 seconds). [2025-11-04T19:21:05.767080] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-10. [2025-11-04T19:21:06.291898] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-11-04T19:21:06.567079] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-11. [2025-11-04T19:21:07.169752] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-11 (0.60 seconds). [2025-11-04T19:21:07.170686] [INFO ] [REMOTE ] HEQ: Full fetch complete (9972 records). [2025-11-04T19:21:07.170755] [DEBUG] [VALIDATOR ] HEQ: Validating primary data. [2025-11-04T19:21:07.375068] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-08. [2025-11-04T19:21:07.548603] [DEBUG] [VALIDATOR ] HEQ: All expected trading days are present. [2025-11-04T19:21:07.548707] [DEBUG] [VALIDATOR ] HEQ: Primary data validation passed. [2025-11-04T19:21:07.548974] [INFO ] [MAIN ] HEQ: Fetched primary data is valid. [2025-11-04T19:21:07.549015] [DEBUG] [LOCAL ] HEQ: Saving primary data. [2025-11-04T19:21:07.549504] [DEBUG] [LOCAL ] HEQ: Trimmed 9972 records to 6722 (keeping 200 days of data) [2025-11-04T19:21:07.592015] [DEBUG] [LOCAL ] HEQ: Primary data saved (6722 records). [2025-11-04T19:21:07.592148] [INFO ] [MAIN ] HEQ: Full history saved. [2025-11-04T19:21:07.592229] [INFO ] [MAIN ] Update Progress: 1006/2367 symbols completed. HEQ took 22.6s. Estimated remaining: 3h 10m 51s. Rate: 7.1 symbols/min [2025-11-04T19:21:07.780749] [INFO ] [MAIN ] HFRO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:21:07.777273] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-11-04T19:21:07.797585] [DEBUG] [LOCAL ] HFRO: Loaded 15642 existing records from primary storage. [2025-11-04T19:21:07.797851] [INFO ] [MAIN ] HFRO: Loaded 15642 existing records. [2025-11-04T19:21:07.972297] [WARN ] [MAIN ] HFRO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:21:07.972533] [INFO ] [MAIN ] HFRO: Wiping existing primary data before full refresh. [2025-11-04T19:21:07.972580] [INFO ] [LOCAL ] HFRO: Starting deletion of primary data. [2025-11-04T19:21:07.973127] [DEBUG] [LOCAL ] HFRO: Deleted primary data file. [2025-11-04T19:21:07.973185] [INFO ] [MAIN ] HFRO: Starting full primary refresh. [2025-11-04T19:21:08.172154] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-05. [2025-11-04T19:21:08.340682] [INFO ] [REMOTE ] HFRO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:21:08.967095] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-06. [2025-11-04T19:21:09.166357] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-06 (0.20 seconds). [2025-11-04T19:21:09.532162] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T19:21:09.767044] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-01. [2025-11-04T19:21:10.567048] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-06. [2025-11-04T19:21:10.596982] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-11-04T19:21:11.367031] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-09. [2025-11-04T19:21:11.684166] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-11-04T19:21:11.720691] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T19:21:12.167080] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-10. [2025-11-04T19:21:12.438949] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-10 (0.27 seconds). [2025-11-04T19:21:12.967069] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-07. [2025-11-04T19:21:13.161604] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-07 (0.19 seconds). [2025-11-04T19:21:13.767086] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-08. [2025-11-04T19:21:14.028708] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-08 (0.26 seconds). [2025-11-04T19:21:14.567085] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-09. [2025-11-04T19:21:14.754969] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-09 (0.19 seconds). [2025-11-04T19:21:15.367058] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-11. [2025-11-04T19:21:15.669166] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-11 (0.30 seconds). [2025-11-04T19:21:15.669830] [INFO ] [REMOTE ] HFND: Full fetch complete (1818 records). [2025-11-04T19:21:15.669898] [DEBUG] [VALIDATOR ] HFND: Validating primary data. [2025-11-04T19:21:15.852981] [WARN ] [VALIDATOR ] HFND: Validation failed - insufficient historical data. Have 196 unique days, need 200 days [2025-11-04T19:21:15.853191] [FATAL] [MAIN ] HFND: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:21:15.853264] [WARN ] [MAIN ] HFND: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:21:15.853336] [INFO ] [LOCAL ] HFND: Starting deletion of primary data. [2025-11-04T19:21:15.853401] [DEBUG] [LOCAL ] HFND: No primary data file found to delete. [2025-11-04T19:21:15.853436] [INFO ] [LOCAL ] HFND: Starting deletion of derivative data. [2025-11-04T19:21:15.853476] [DEBUG] [LOCAL ] HFND: No derivative data file found to delete. [2025-11-04T19:21:15.853533] [INFO ] [MAIN ] Update Progress: 1007/2367 symbols completed. HFND took 27.4s. Estimated remaining: 3h 10m 43s. Rate: 7.1 symbols/min [2025-11-04T19:21:16.025277] [INFO ] [MAIN ] HGER: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:21:16.043872] [DEBUG] [LOCAL ] HGER: Loaded 16238 existing records from primary storage. [2025-11-04T19:21:16.044147] [INFO ] [MAIN ] HGER: Loaded 16238 existing records. [2025-11-04T19:21:16.172128] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-10. [2025-11-04T19:21:16.229386] [WARN ] [MAIN ] HGER: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:21:16.229556] [INFO ] [MAIN ] HGER: Wiping existing primary data before full refresh. [2025-11-04T19:21:16.229600] [INFO ] [LOCAL ] HGER: Starting deletion of primary data. [2025-11-04T19:21:16.230019] [DEBUG] [LOCAL ] HGER: Deleted primary data file. [2025-11-04T19:21:16.230075] [INFO ] [MAIN ] HGER: Starting full primary refresh. [2025-11-04T19:21:16.514200] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-10 (0.34 seconds). [2025-11-04T19:21:16.591024] [INFO ] [REMOTE ] HGER: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:21:16.967095] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-07. [2025-11-04T19:21:17.767073] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-02. [2025-11-04T19:21:18.511208] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-11-04T19:21:18.567048] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-11. [2025-11-04T19:21:18.838732] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-11-04T19:21:18.863053] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-11 (0.30 seconds). [2025-11-04T19:21:18.863719] [INFO ] [REMOTE ] HF: Full fetch complete (416 records). [2025-11-04T19:21:18.863791] [DEBUG] [VALIDATOR ] HF: Validating primary data. [2025-11-04T19:21:19.060168] [WARN ] [VALIDATOR ] HF: Validation failed - insufficient historical data. Have 143 unique days, need 200 days [2025-11-04T19:21:19.060364] [FATAL] [MAIN ] HF: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:21:19.060434] [WARN ] [MAIN ] HF: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:21:19.060506] [INFO ] [LOCAL ] HF: Starting deletion of primary data. [2025-11-04T19:21:19.060569] [DEBUG] [LOCAL ] HF: No primary data file found to delete. [2025-11-04T19:21:19.060606] [INFO ] [LOCAL ] HF: Starting deletion of derivative data. [2025-11-04T19:21:19.060670] [DEBUG] [LOCAL ] HF: No derivative data file found to delete. [2025-11-04T19:21:19.060769] [INFO ] [MAIN ] Update Progress: 1008/2367 symbols completed. HF took 31.5s. Estimated remaining: 3h 10m 27s. Rate: 7.1 symbols/min [2025-11-04T19:21:19.225160] [INFO ] [MAIN ] HGLB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:21:19.234221] [DEBUG] [LOCAL ] HGLB: Loaded 9249 existing records from primary storage. [2025-11-04T19:21:19.234416] [INFO ] [MAIN ] HGLB: Loaded 9249 existing records. [2025-11-04T19:21:19.372697] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-08. [2025-11-04T19:21:19.414175] [WARN ] [MAIN ] HGLB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:21:19.414482] [INFO ] [MAIN ] HGLB: Wiping existing primary data before full refresh. [2025-11-04T19:21:19.414532] [INFO ] [LOCAL ] HGLB: Starting deletion of primary data. [2025-11-04T19:21:19.414851] [DEBUG] [LOCAL ] HGLB: Deleted primary data file. [2025-11-04T19:21:19.414909] [INFO ] [MAIN ] HGLB: Starting full primary refresh. [2025-11-04T19:21:19.869759] [INFO ] [REMOTE ] HGLB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:21:20.167138] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-01. [2025-11-04T19:21:20.738100] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-11-04T19:21:20.846499] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-11-04T19:21:20.967051] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-02. [2025-11-04T19:21:21.520178] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T19:21:21.767058] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-01. [2025-11-04T19:21:22.224863] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-11-04T19:21:22.567110] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-03. [2025-11-04T19:21:23.175836] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-11-04T19:21:23.367024] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-09. [2025-11-04T19:21:24.167036] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-03. [2025-11-04T19:21:24.737756] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-11-04T19:21:24.932051] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-11-04T19:21:24.967049] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-04. [2025-11-04T19:21:25.732194] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-11-04T19:21:25.767026] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-05. [2025-11-04T19:21:26.335106] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-11-04T19:21:26.567078] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-10. [2025-11-04T19:21:27.367042] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-06. [2025-11-04T19:21:28.033295] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T19:21:28.060278] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-11-04T19:21:28.167045] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-11. [2025-11-04T19:21:28.222786] [DEBUG] [REMOTE ] HESM: Error fetching primary data for 2025-11: 'API call returned an empty response.' [2025-11-04T19:21:28.223114] [WARN ] [REMOTE ] HESM: No data returned for month 2025-11 [2025-11-04T19:21:28.223203] [FATAL] [REMOTE ] HESM: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:21:28.223279] [FATAL] [MAIN ] HESM: Failed to fetch primary data. [2025-11-04T19:21:28.223314] [WARN ] [MAIN ] HESM: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:21:28.223378] [INFO ] [LOCAL ] HESM: Starting deletion of primary data. [2025-11-04T19:21:28.223439] [DEBUG] [LOCAL ] HESM: No primary data file found to delete. [2025-11-04T19:21:28.223473] [INFO ] [LOCAL ] HESM: Starting deletion of derivative data. [2025-11-04T19:21:28.223626] [DEBUG] [LOCAL ] HESM: Deleted derivative data file. [2025-11-04T19:21:28.223754] [INFO ] [MAIN ] Update Progress: 1009/2367 symbols completed. HESM took 42.8s. Estimated remaining: 3h 10m 20s. Rate: 7.1 symbols/min [2025-11-04T19:21:28.403516] [INFO ] [MAIN ] HGTY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:21:28.418816] [DEBUG] [LOCAL ] HGTY: Loaded 13533 existing records from primary storage. [2025-11-04T19:21:28.419056] [INFO ] [MAIN ] HGTY: Loaded 13533 existing records. [2025-11-04T19:21:28.604266] [WARN ] [MAIN ] HGTY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:21:28.604633] [INFO ] [MAIN ] HGTY: Wiping existing primary data before full refresh. [2025-11-04T19:21:28.604716] [INFO ] [LOCAL ] HGTY: Starting deletion of primary data. [2025-11-04T19:21:28.605154] [DEBUG] [LOCAL ] HGTY: Deleted primary data file. [2025-11-04T19:21:28.605210] [INFO ] [MAIN ] HGTY: Starting full primary refresh. [2025-11-04T19:21:28.967812] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-07. [2025-11-04T19:21:28.967744] [INFO ] [REMOTE ] HGTY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:21:29.628389] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-11-04T19:21:29.767087] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-04. [2025-11-04T19:21:30.223807] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-11-04T19:21:30.567056] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-05. [2025-11-04T19:21:30.995344] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T19:21:31.367023] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-02. [2025-11-04T19:21:31.751241] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T19:21:32.167070] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-08. [2025-11-04T19:21:32.881499] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T19:21:32.967088] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-09. [2025-11-04T19:21:33.671421] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-11-04T19:21:33.767034] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-01. [2025-11-04T19:21:34.301469] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T19:21:34.567074] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-10. [2025-11-04T19:21:35.135628] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-11-04T19:21:35.367030] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-11. [2025-11-04T19:21:35.865128] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T19:21:35.866060] [INFO ] [REMOTE ] HFRO: Full fetch complete (25659 records). [2025-11-04T19:21:35.866129] [DEBUG] [VALIDATOR ] HFRO: Validating primary data. [2025-11-04T19:21:36.172127] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-03. [2025-11-04T19:21:36.245546] [DEBUG] [VALIDATOR ] HFRO: All expected trading days are present. [2025-11-04T19:21:36.245622] [DEBUG] [VALIDATOR ] HFRO: Primary data validation passed. [2025-11-04T19:21:36.246225] [INFO ] [MAIN ] HFRO: Fetched primary data is valid. [2025-11-04T19:21:36.246290] [DEBUG] [LOCAL ] HFRO: Saving primary data. [2025-11-04T19:21:36.247020] [DEBUG] [LOCAL ] HFRO: Trimmed 25659 records to 15604 (keeping 200 days of data) [2025-11-04T19:21:36.339638] [DEBUG] [LOCAL ] HFRO: Primary data saved (15604 records). [2025-11-04T19:21:36.339818] [INFO ] [MAIN ] HFRO: Full history saved. [2025-11-04T19:21:36.339936] [INFO ] [MAIN ] Update Progress: 1010/2367 symbols completed. HFRO took 28.7s. Estimated remaining: 3h 10m 11s. Rate: 7.1 symbols/min [2025-11-04T19:21:36.533361] [INFO ] [MAIN ] HGV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:21:36.573919] [DEBUG] [LOCAL ] HGV: Loaded 45816 existing records from primary storage. [2025-11-04T19:21:36.574305] [INFO ] [MAIN ] HGV: Loaded 45816 existing records. [2025-11-04T19:21:36.766362] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-11-04T19:21:36.770080] [WARN ] [MAIN ] HGV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:21:36.770328] [INFO ] [MAIN ] HGV: Wiping existing primary data before full refresh. [2025-11-04T19:21:36.772271] [INFO ] [LOCAL ] HGV: Starting deletion of primary data. [2025-11-04T19:21:36.773726] [DEBUG] [LOCAL ] HGV: Deleted primary data file. [2025-11-04T19:21:36.773783] [INFO ] [MAIN ] HGV: Starting full primary refresh. [2025-11-04T19:21:36.972166] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-06. [2025-11-04T19:21:37.130823] [INFO ] [REMOTE ] HGV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:21:37.703726] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-11-04T19:21:37.767009] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-07. [2025-11-04T19:21:38.323733] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-11-04T19:21:38.567073] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-04. [2025-11-04T19:21:39.143959] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-11-04T19:21:39.367055] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-08. [2025-11-04T19:21:39.827020] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-11-04T19:21:40.167073] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-05. [2025-11-04T19:21:40.718186] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T19:21:40.967054] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-02. [2025-11-04T19:21:41.443195] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T19:21:41.767045] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-03. [2025-11-04T19:21:42.388257] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-11-04T19:21:42.567061] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-04. [2025-11-04T19:21:43.180358] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-11-04T19:21:43.367040] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-05. [2025-11-04T19:21:43.859454] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-11-04T19:21:44.167043] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-01. [2025-11-04T19:21:44.967049] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-09. [2025-11-04T19:21:45.358403] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-11-04T19:21:45.568725] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-11-04T19:21:45.767017] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-06. [2025-11-04T19:21:46.340288] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-11-04T19:21:46.567044] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-02. [2025-11-04T19:21:47.367070] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-10. [2025-11-04T19:21:47.535458] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T19:21:47.813790] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-11-04T19:21:48.167047] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-03. [2025-11-04T19:21:48.967054] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-06. [2025-11-04T19:21:49.464762] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T19:21:49.502298] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-11-04T19:21:49.767036] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-11. [2025-11-04T19:21:50.321397] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T19:21:50.322438] [INFO ] [REMOTE ] HGLB: Full fetch complete (14639 records). [2025-11-04T19:21:50.322506] [DEBUG] [VALIDATOR ] HGLB: Validating primary data. [2025-11-04T19:21:50.572341] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-04. [2025-11-04T19:21:50.692508] [DEBUG] [VALIDATOR ] HGLB: All expected trading days are present. [2025-11-04T19:21:50.692583] [DEBUG] [VALIDATOR ] HGLB: Primary data validation passed. [2025-11-04T19:21:50.692920] [INFO ] [MAIN ] HGLB: Fetched primary data is valid. [2025-11-04T19:21:50.692981] [DEBUG] [LOCAL ] HGLB: Saving primary data. [2025-11-04T19:21:50.693438] [DEBUG] [LOCAL ] HGLB: Trimmed 14639 records to 9354 (keeping 200 days of data) [2025-11-04T19:21:50.748074] [DEBUG] [LOCAL ] HGLB: Primary data saved (9354 records). [2025-11-04T19:21:50.748204] [INFO ] [MAIN ] HGLB: Full history saved. [2025-11-04T19:21:50.748285] [INFO ] [MAIN ] Update Progress: 1011/2367 symbols completed. HGLB took 31.7s. Estimated remaining: 3h 10m 11s. Rate: 7.1 symbols/min [2025-11-04T19:21:50.921901] [INFO ] [MAIN ] HHH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:21:50.951406] [DEBUG] [LOCAL ] HHH: Loaded 32780 existing records from primary storage. [2025-11-04T19:21:50.951684] [INFO ] [MAIN ] HHH: Loaded 32780 existing records. [2025-11-04T19:21:51.129085] [WARN ] [MAIN ] HHH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:21:51.129350] [INFO ] [MAIN ] HHH: Wiping existing primary data before full refresh. [2025-11-04T19:21:51.129406] [INFO ] [LOCAL ] HHH: Starting deletion of primary data. [2025-11-04T19:21:51.129954] [DEBUG] [LOCAL ] HHH: Deleted primary data file. [2025-11-04T19:21:51.130013] [INFO ] [MAIN ] HHH: Starting full primary refresh. [2025-11-04T19:21:51.373709] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-07. [2025-11-04T19:21:51.491141] [INFO ] [REMOTE ] HHH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:21:52.003424] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T19:21:52.076193] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T19:21:52.167046] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-05. [2025-11-04T19:21:52.967058] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-07. [2025-11-04T19:21:53.275270] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T19:21:53.567514] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T19:21:53.767068] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-08. [2025-11-04T19:21:54.438815] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T19:21:54.567100] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-09. [2025-11-04T19:21:55.230891] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-11-04T19:21:55.367044] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-08. [2025-11-04T19:21:56.171447] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-10. [2025-11-04T19:21:56.175174] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-11-04T19:21:56.967018] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-06. [2025-11-04T19:21:57.160011] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-11-04T19:21:57.767039] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-09. [2025-11-04T19:21:57.984261] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T19:21:58.309842] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T19:21:58.567016] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-01. [2025-11-04T19:21:59.367016] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-07. [2025-11-04T19:21:59.514276] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T19:22:00.167095] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-10. [2025-11-04T19:22:00.899015] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-11-04T19:22:00.901491] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T19:22:00.967022] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-08. [2025-11-04T19:22:01.767103] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-11. [2025-11-04T19:22:02.186101] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T19:22:02.510327] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T19:22:02.511695] [INFO ] [REMOTE ] HGTY: Full fetch complete (17752 records). [2025-11-04T19:22:02.511789] [DEBUG] [VALIDATOR ] HGTY: Validating primary data. [2025-11-04T19:22:02.572231] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-09. [2025-11-04T19:22:02.890438] [DEBUG] [VALIDATOR ] HGTY: All expected trading days are present. [2025-11-04T19:22:02.890516] [DEBUG] [VALIDATOR ] HGTY: Primary data validation passed. [2025-11-04T19:22:02.890968] [INFO ] [MAIN ] HGTY: Fetched primary data is valid. [2025-11-04T19:22:02.891034] [DEBUG] [LOCAL ] HGTY: Saving primary data. [2025-11-04T19:22:02.891583] [DEBUG] [LOCAL ] HGTY: Trimmed 17752 records to 13570 (keeping 200 days of data) [2025-11-04T19:22:02.965964] [DEBUG] [LOCAL ] HGTY: Primary data saved (13570 records). [2025-11-04T19:22:02.966083] [INFO ] [MAIN ] HGTY: Full history saved. [2025-11-04T19:22:02.966155] [INFO ] [MAIN ] Update Progress: 1012/2367 symbols completed. HGTY took 34.7s. Estimated remaining: 3h 10m 7s. Rate: 7.1 symbols/min [2025-11-04T19:22:03.134253] [INFO ] [MAIN ] HI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:22:03.165877] [DEBUG] [LOCAL ] HI: Loaded 36381 existing records from primary storage. [2025-11-04T19:22:03.166145] [INFO ] [MAIN ] HI: Loaded 36381 existing records. [2025-11-04T19:22:03.338329] [WARN ] [MAIN ] HI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:22:03.338548] [INFO ] [MAIN ] HI: Wiping existing primary data before full refresh. [2025-11-04T19:22:03.338601] [INFO ] [LOCAL ] HI: Starting deletion of primary data. [2025-11-04T19:22:03.339365] [DEBUG] [LOCAL ] HI: Deleted primary data file. [2025-11-04T19:22:03.339425] [INFO ] [MAIN ] HI: Starting full primary refresh. [2025-11-04T19:22:03.372225] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-02. [2025-11-04T19:22:03.692227] [INFO ] [REMOTE ] HI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:22:03.809259] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-11-04T19:22:04.167046] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-11. [2025-11-04T19:22:04.469388] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T19:22:04.881622] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T19:22:04.882767] [INFO ] [REMOTE ] HGER: Full fetch complete (20110 records). [2025-11-04T19:22:04.882835] [DEBUG] [VALIDATOR ] HGER: Validating primary data. [2025-11-04T19:22:04.972191] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-10. [2025-11-04T19:22:05.290586] [DEBUG] [VALIDATOR ] HGER: All expected trading days are present. [2025-11-04T19:22:05.290714] [DEBUG] [VALIDATOR ] HGER: Primary data validation passed. [2025-11-04T19:22:05.291161] [INFO ] [MAIN ] HGER: Fetched primary data is valid. [2025-11-04T19:22:05.291222] [DEBUG] [LOCAL ] HGER: Saving primary data. [2025-11-04T19:22:05.291876] [DEBUG] [LOCAL ] HGER: Trimmed 20110 records to 16321 (keeping 200 days of data) [2025-11-04T19:22:05.385955] [DEBUG] [LOCAL ] HGER: Primary data saved (16321 records). [2025-11-04T19:22:05.386093] [INFO ] [MAIN ] HGER: Full history saved. [2025-11-04T19:22:05.386171] [INFO ] [MAIN ] Update Progress: 1013/2367 symbols completed. HGER took 49.5s. Estimated remaining: 3h 9m 51s. Rate: 7.1 symbols/min [2025-11-04T19:22:05.553343] [INFO ] [MAIN ] HIG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:22:05.599549] [DEBUG] [LOCAL ] HIG: Loaded 50976 existing records from primary storage. [2025-11-04T19:22:05.599909] [INFO ] [MAIN ] HIG: Loaded 50976 existing records. [2025-11-04T19:22:05.774576] [WARN ] [MAIN ] HIG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:22:05.770765] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-01. [2025-11-04T19:22:05.775011] [INFO ] [MAIN ] HIG: Wiping existing primary data before full refresh. [2025-11-04T19:22:05.775082] [INFO ] [LOCAL ] HIG: Starting deletion of primary data. [2025-11-04T19:22:05.776596] [DEBUG] [LOCAL ] HIG: Deleted primary data file. [2025-11-04T19:22:05.776711] [INFO ] [MAIN ] HIG: Starting full primary refresh. [2025-11-04T19:22:06.135427] [INFO ] [REMOTE ] HIG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:22:06.199482] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T19:22:06.567064] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-01. [2025-11-04T19:22:06.677253] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-11-04T19:22:07.367031] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-03. [2025-11-04T19:22:07.931203] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T19:22:08.167027] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-11. [2025-11-04T19:22:08.256993] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-11-04T19:22:08.967101] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-04. [2025-11-04T19:22:09.135222] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-11 (0.97 seconds). [2025-11-04T19:22:09.137186] [INFO ] [REMOTE ] HGV: Full fetch complete (66007 records). [2025-11-04T19:22:09.137297] [DEBUG] [VALIDATOR ] HGV: Validating primary data. [2025-11-04T19:22:09.555347] [DEBUG] [VALIDATOR ] HGV: All expected trading days are present. [2025-11-04T19:22:09.555429] [DEBUG] [VALIDATOR ] HGV: Primary data validation passed. [2025-11-04T19:22:09.556272] [INFO ] [MAIN ] HGV: Fetched primary data is valid. [2025-11-04T19:22:09.556341] [DEBUG] [LOCAL ] HGV: Saving primary data. [2025-11-04T19:22:09.557932] [DEBUG] [LOCAL ] HGV: Trimmed 66007 records to 45827 (keeping 200 days of data) [2025-11-04T19:22:09.790614] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-02. [2025-11-04T19:22:09.883579] [DEBUG] [LOCAL ] HGV: Primary data saved (45827 records). [2025-11-04T19:22:09.883974] [INFO ] [MAIN ] HGV: Full history saved. [2025-11-04T19:22:09.884171] [INFO ] [MAIN ] Update Progress: 1014/2367 symbols completed. HGV took 33.5s. Estimated remaining: 3h 9m 37s. Rate: 7.1 symbols/min [2025-11-04T19:22:10.030977] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-11-04T19:22:10.123169] [INFO ] [MAIN ] HII: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:22:10.160434] [DEBUG] [LOCAL ] HII: Loaded 32647 existing records from primary storage. [2025-11-04T19:22:10.160791] [INFO ] [MAIN ] HII: Loaded 32647 existing records. [2025-11-04T19:22:10.368256] [WARN ] [MAIN ] HII: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:22:10.368489] [INFO ] [MAIN ] HII: Wiping existing primary data before full refresh. [2025-11-04T19:22:10.368537] [INFO ] [LOCAL ] HII: Starting deletion of primary data. [2025-11-04T19:22:10.369138] [DEBUG] [LOCAL ] HII: Deleted primary data file. [2025-11-04T19:22:10.369195] [INFO ] [MAIN ] HII: Starting full primary refresh. [2025-11-04T19:22:10.576212] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-05. [2025-11-04T19:22:10.726559] [INFO ] [REMOTE ] HII: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:22:10.807860] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T19:22:11.367139] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-03. [2025-11-04T19:22:11.903155] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T19:22:12.167097] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-02. [2025-11-04T19:22:12.291396] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-11-04T19:22:12.967072] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-01. [2025-11-04T19:22:13.397204] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T19:22:13.767012] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-04. [2025-11-04T19:22:13.972312] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-11-04T19:22:14.567085] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-02. [2025-11-04T19:22:14.797259] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-11-04T19:22:15.367267] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-03. [2025-11-04T19:22:15.860259] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T19:22:16.167080] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-03. [2025-11-04T19:22:16.869605] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-11-04T19:22:16.967082] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-04. [2025-11-04T19:22:17.570162] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T19:22:17.767042] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-04. [2025-11-04T19:22:18.278592] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T19:22:18.567052] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-06. [2025-11-04T19:22:19.014618] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-11-04T19:22:19.367038] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-05. [2025-11-04T19:22:19.692528] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-11-04T19:22:20.167065] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-07. [2025-11-04T19:22:20.560441] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T19:22:20.967075] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-06. [2025-11-04T19:22:21.229505] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-11-04T19:22:21.767018] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-05. [2025-11-04T19:22:21.841738] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T19:22:22.567044] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-08. [2025-11-04T19:22:23.367036] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-05. [2025-11-04T19:22:23.424943] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-11-04T19:22:23.701265] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T19:22:24.167042] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-07. [2025-11-04T19:22:24.424264] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-11-04T19:22:24.967060] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-06. [2025-11-04T19:22:25.226981] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-11-04T19:22:25.767046] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-08. [2025-11-04T19:22:26.262075] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T19:22:26.567085] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-06. [2025-11-04T19:22:26.881305] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T19:22:27.367091] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-09. [2025-11-04T19:22:27.628946] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-11-04T19:22:28.167187] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-09. [2025-11-04T19:22:28.350551] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T19:22:28.967069] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-07. [2025-11-04T19:22:29.288016] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T19:22:29.767058] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-07. [2025-11-04T19:22:30.458995] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-11-04T19:22:30.567122] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-08. [2025-11-04T19:22:30.939947] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T19:22:31.367040] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-10. [2025-11-04T19:22:31.922509] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-11-04T19:22:32.167076] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-10. [2025-11-04T19:22:32.406902] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-11-04T19:22:32.967145] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-08. [2025-11-04T19:22:33.475475] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T19:22:33.767035] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-11. [2025-11-04T19:22:33.982631] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-11-04T19:22:34.567058] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-09. [2025-11-04T19:22:35.091375] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-11 (1.32 seconds). [2025-11-04T19:22:35.093172] [INFO ] [REMOTE ] HI: Full fetch complete (52190 records). [2025-11-04T19:22:35.093278] [DEBUG] [VALIDATOR ] HI: Validating primary data. [2025-11-04T19:22:35.372472] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-11. [2025-11-04T19:22:35.484763] [DEBUG] [VALIDATOR ] HI: All expected trading days are present. [2025-11-04T19:22:35.484867] [DEBUG] [VALIDATOR ] HI: Primary data validation passed. [2025-11-04T19:22:35.485606] [INFO ] [MAIN ] HI: Fetched primary data is valid. [2025-11-04T19:22:35.485764] [DEBUG] [LOCAL ] HI: Saving primary data. [2025-11-04T19:22:35.487612] [DEBUG] [LOCAL ] HI: Trimmed 52190 records to 36432 (keeping 200 days of data) [2025-11-04T19:22:35.570894] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T19:22:35.736247] [DEBUG] [LOCAL ] HI: Primary data saved (36432 records). [2025-11-04T19:22:35.736396] [INFO ] [MAIN ] HI: Full history saved. [2025-11-04T19:22:35.736475] [INFO ] [MAIN ] Update Progress: 1015/2367 symbols completed. HI took 32.8s. Estimated remaining: 3h 9m 52s. Rate: 7.1 symbols/min [2025-11-04T19:22:35.904883] [INFO ] [MAIN ] HIMS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:22:35.953910] [DEBUG] [LOCAL ] HIMS: Loaded 53817 existing records from primary storage. [2025-11-04T19:22:35.954295] [INFO ] [MAIN ] HIMS: Loaded 53817 existing records. [2025-11-04T19:22:36.148629] [WARN ] [MAIN ] HIMS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:22:36.148862] [INFO ] [MAIN ] HIMS: Wiping existing primary data before full refresh. [2025-11-04T19:22:36.148910] [INFO ] [LOCAL ] HIMS: Starting deletion of primary data. [2025-11-04T19:22:36.149931] [DEBUG] [LOCAL ] HIMS: Deleted primary data file. [2025-11-04T19:22:36.149991] [INFO ] [MAIN ] HIMS: Starting full primary refresh. [2025-11-04T19:22:36.172212] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-09. [2025-11-04T19:22:36.408492] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T19:22:36.441823] [INFO ] [REMOTE ] HHH: Full fetch complete (45859 records). [2025-11-04T19:22:36.442049] [DEBUG] [VALIDATOR ] HHH: Validating primary data. [2025-11-04T19:22:36.626040] [INFO ] [REMOTE ] HIMS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:22:36.901753] [DEBUG] [VALIDATOR ] HHH: All expected trading days are present. [2025-11-04T19:22:36.901843] [DEBUG] [VALIDATOR ] HHH: Primary data validation passed. [2025-11-04T19:22:36.902506] [INFO ] [MAIN ] HHH: Fetched primary data is valid. [2025-11-04T19:22:36.902545] [DEBUG] [LOCAL ] HHH: Saving primary data. [2025-11-04T19:22:36.903551] [DEBUG] [LOCAL ] HHH: Trimmed 45859 records to 32867 (keeping 200 days of data) [2025-11-04T19:22:36.987723] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-10. [2025-11-04T19:22:37.101875] [DEBUG] [LOCAL ] HHH: Primary data saved (32867 records). [2025-11-04T19:22:37.102015] [INFO ] [MAIN ] HHH: Full history saved. [2025-11-04T19:22:37.102093] [INFO ] [MAIN ] Update Progress: 1016/2367 symbols completed. HHH took 46.4s. Estimated remaining: 3h 9m 34s. Rate: 7.1 symbols/min [2025-11-04T19:22:37.267938] [INFO ] [MAIN ] HIO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:22:37.288223] [DEBUG] [LOCAL ] HIO: Loaded 23577 existing records from primary storage. [2025-11-04T19:22:37.288467] [INFO ] [MAIN ] HIO: Loaded 23577 existing records. [2025-11-04T19:22:37.471695] [WARN ] [MAIN ] HIO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:22:37.471901] [INFO ] [MAIN ] HIO: Wiping existing primary data before full refresh. [2025-11-04T19:22:37.471947] [INFO ] [LOCAL ] HIO: Starting deletion of primary data. [2025-11-04T19:22:37.472437] [DEBUG] [LOCAL ] HIO: Deleted primary data file. [2025-11-04T19:22:37.472494] [INFO ] [MAIN ] HIO: Starting full primary refresh. [2025-11-04T19:22:37.769948] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-01. [2025-11-04T19:22:37.841727] [INFO ] [REMOTE ] HIO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:22:37.870304] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-11-04T19:22:38.213738] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T19:22:38.567044] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-01. [2025-11-04T19:22:39.367107] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-10. [2025-11-04T19:22:39.454990] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-11-04T19:22:39.553760] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-11-04T19:22:40.167055] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-02. [2025-11-04T19:22:40.824490] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T19:22:40.856117] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-11-04T19:22:40.967024] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-11. [2025-11-04T19:22:41.290718] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-11 (0.32 seconds). [2025-11-04T19:22:41.292699] [INFO ] [REMOTE ] HIG: Full fetch complete (73760 records). [2025-11-04T19:22:41.292820] [DEBUG] [VALIDATOR ] HIG: Validating primary data. [2025-11-04T19:22:41.674339] [DEBUG] [VALIDATOR ] HIG: All expected trading days are present. [2025-11-04T19:22:41.674418] [DEBUG] [VALIDATOR ] HIG: Primary data validation passed. [2025-11-04T19:22:41.675289] [INFO ] [MAIN ] HIG: Fetched primary data is valid. [2025-11-04T19:22:41.675350] [DEBUG] [LOCAL ] HIG: Saving primary data. [2025-11-04T19:22:41.676716] [DEBUG] [LOCAL ] HIG: Trimmed 73760 records to 50975 (keeping 200 days of data) [2025-11-04T19:22:41.768788] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-03. [2025-11-04T19:22:41.995141] [DEBUG] [LOCAL ] HIG: Primary data saved (50975 records). [2025-11-04T19:22:41.995279] [INFO ] [MAIN ] HIG: Full history saved. [2025-11-04T19:22:41.995360] [INFO ] [MAIN ] Update Progress: 1017/2367 symbols completed. HIG took 36.6s. Estimated remaining: 3h 9m 21s. Rate: 7.1 symbols/min [2025-11-04T19:22:42.162193] [INFO ] [MAIN ] HIPO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:22:42.181426] [DEBUG] [LOCAL ] HIPO: Loaded 20894 existing records from primary storage. [2025-11-04T19:22:42.181690] [INFO ] [MAIN ] HIPO: Loaded 20894 existing records. [2025-11-04T19:22:42.357511] [WARN ] [MAIN ] HIPO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:22:42.357751] [INFO ] [MAIN ] HIPO: Wiping existing primary data before full refresh. [2025-11-04T19:22:42.357824] [INFO ] [LOCAL ] HIPO: Starting deletion of primary data. [2025-11-04T19:22:42.358436] [DEBUG] [LOCAL ] HIPO: Deleted primary data file. [2025-11-04T19:22:42.358497] [INFO ] [MAIN ] HIPO: Starting full primary refresh. [2025-11-04T19:22:42.572210] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-11. [2025-11-04T19:22:42.711893] [INFO ] [REMOTE ] HIPO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:22:42.860808] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T19:22:43.367030] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-01. [2025-11-04T19:22:43.871101] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T19:22:43.872860] [INFO ] [REMOTE ] HII: Full fetch complete (52354 records). [2025-11-04T19:22:43.872946] [DEBUG] [VALIDATOR ] HII: Validating primary data. [2025-11-04T19:22:44.170463] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-02. [2025-11-04T19:22:44.268170] [DEBUG] [VALIDATOR ] HII: All expected trading days are present. [2025-11-04T19:22:44.268384] [DEBUG] [VALIDATOR ] HII: Primary data validation passed. [2025-11-04T19:22:44.269203] [INFO ] [MAIN ] HII: Fetched primary data is valid. [2025-11-04T19:22:44.269277] [DEBUG] [LOCAL ] HII: Saving primary data. [2025-11-04T19:22:44.281936] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-11-04T19:22:44.282059] [DEBUG] [LOCAL ] HII: Trimmed 52354 records to 32686 (keeping 200 days of data) [2025-11-04T19:22:44.490939] [DEBUG] [LOCAL ] HII: Primary data saved (32686 records). [2025-11-04T19:22:44.491080] [INFO ] [MAIN ] HII: Full history saved. [2025-11-04T19:22:44.491160] [INFO ] [MAIN ] Update Progress: 1018/2367 symbols completed. HII took 34.6s. Estimated remaining: 3h 9m 5s. Rate: 7.1 symbols/min [2025-11-04T19:22:44.659128] [INFO ] [MAIN ] HIW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:22:44.696402] [DEBUG] [LOCAL ] HIW: Loaded 44539 existing records from primary storage. [2025-11-04T19:22:44.696785] [INFO ] [MAIN ] HIW: Loaded 44539 existing records. [2025-11-04T19:22:44.871059] [WARN ] [MAIN ] HIW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:22:44.871259] [INFO ] [MAIN ] HIW: Wiping existing primary data before full refresh. [2025-11-04T19:22:44.871307] [INFO ] [LOCAL ] HIW: Starting deletion of primary data. [2025-11-04T19:22:44.872069] [DEBUG] [LOCAL ] HIW: Deleted primary data file. [2025-11-04T19:22:44.872130] [INFO ] [MAIN ] HIW: Starting full primary refresh. [2025-11-04T19:22:44.970836] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-04. [2025-11-04T19:22:45.223189] [INFO ] [REMOTE ] HIW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:22:45.767024] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-01. [2025-11-04T19:22:46.021085] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-11-04T19:22:46.048240] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T19:22:46.567040] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-05. [2025-11-04T19:22:46.897943] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T19:22:47.311570] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-11-04T19:22:47.367052] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-06. [2025-11-04T19:22:48.156717] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T19:22:48.167003] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-02. [2025-11-04T19:22:48.940794] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-11-04T19:22:48.967001] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-07. [2025-11-04T19:22:49.641155] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T19:22:49.767005] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-02. [2025-11-04T19:22:50.567101] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-08. [2025-11-04T19:22:50.986579] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T19:22:51.367008] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-03. [2025-11-04T19:22:51.391601] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-11-04T19:22:52.125230] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T19:22:52.167008] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-04. [2025-11-04T19:22:52.948430] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-11-04T19:22:52.966999] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-05. [2025-11-04T19:22:53.693302] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T19:22:53.767038] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-09. [2025-11-04T19:22:54.567035] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-03. [2025-11-04T19:22:54.655787] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-11-04T19:22:55.367031] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-06. [2025-11-04T19:22:55.797040] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-11-04T19:22:56.167009] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-04. [2025-11-04T19:22:56.308894] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-11-04T19:22:56.967065] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-03. [2025-11-04T19:22:57.232883] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T19:22:57.767102] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-05. [2025-11-04T19:22:58.567111] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-07. [2025-11-04T19:22:59.106163] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T19:22:59.309637] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-11-04T19:22:59.351594] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-03 (2.38 seconds). [2025-11-04T19:22:59.366994] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-04. [2025-11-04T19:23:00.167081] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-06. [2025-11-04T19:23:00.967038] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-10. [2025-11-04T19:23:01.767063] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-08. [2025-11-04T19:23:05.498328] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-08 (3.73 seconds). [2025-11-04T19:23:05.503913] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-09. [2025-11-04T19:23:05.982043] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-06 (5.81 seconds). [2025-11-04T19:23:05.990847] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-07. [2025-11-04T19:23:06.057183] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-10 (5.09 seconds). [2025-11-04T19:23:06.064842] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-11. [2025-11-04T19:23:06.193188] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-11-04T19:23:06.198708] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-10. [2025-11-04T19:23:06.581444] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-04 (7.21 seconds). [2025-11-04T19:23:06.594492] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-05. [2025-11-04T19:23:06.834190] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-11-04T19:23:06.838924] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-11. [2025-11-04T19:23:06.900609] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T19:23:06.901842] [INFO ] [REMOTE ] HIO: Full fetch complete (37866 records). [2025-11-04T19:23:06.901913] [DEBUG] [VALIDATOR ] HIO: Validating primary data. [2025-11-04T19:23:07.297313] [DEBUG] [VALIDATOR ] HIO: All expected trading days are present. [2025-11-04T19:23:07.297422] [DEBUG] [VALIDATOR ] HIO: Primary data validation passed. [2025-11-04T19:23:07.298229] [INFO ] [MAIN ] HIO: Fetched primary data is valid. [2025-11-04T19:23:07.298304] [DEBUG] [LOCAL ] HIO: Saving primary data. [2025-11-04T19:23:07.324862] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T19:23:07.324038] [DEBUG] [LOCAL ] HIO: Trimmed 37866 records to 23623 (keeping 200 days of data) [2025-11-04T19:23:07.380787] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-08. [2025-11-04T19:23:07.471207] [DEBUG] [LOCAL ] HIO: Primary data saved (23623 records). [2025-11-04T19:23:07.471333] [INFO ] [MAIN ] HIO: Full history saved. [2025-11-04T19:23:07.471407] [INFO ] [MAIN ] Update Progress: 1019/2367 symbols completed. HIO took 30.4s. Estimated remaining: 3h 9m 16s. Rate: 7.1 symbols/min [2025-11-04T19:23:07.645995] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T19:23:07.647942] [INFO ] [MAIN ] HIX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:23:07.649967] [INFO ] [REMOTE ] HIPO: Full fetch complete (30944 records). [2025-11-04T19:23:07.650152] [DEBUG] [VALIDATOR ] HIPO: Validating primary data. [2025-11-04T19:23:07.831775] [DEBUG] [LOCAL ] HIX: Loaded 20029 existing records from primary storage. [2025-11-04T19:23:07.857750] [INFO ] [MAIN ] HIX: Loaded 20029 existing records. [2025-11-04T19:23:08.237349] [DEBUG] [VALIDATOR ] HIPO: All expected trading days are present. [2025-11-04T19:23:08.224415] [WARN ] [MAIN ] HIX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:23:08.237596] [DEBUG] [VALIDATOR ] HIPO: Primary data validation passed. [2025-11-04T19:23:08.238208] [INFO ] [MAIN ] HIPO: Fetched primary data is valid. [2025-11-04T19:23:08.237543] [INFO ] [MAIN ] HIX: Wiping existing primary data before full refresh. [2025-11-04T19:23:08.238346] [DEBUG] [LOCAL ] HIPO: Saving primary data. [2025-11-04T19:23:08.238305] [INFO ] [LOCAL ] HIX: Starting deletion of primary data. [2025-11-04T19:23:08.238976] [DEBUG] [LOCAL ] HIX: Deleted primary data file. [2025-11-04T19:23:08.239040] [INFO ] [MAIN ] HIX: Starting full primary refresh. [2025-11-04T19:23:08.309223] [DEBUG] [LOCAL ] HIPO: Trimmed 30944 records to 20966 (keeping 200 days of data) [2025-11-04T19:23:08.730065] [INFO ] [REMOTE ] HIX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:23:08.739102] [DEBUG] [LOCAL ] HIPO: Primary data saved (20966 records). [2025-11-04T19:23:08.739265] [INFO ] [MAIN ] HIPO: Full history saved. [2025-11-04T19:23:08.739788] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-01. [2025-11-04T19:23:08.739393] [INFO ] [MAIN ] Update Progress: 1020/2367 symbols completed. HIPO took 26.7s. Estimated remaining: 3h 8m 58s. Rate: 7.1 symbols/min [2025-11-04T19:23:08.901595] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-11-04T19:23:08.959928] [INFO ] [MAIN ] HKD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:23:08.970572] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-09. [2025-11-04T19:23:08.981292] [DEBUG] [LOCAL ] HKD: Loaded 19173 existing records from primary storage. [2025-11-04T19:23:08.981572] [INFO ] [MAIN ] HKD: Loaded 19173 existing records. [2025-11-04T19:23:09.167317] [WARN ] [MAIN ] HKD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:23:09.167569] [INFO ] [MAIN ] HKD: Wiping existing primary data before full refresh. [2025-11-04T19:23:09.167790] [INFO ] [LOCAL ] HKD: Starting deletion of primary data. [2025-11-04T19:23:09.168262] [DEBUG] [LOCAL ] HKD: Deleted primary data file. [2025-11-04T19:23:09.168323] [INFO ] [MAIN ] HKD: Starting full primary refresh. [2025-11-04T19:23:09.403716] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-05 (2.81 seconds). [2025-11-04T19:23:09.615095] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T19:23:09.638608] [INFO ] [REMOTE ] HKD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:23:09.767013] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-02. [2025-11-04T19:23:10.284499] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T19:23:10.392851] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-11-04T19:23:10.567056] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-03. [2025-11-04T19:23:11.268297] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T19:23:11.367042] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-01. [2025-11-04T19:23:12.082562] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-11-04T19:23:12.167047] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-06. [2025-11-04T19:23:12.967027] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-10. [2025-11-04T19:23:13.767091] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-04. [2025-11-04T19:23:14.068414] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-11-04T19:23:14.284702] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T19:23:14.559132] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-11-04T19:23:14.566985] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-05. [2025-11-04T19:23:15.327961] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-11-04T19:23:15.367117] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-07. [2025-11-04T19:23:16.167123] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-02. [2025-11-04T19:23:16.967024] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-11. [2025-11-04T19:23:17.013687] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T19:23:17.773973] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-03. [2025-11-04T19:23:17.803965] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-07 (2.44 seconds). [2025-11-04T19:23:18.066633] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-11 (1.10 seconds). [2025-11-04T19:23:18.068544] [INFO ] [REMOTE ] HIW: Full fetch complete (65209 records). [2025-11-04T19:23:18.068623] [DEBUG] [VALIDATOR ] HIW: Validating primary data. [2025-11-04T19:23:18.448629] [DEBUG] [VALIDATOR ] HIW: All expected trading days are present. [2025-11-04T19:23:18.448751] [DEBUG] [VALIDATOR ] HIW: Primary data validation passed. [2025-11-04T19:23:18.449775] [INFO ] [MAIN ] HIW: Fetched primary data is valid. [2025-11-04T19:23:18.449893] [DEBUG] [LOCAL ] HIW: Saving primary data. [2025-11-04T19:23:18.451325] [DEBUG] [LOCAL ] HIW: Trimmed 65209 records to 44594 (keeping 200 days of data) [2025-11-04T19:23:18.560850] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T19:23:18.575565] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-06. [2025-11-04T19:23:18.731662] [DEBUG] [LOCAL ] HIW: Primary data saved (44594 records). [2025-11-04T19:23:18.731788] [INFO ] [MAIN ] HIW: Full history saved. [2025-11-04T19:23:18.731865] [INFO ] [MAIN ] Update Progress: 1021/2367 symbols completed. HIW took 34.2s. Estimated remaining: 3h 8m 51s. Rate: 7.1 symbols/min [2025-11-04T19:23:18.898150] [INFO ] [MAIN ] HL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:23:18.943901] [DEBUG] [LOCAL ] HL: Loaded 53677 existing records from primary storage. [2025-11-04T19:23:18.944195] [INFO ] [MAIN ] HL: Loaded 53677 existing records. [2025-11-04T19:23:19.116039] [WARN ] [MAIN ] HL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:23:19.116206] [INFO ] [MAIN ] HL: Wiping existing primary data before full refresh. [2025-11-04T19:23:19.116251] [INFO ] [LOCAL ] HL: Starting deletion of primary data. [2025-11-04T19:23:19.117105] [DEBUG] [LOCAL ] HL: Deleted primary data file. [2025-11-04T19:23:19.117163] [INFO ] [MAIN ] HL: Starting full primary refresh. [2025-11-04T19:23:19.367021] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-08. [2025-11-04T19:23:19.461080] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T19:23:19.488868] [INFO ] [REMOTE ] HL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:23:20.167026] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-07. [2025-11-04T19:23:20.897881] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-11-04T19:23:20.967080] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-08. [2025-11-04T19:23:21.773108] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-04. [2025-11-04T19:23:21.813152] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-08 (2.45 seconds). [2025-11-04T19:23:21.815472] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-11-04T19:23:22.511062] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-11-04T19:23:22.567048] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-05. [2025-11-04T19:23:23.246195] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-11-04T19:23:23.367041] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-01. [2025-11-04T19:23:24.167071] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-06. [2025-11-04T19:23:24.837759] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T19:23:24.967011] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-09. [2025-11-04T19:23:25.241222] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T19:23:25.722603] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-11-04T19:23:25.767024] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-02. [2025-11-04T19:23:26.567094] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-07. [2025-11-04T19:23:27.337903] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T19:23:27.373267] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-08. [2025-11-04T19:23:27.393486] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-11-04T19:23:28.071210] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T19:23:28.167024] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-09. [2025-11-04T19:23:28.967057] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-10. [2025-11-04T19:23:29.686450] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-11-04T19:23:29.767062] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-11. [2025-11-04T19:23:30.529300] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-11 (0.76 seconds). [2025-11-04T19:23:30.532452] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-09 (2.37 seconds). [2025-11-04T19:23:30.532549] [INFO ] [REMOTE ] HIX: Full fetch complete (29441 records). [2025-11-04T19:23:30.543383] [DEBUG] [VALIDATOR ] HIX: Validating primary data. [2025-11-04T19:23:30.578611] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-09. [2025-11-04T19:23:30.918768] [DEBUG] [VALIDATOR ] HIX: All expected trading days are present. [2025-11-04T19:23:30.918844] [DEBUG] [VALIDATOR ] HIX: Primary data validation passed. [2025-11-04T19:23:30.919355] [INFO ] [MAIN ] HIX: Fetched primary data is valid. [2025-11-04T19:23:30.919393] [DEBUG] [LOCAL ] HIX: Saving primary data. [2025-11-04T19:23:30.920097] [DEBUG] [LOCAL ] HIX: Trimmed 29441 records to 20052 (keeping 200 days of data) [2025-11-04T19:23:31.032844] [DEBUG] [LOCAL ] HIX: Primary data saved (20052 records). [2025-11-04T19:23:31.032959] [INFO ] [MAIN ] HIX: Full history saved. [2025-11-04T19:23:31.033032] [INFO ] [MAIN ] Update Progress: 1022/2367 symbols completed. HIX took 23.6s. Estimated remaining: 3h 8m 48s. Rate: 7.1 symbols/min [2025-11-04T19:23:31.203159] [INFO ] [MAIN ] HLF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:23:31.242043] [DEBUG] [LOCAL ] HLF: Loaded 48033 existing records from primary storage. [2025-11-04T19:23:31.242337] [INFO ] [MAIN ] HLF: Loaded 48033 existing records. [2025-11-04T19:23:31.380636] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-03. [2025-11-04T19:23:31.423347] [WARN ] [MAIN ] HLF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:23:31.424862] [INFO ] [MAIN ] HLF: Wiping existing primary data before full refresh. [2025-11-04T19:23:31.424930] [INFO ] [LOCAL ] HLF: Starting deletion of primary data. [2025-11-04T19:23:31.425948] [DEBUG] [LOCAL ] HLF: Deleted primary data file. [2025-11-04T19:23:31.426007] [INFO ] [MAIN ] HLF: Starting full primary refresh. [2025-11-04T19:23:31.803356] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T19:23:31.808816] [INFO ] [REMOTE ] HLF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:23:32.167006] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-10. [2025-11-04T19:23:32.967026] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-10. [2025-11-04T19:23:32.994006] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-11-04T19:23:33.065667] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-11-04T19:23:33.767093] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-01. [2025-11-04T19:23:34.567080] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-04. [2025-11-04T19:23:34.904598] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T19:23:35.367029] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-11. [2025-11-04T19:23:35.512236] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-10 (2.55 seconds). [2025-11-04T19:23:36.138702] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-11-04T19:23:36.166998] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-05. [2025-11-04T19:23:36.438885] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T19:23:36.440619] [INFO ] [REMOTE ] HKD: Full fetch complete (28350 records). [2025-11-04T19:23:36.440722] [DEBUG] [VALIDATOR ] HKD: Validating primary data. [2025-11-04T19:23:36.812785] [DEBUG] [VALIDATOR ] HKD: All expected trading days are present. [2025-11-04T19:23:36.812877] [DEBUG] [VALIDATOR ] HKD: Primary data validation passed. [2025-11-04T19:23:36.813396] [INFO ] [MAIN ] HKD: Fetched primary data is valid. [2025-11-04T19:23:36.813434] [DEBUG] [LOCAL ] HKD: Saving primary data. [2025-11-04T19:23:36.814084] [DEBUG] [LOCAL ] HKD: Trimmed 28350 records to 19442 (keeping 200 days of data) [2025-11-04T19:23:36.923122] [DEBUG] [LOCAL ] HKD: Primary data saved (19442 records). [2025-11-04T19:23:36.923274] [INFO ] [MAIN ] HKD: Full history saved. [2025-11-04T19:23:36.923357] [INFO ] [MAIN ] Update Progress: 1023/2367 symbols completed. HKD took 28.2s. Estimated remaining: 3h 8m 36s. Rate: 7.1 symbols/min [2025-11-04T19:23:36.978795] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-11. [2025-11-04T19:23:37.099042] [INFO ] [MAIN ] HLI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:23:37.128915] [DEBUG] [LOCAL ] HLI: Loaded 30617 existing records from primary storage. [2025-11-04T19:23:37.129188] [INFO ] [MAIN ] HLI: Loaded 30617 existing records. [2025-11-04T19:23:37.309262] [WARN ] [MAIN ] HLI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:23:37.309494] [INFO ] [MAIN ] HLI: Wiping existing primary data before full refresh. [2025-11-04T19:23:37.309540] [INFO ] [LOCAL ] HLI: Starting deletion of primary data. [2025-11-04T19:23:37.310329] [DEBUG] [LOCAL ] HLI: Deleted primary data file. [2025-11-04T19:23:37.310384] [INFO ] [MAIN ] HLI: Starting full primary refresh. [2025-11-04T19:23:37.675836] [INFO ] [REMOTE ] HLI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:23:37.716306] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-11-04T19:23:37.767003] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-06. [2025-11-04T19:23:38.567070] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-01. [2025-11-04T19:23:39.367545] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-11 (2.39 seconds). [2025-11-04T19:23:39.367359] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-02. [2025-11-04T19:23:39.370981] [INFO ] [REMOTE ] HIMS: Full fetch complete (77997 records). [2025-11-04T19:23:39.381366] [DEBUG] [VALIDATOR ] HIMS: Validating primary data. [2025-11-04T19:23:39.403540] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-11-04T19:23:39.891808] [DEBUG] [VALIDATOR ] HIMS: All expected trading days are present. [2025-11-04T19:23:39.893191] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T19:23:39.900176] [DEBUG] [VALIDATOR ] HIMS: Primary data validation passed. [2025-11-04T19:23:39.901841] [INFO ] [MAIN ] HIMS: Fetched primary data is valid. [2025-11-04T19:23:39.901906] [DEBUG] [LOCAL ] HIMS: Saving primary data. [2025-11-04T19:23:39.903205] [DEBUG] [LOCAL ] HIMS: Trimmed 77997 records to 53817 (keeping 200 days of data) [2025-11-04T19:23:40.182450] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-02. [2025-11-04T19:23:40.232524] [DEBUG] [LOCAL ] HIMS: Primary data saved (53817 records). [2025-11-04T19:23:40.232710] [INFO ] [MAIN ] HIMS: Full history saved. [2025-11-04T19:23:40.232817] [INFO ] [MAIN ] Update Progress: 1024/2367 symbols completed. HIMS took 1m 4s. Estimated remaining: 3h 8m 21s. Rate: 7.1 symbols/min [2025-11-04T19:23:40.412378] [INFO ] [MAIN ] HLIO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:23:40.438443] [DEBUG] [LOCAL ] HLIO: Loaded 27731 existing records from primary storage. [2025-11-04T19:23:40.438800] [INFO ] [MAIN ] HLIO: Loaded 27731 existing records. [2025-11-04T19:23:40.615527] [WARN ] [MAIN ] HLIO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:23:40.615791] [INFO ] [MAIN ] HLIO: Wiping existing primary data before full refresh. [2025-11-04T19:23:40.615850] [INFO ] [LOCAL ] HLIO: Starting deletion of primary data. [2025-11-04T19:23:40.616518] [DEBUG] [LOCAL ] HLIO: Deleted primary data file. [2025-11-04T19:23:40.616577] [INFO ] [MAIN ] HLIO: Starting full primary refresh. [2025-11-04T19:23:40.971628] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-07. [2025-11-04T19:23:40.999556] [INFO ] [REMOTE ] HLIO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:23:41.031503] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-11-04T19:23:41.062533] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-11-04T19:23:41.767065] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-03. [2025-11-04T19:23:42.566996] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-01. [2025-11-04T19:23:42.783009] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-11-04T19:23:43.205064] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T19:23:43.286369] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-11-04T19:23:43.366998] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-04. [2025-11-04T19:23:44.167050] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-03. [2025-11-04T19:23:44.681548] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T19:23:44.967038] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-05. [2025-11-04T19:23:45.272289] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T19:23:45.767077] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-02. [2025-11-04T19:23:46.306222] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T19:23:46.386344] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-11-04T19:23:46.567049] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-06. [2025-11-04T19:23:47.367017] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-04. [2025-11-04T19:23:47.753623] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-11-04T19:23:48.167079] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-03. [2025-11-04T19:23:48.763500] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T19:23:48.943341] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-11-04T19:23:48.967002] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-04. [2025-11-04T19:23:49.767082] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-05. [2025-11-04T19:23:49.942598] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-11-04T19:23:50.567093] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-05. [2025-11-04T19:23:50.836113] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T19:23:51.367048] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-06. [2025-11-04T19:23:51.600423] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-11-04T19:23:52.171480] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-06. [2025-11-04T19:23:52.173475] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T19:23:52.967015] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-07. [2025-11-04T19:23:53.123952] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-11-04T19:23:53.767028] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-07. [2025-11-04T19:23:53.860176] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-11-04T19:23:54.567091] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-08. [2025-11-04T19:23:55.068675] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T19:23:55.367008] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-08. [2025-11-04T19:23:55.539896] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-11-04T19:23:56.167050] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-08. [2025-11-04T19:23:56.744472] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T19:23:56.967060] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-07. [2025-11-04T19:23:57.553355] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T19:23:57.735906] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T19:23:57.767039] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-08. [2025-11-04T19:23:58.567017] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-09. [2025-11-04T19:23:58.728814] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-11-04T19:23:59.367025] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-09. [2025-11-04T19:23:59.900500] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T19:24:00.170179] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-09. [2025-11-04T19:24:00.171429] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-11-04T19:24:00.967008] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-10. [2025-11-04T19:24:01.131083] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T19:24:01.767086] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-10. [2025-11-04T19:24:02.123126] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T19:24:02.567107] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-11. [2025-11-04T19:24:02.700342] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-11-04T19:24:03.367053] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-09. [2025-11-04T19:24:03.669370] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-11 (1.10 seconds). [2025-11-04T19:24:03.671227] [INFO ] [REMOTE ] HLF: Full fetch complete (70605 records). [2025-11-04T19:24:03.671320] [DEBUG] [VALIDATOR ] HLF: Validating primary data. [2025-11-04T19:24:04.051179] [DEBUG] [VALIDATOR ] HLF: All expected trading days are present. [2025-11-04T19:24:04.051261] [DEBUG] [VALIDATOR ] HLF: Primary data validation passed. [2025-11-04T19:24:04.052111] [INFO ] [MAIN ] HLF: Fetched primary data is valid. [2025-11-04T19:24:04.052232] [DEBUG] [LOCAL ] HLF: Saving primary data. [2025-11-04T19:24:04.053683] [DEBUG] [LOCAL ] HLF: Trimmed 70605 records to 48020 (keeping 200 days of data) [2025-11-04T19:24:04.209388] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-10. [2025-11-04T19:24:04.320959] [DEBUG] [LOCAL ] HLF: Primary data saved (48020 records). [2025-11-04T19:24:04.321083] [INFO ] [MAIN ] HLF: Full history saved. [2025-11-04T19:24:04.321157] [INFO ] [MAIN ] Update Progress: 1025/2367 symbols completed. HLF took 33.3s. Estimated remaining: 3h 8m 33s. Rate: 7.1 symbols/min [2025-11-04T19:24:04.494075] [INFO ] [MAIN ] HLLY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:24:04.527663] [DEBUG] [LOCAL ] HLLY: Loaded 32961 existing records from primary storage. [2025-11-04T19:24:04.527964] [INFO ] [MAIN ] HLLY: Loaded 32961 existing records. [2025-11-04T19:24:04.709230] [WARN ] [MAIN ] HLLY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:24:04.709446] [INFO ] [MAIN ] HLLY: Wiping existing primary data before full refresh. [2025-11-04T19:24:04.709494] [INFO ] [LOCAL ] HLLY: Starting deletion of primary data. [2025-11-04T19:24:04.710065] [DEBUG] [LOCAL ] HLLY: Deleted primary data file. [2025-11-04T19:24:04.710123] [INFO ] [MAIN ] HLLY: Starting full primary refresh. [2025-11-04T19:24:04.980297] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-11. [2025-11-04T19:24:05.089207] [INFO ] [REMOTE ] HLLY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:24:05.152367] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T19:24:05.161407] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-11-04T19:24:05.767113] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-01. [2025-11-04T19:24:06.031523] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T19:24:06.033111] [INFO ] [REMOTE ] HLI: Full fetch complete (45682 records). [2025-11-04T19:24:06.033182] [DEBUG] [VALIDATOR ] HLI: Validating primary data. [2025-11-04T19:24:06.454908] [DEBUG] [VALIDATOR ] HLI: All expected trading days are present. [2025-11-04T19:24:06.455464] [DEBUG] [VALIDATOR ] HLI: Primary data validation passed. [2025-11-04T19:24:06.456512] [INFO ] [MAIN ] HLI: Fetched primary data is valid. [2025-11-04T19:24:06.456592] [DEBUG] [LOCAL ] HLI: Saving primary data. [2025-11-04T19:24:06.458231] [DEBUG] [LOCAL ] HLI: Trimmed 45682 records to 30730 (keeping 200 days of data) [2025-11-04T19:24:06.570523] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-11. [2025-11-04T19:24:06.593531] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-11-04T19:24:06.709577] [DEBUG] [LOCAL ] HLI: Primary data saved (30730 records). [2025-11-04T19:24:06.709864] [INFO ] [MAIN ] HLI: Full history saved. [2025-11-04T19:24:06.709993] [INFO ] [MAIN ] Update Progress: 1026/2367 symbols completed. HLI took 29.8s. Estimated remaining: 3h 8m 17s. Rate: 7.1 symbols/min [2025-11-04T19:24:06.922978] [INFO ] [MAIN ] HLN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:24:06.977728] [DEBUG] [LOCAL ] HLN: Loaded 53091 existing records from primary storage. [2025-11-04T19:24:06.978168] [INFO ] [MAIN ] HLN: Loaded 53091 existing records. [2025-11-04T19:24:07.179817] [WARN ] [MAIN ] HLN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:24:07.180165] [INFO ] [MAIN ] HLN: Wiping existing primary data before full refresh. [2025-11-04T19:24:07.180227] [INFO ] [LOCAL ] HLN: Starting deletion of primary data. [2025-11-04T19:24:07.181978] [DEBUG] [LOCAL ] HLN: Deleted primary data file. [2025-11-04T19:24:07.182052] [INFO ] [MAIN ] HLN: Starting full primary refresh. [2025-11-04T19:24:07.377287] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-10. [2025-11-04T19:24:07.486593] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T19:24:07.529539] [INFO ] [REMOTE ] HLIO: Full fetch complete (37654 records). [2025-11-04T19:24:07.552764] [DEBUG] [VALIDATOR ] HLIO: Validating primary data. [2025-11-04T19:24:07.873096] [INFO ] [REMOTE ] HLN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:24:08.185173] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-02. [2025-11-04T19:24:08.257972] [DEBUG] [VALIDATOR ] HLIO: All expected trading days are present. [2025-11-04T19:24:08.258087] [DEBUG] [VALIDATOR ] HLIO: Primary data validation passed. [2025-11-04T19:24:08.259071] [INFO ] [MAIN ] HLIO: Fetched primary data is valid. [2025-11-04T19:24:08.259155] [DEBUG] [LOCAL ] HLIO: Saving primary data. [2025-11-04T19:24:08.260526] [DEBUG] [LOCAL ] HLIO: Trimmed 37654 records to 27882 (keeping 200 days of data) [2025-11-04T19:24:08.474815] [DEBUG] [LOCAL ] HLIO: Primary data saved (27882 records). [2025-11-04T19:24:08.475008] [INFO ] [MAIN ] HLIO: Full history saved. [2025-11-04T19:24:08.475121] [INFO ] [MAIN ] Update Progress: 1027/2367 symbols completed. HLIO took 28.2s. Estimated remaining: 3h 7m 60s. Rate: 7.1 symbols/min [2025-11-04T19:24:08.732014] [INFO ] [MAIN ] HLT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:24:08.788695] [DEBUG] [LOCAL ] HLT: Loaded 50355 existing records from primary storage. [2025-11-04T19:24:08.789197] [INFO ] [MAIN ] HLT: Loaded 50355 existing records. [2025-11-04T19:24:08.970829] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-01. [2025-11-04T19:24:09.038135] [WARN ] [MAIN ] HLT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:24:09.050424] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T19:24:09.049706] [INFO ] [MAIN ] HLT: Wiping existing primary data before full refresh. [2025-11-04T19:24:09.050696] [INFO ] [LOCAL ] HLT: Starting deletion of primary data. [2025-11-04T19:24:09.058198] [DEBUG] [LOCAL ] HLT: Deleted primary data file. [2025-11-04T19:24:09.058279] [INFO ] [MAIN ] HLT: Starting full primary refresh. [2025-11-04T19:24:09.417790] [INFO ] [REMOTE ] HLT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:24:09.557964] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-10 (2.18 seconds). [2025-11-04T19:24:09.767086] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-11. [2025-11-04T19:24:10.464249] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-11-04T19:24:10.567006] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-02. [2025-11-04T19:24:11.258125] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-11 (1.49 seconds). [2025-11-04T19:24:11.261233] [INFO ] [REMOTE ] HL: Full fetch complete (77839 records). [2025-11-04T19:24:11.261424] [DEBUG] [VALIDATOR ] HL: Validating primary data. [2025-11-04T19:24:11.373996] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-01. [2025-11-04T19:24:11.702297] [DEBUG] [VALIDATOR ] HL: All expected trading days are present. [2025-11-04T19:24:11.702448] [DEBUG] [VALIDATOR ] HL: Primary data validation passed. [2025-11-04T19:24:11.704118] [INFO ] [MAIN ] HL: Fetched primary data is valid. [2025-11-04T19:24:11.704199] [DEBUG] [LOCAL ] HL: Saving primary data. [2025-11-04T19:24:11.707042] [DEBUG] [LOCAL ] HL: Trimmed 77839 records to 53677 (keeping 200 days of data) [2025-11-04T19:24:11.982145] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-11-04T19:24:12.115958] [DEBUG] [LOCAL ] HL: Primary data saved (53677 records). [2025-11-04T19:24:12.116168] [INFO ] [MAIN ] HL: Full history saved. [2025-11-04T19:24:12.116279] [INFO ] [MAIN ] Update Progress: 1028/2367 symbols completed. HL took 53.4s. Estimated remaining: 3h 7m 45s. Rate: 7.1 symbols/min [2025-11-04T19:24:12.177209] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-03. [2025-11-04T19:24:12.305513] [INFO ] [MAIN ] HLX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:24:12.353695] [DEBUG] [LOCAL ] HLX: Loaded 47136 existing records from primary storage. [2025-11-04T19:24:12.354160] [INFO ] [MAIN ] HLX: Loaded 47136 existing records. [2025-11-04T19:24:12.547476] [WARN ] [MAIN ] HLX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:24:12.547820] [INFO ] [MAIN ] HLX: Wiping existing primary data before full refresh. [2025-11-04T19:24:12.547876] [INFO ] [LOCAL ] HLX: Starting deletion of primary data. [2025-11-04T19:24:12.548736] [DEBUG] [LOCAL ] HLX: Deleted primary data file. [2025-11-04T19:24:12.548796] [INFO ] [MAIN ] HLX: Starting full primary refresh. [2025-11-04T19:24:12.947721] [INFO ] [REMOTE ] HLX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:24:12.976205] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-03. [2025-11-04T19:24:12.982818] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-11-04T19:24:13.202128] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-11-04T19:24:13.767046] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-04. [2025-11-04T19:24:14.380854] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T19:24:14.567028] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-04. [2025-11-04T19:24:14.862308] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T19:24:15.367037] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-05. [2025-11-04T19:24:16.057100] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-11-04T19:24:16.167041] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-02. [2025-11-04T19:24:16.255047] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T19:24:16.967134] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-06. [2025-11-04T19:24:17.404432] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T19:24:17.767013] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-03. [2025-11-04T19:24:17.817636] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T19:24:18.567089] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-01. [2025-11-04T19:24:19.270396] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-11-04T19:24:19.367031] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-04. [2025-11-04T19:24:19.879513] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T19:24:20.167077] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-05. [2025-11-04T19:24:20.714503] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:24:20.967081] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-05. [2025-11-04T19:24:21.760889] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-11-04T19:24:21.774068] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-02. [2025-11-04T19:24:22.567019] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-06. [2025-11-04T19:24:22.613484] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-11-04T19:24:22.836198] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T19:24:23.367047] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-07. [2025-11-04T19:24:23.893893] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T19:24:24.167085] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-07. [2025-11-04T19:24:24.375083] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-11-04T19:24:24.967061] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-03. [2025-11-04T19:24:25.636772] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-11-04T19:24:25.767043] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-06. [2025-11-04T19:24:26.354861] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T19:24:26.567027] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-08. [2025-11-04T19:24:27.231280] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T19:24:27.367009] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-04. [2025-11-04T19:24:27.989841] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T19:24:28.167050] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-09. [2025-11-04T19:24:28.545285] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T19:24:28.967074] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-07. [2025-11-04T19:24:29.603115] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T19:24:29.767048] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-10. [2025-11-04T19:24:30.342011] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T19:24:30.567090] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-05. [2025-11-04T19:24:31.106007] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-11-04T19:24:31.367060] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-11. [2025-11-04T19:24:31.879811] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T19:24:32.167042] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-06. [2025-11-04T19:24:32.659196] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-11 (1.29 seconds). [2025-11-04T19:24:32.661170] [INFO ] [REMOTE ] HLN: Full fetch complete (76999 records). [2025-11-04T19:24:32.661300] [DEBUG] [VALIDATOR ] HLN: Validating primary data. [2025-11-04T19:24:32.972253] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-08. [2025-11-04T19:24:33.066200] [DEBUG] [VALIDATOR ] HLN: All expected trading days are present. [2025-11-04T19:24:33.066290] [DEBUG] [VALIDATOR ] HLN: Primary data validation passed. [2025-11-04T19:24:33.067286] [INFO ] [MAIN ] HLN: Fetched primary data is valid. [2025-11-04T19:24:33.067351] [DEBUG] [LOCAL ] HLN: Saving primary data. [2025-11-04T19:24:33.068960] [DEBUG] [LOCAL ] HLN: Trimmed 76999 records to 53076 (keeping 200 days of data) [2025-11-04T19:24:33.384701] [DEBUG] [LOCAL ] HLN: Primary data saved (53076 records). [2025-11-04T19:24:33.384953] [INFO ] [MAIN ] HLN: Full history saved. [2025-11-04T19:24:33.385176] [INFO ] [MAIN ] Update Progress: 1029/2367 symbols completed. HLN took 26.7s. Estimated remaining: 3h 7m 54s. Rate: 7.1 symbols/min [2025-11-04T19:24:33.540322] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-11-04T19:24:33.614917] [INFO ] [MAIN ] HMC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:24:33.654285] [DEBUG] [LOCAL ] HMC: Loaded 46056 existing records from primary storage. [2025-11-04T19:24:33.654623] [INFO ] [MAIN ] HMC: Loaded 46056 existing records. [2025-11-04T19:24:33.772134] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-08. [2025-11-04T19:24:33.836808] [WARN ] [MAIN ] HMC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:24:33.837024] [INFO ] [MAIN ] HMC: Wiping existing primary data before full refresh. [2025-11-04T19:24:33.837072] [INFO ] [LOCAL ] HMC: Starting deletion of primary data. [2025-11-04T19:24:33.837815] [DEBUG] [LOCAL ] HMC: Deleted primary data file. [2025-11-04T19:24:33.837874] [INFO ] [MAIN ] HMC: Starting full primary refresh. [2025-11-04T19:24:34.209188] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T19:24:34.312377] [INFO ] [REMOTE ] HMC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:24:34.567040] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-01. [2025-11-04T19:24:35.367023] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-09. [2025-11-04T19:24:35.514114] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-11-04T19:24:36.146133] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-11-04T19:24:36.166996] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-02. [2025-11-04T19:24:36.415405] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T19:24:36.967053] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-09. [2025-11-04T19:24:37.389724] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T19:24:37.767044] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-03. [2025-11-04T19:24:38.283493] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-11-04T19:24:38.567010] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-07. [2025-11-04T19:24:39.111400] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T19:24:39.367043] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-10. [2025-11-04T19:24:39.988367] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-11-04T19:24:40.167038] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-08. [2025-11-04T19:24:40.750310] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T19:24:40.967057] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-11. [2025-11-04T19:24:41.355247] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T19:24:41.459277] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-11 (0.49 seconds). [2025-11-04T19:24:41.461243] [INFO ] [REMOTE ] HLT: Full fetch complete (73695 records). [2025-11-04T19:24:41.461339] [DEBUG] [VALIDATOR ] HLT: Validating primary data. [2025-11-04T19:24:41.772334] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-04. [2025-11-04T19:24:41.888124] [DEBUG] [VALIDATOR ] HLT: All expected trading days are present. [2025-11-04T19:24:41.888213] [DEBUG] [VALIDATOR ] HLT: Primary data validation passed. [2025-11-04T19:24:41.889203] [INFO ] [MAIN ] HLT: Fetched primary data is valid. [2025-11-04T19:24:41.889294] [DEBUG] [LOCAL ] HLT: Saving primary data. [2025-11-04T19:24:41.890798] [DEBUG] [LOCAL ] HLT: Trimmed 73695 records to 50357 (keeping 200 days of data) [2025-11-04T19:24:42.200804] [DEBUG] [LOCAL ] HLT: Primary data saved (50357 records). [2025-11-04T19:24:42.200943] [INFO ] [MAIN ] HLT: Full history saved. [2025-11-04T19:24:42.201021] [INFO ] [MAIN ] Update Progress: 1030/2367 symbols completed. HLT took 33.7s. Estimated remaining: 3h 7m 46s. Rate: 7.1 symbols/min [2025-11-04T19:24:42.377506] [INFO ] [MAIN ] HMN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:24:42.397998] [DEBUG] [LOCAL ] HMN: Loaded 22979 existing records from primary storage. [2025-11-04T19:24:42.398253] [INFO ] [MAIN ] HMN: Loaded 22979 existing records. [2025-11-04T19:24:42.576094] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-09. [2025-11-04T19:24:42.581359] [WARN ] [MAIN ] HMN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:24:42.582548] [INFO ] [MAIN ] HMN: Wiping existing primary data before full refresh. [2025-11-04T19:24:42.582599] [INFO ] [LOCAL ] HMN: Starting deletion of primary data. [2025-11-04T19:24:42.583099] [DEBUG] [LOCAL ] HMN: Deleted primary data file. [2025-11-04T19:24:42.583158] [INFO ] [MAIN ] HMN: Starting full primary refresh. [2025-11-04T19:24:42.937530] [INFO ] [REMOTE ] HMN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:24:43.089219] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T19:24:43.367045] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-05. [2025-11-04T19:24:43.750376] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T19:24:44.167077] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-10. [2025-11-04T19:24:44.576486] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T19:24:44.967101] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-06. [2025-11-04T19:24:45.226918] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-11-04T19:24:45.767050] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-01. [2025-11-04T19:24:46.208509] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T19:24:46.453293] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T19:24:46.567085] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-10. [2025-11-04T19:24:47.367049] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-02. [2025-11-04T19:24:47.959101] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T19:24:48.136210] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-11-04T19:24:48.167012] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-03. [2025-11-04T19:24:48.847361] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-11-04T19:24:48.967040] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-04. [2025-11-04T19:24:49.774735] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-11. [2025-11-04T19:24:49.778872] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-11-04T19:24:50.567083] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-11. [2025-11-04T19:24:50.737497] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-11 (0.96 seconds). [2025-11-04T19:24:50.739023] [INFO ] [REMOTE ] HLLY: Full fetch complete (45960 records). [2025-11-04T19:24:50.739104] [DEBUG] [VALIDATOR ] HLLY: Validating primary data. [2025-11-04T19:24:51.110942] [DEBUG] [VALIDATOR ] HLLY: All expected trading days are present. [2025-11-04T19:24:51.111021] [DEBUG] [VALIDATOR ] HLLY: Primary data validation passed. [2025-11-04T19:24:51.111662] [INFO ] [MAIN ] HLLY: Fetched primary data is valid. [2025-11-04T19:24:51.111722] [DEBUG] [LOCAL ] HLLY: Saving primary data. [2025-11-04T19:24:51.112724] [DEBUG] [LOCAL ] HLLY: Trimmed 45960 records to 32989 (keeping 200 days of data) [2025-11-04T19:24:51.287664] [DEBUG] [LOCAL ] HLLY: Primary data saved (32989 records). [2025-11-04T19:24:51.287793] [INFO ] [MAIN ] HLLY: Full history saved. [2025-11-04T19:24:51.287870] [INFO ] [MAIN ] Update Progress: 1031/2367 symbols completed. HLLY took 47.0s. Estimated remaining: 3h 7m 38s. Rate: 7.1 symbols/min [2025-11-04T19:24:51.376341] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-07. [2025-11-04T19:24:51.454848] [INFO ] [MAIN ] HMY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:24:51.501042] [DEBUG] [LOCAL ] HMY: Loaded 53153 existing records from primary storage. [2025-11-04T19:24:51.501333] [INFO ] [MAIN ] HMY: Loaded 53153 existing records. [2025-11-04T19:24:51.675724] [WARN ] [MAIN ] HMY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:24:51.675891] [INFO ] [MAIN ] HMY: Wiping existing primary data before full refresh. [2025-11-04T19:24:51.675936] [INFO ] [LOCAL ] HMY: Starting deletion of primary data. [2025-11-04T19:24:51.679709] [DEBUG] [LOCAL ] HMY: Deleted primary data file. [2025-11-04T19:24:51.679816] [INFO ] [MAIN ] HMY: Starting full primary refresh. [2025-11-04T19:24:51.711785] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T19:24:51.779986] [INFO ] [REMOTE ] HLX: Full fetch complete (69038 records). [2025-11-04T19:24:51.782325] [DEBUG] [VALIDATOR ] HLX: Validating primary data. [2025-11-04T19:24:52.176806] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-05. [2025-11-04T19:24:52.322476] [INFO ] [REMOTE ] HMY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:24:52.441811] [DEBUG] [VALIDATOR ] HLX: All expected trading days are present. [2025-11-04T19:24:52.441923] [DEBUG] [VALIDATOR ] HLX: Primary data validation passed. [2025-11-04T19:24:52.442879] [INFO ] [MAIN ] HLX: Fetched primary data is valid. [2025-11-04T19:24:52.442946] [DEBUG] [LOCAL ] HLX: Saving primary data. [2025-11-04T19:24:52.444334] [DEBUG] [LOCAL ] HLX: Trimmed 69038 records to 47131 (keeping 200 days of data) [2025-11-04T19:24:52.702454] [DEBUG] [LOCAL ] HLX: Primary data saved (47131 records). [2025-11-04T19:24:52.702593] [INFO ] [MAIN ] HLX: Full history saved. [2025-11-04T19:24:52.702695] [INFO ] [MAIN ] Update Progress: 1032/2367 symbols completed. HLX took 40.6s. Estimated remaining: 3h 7m 21s. Rate: 7.1 symbols/min [2025-11-04T19:24:52.878374] [INFO ] [MAIN ] HNGE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:24:52.878620] [DEBUG] [LOCAL ] HNGE: No existing data file found in primary storage. [2025-11-04T19:24:52.878933] [WARN ] [MAIN ] HNGE: No existing data found, performing full refresh instead. [2025-11-04T19:24:52.879076] [INFO ] [MAIN ] HNGE: Wiping existing primary data before full refresh. [2025-11-04T19:24:52.879204] [INFO ] [LOCAL ] HNGE: Starting deletion of primary data. [2025-11-04T19:24:52.879287] [DEBUG] [LOCAL ] HNGE: No primary data file found to delete. [2025-11-04T19:24:52.879343] [INFO ] [MAIN ] HNGE: Starting full primary refresh. [2025-11-04T19:24:52.976610] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-01. [2025-11-04T19:24:53.090006] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-11-04T19:24:53.289990] [INFO ] [REMOTE ] HNGE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:24:53.309534] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T19:24:53.767053] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-06. [2025-11-04T19:24:54.569119] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-08. [2025-11-04T19:24:54.573435] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T19:24:54.688557] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-11-04T19:24:55.367073] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-02. [2025-11-04T19:24:55.878185] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-11-04T19:24:56.167083] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-09. [2025-11-04T19:24:56.608232] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T19:24:56.967058] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-03. [2025-11-04T19:24:57.426540] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T19:24:57.767053] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-10. [2025-11-04T19:24:58.567104] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-01. [2025-11-04T19:24:58.719766] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-11-04T19:24:58.730733] [DEBUG] [REMOTE ] HNGE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T19:24:58.731267] [WARN ] [REMOTE ] HNGE: No data returned for month 2025-01 [2025-11-04T19:24:58.731465] [FATAL] [REMOTE ] HNGE: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:24:58.731843] [FATAL] [MAIN ] HNGE: Failed to fetch primary data. [2025-11-04T19:24:58.731910] [WARN ] [MAIN ] HNGE: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:24:58.731983] [INFO ] [LOCAL ] HNGE: Starting deletion of primary data. [2025-11-04T19:24:58.732045] [DEBUG] [LOCAL ] HNGE: No primary data file found to delete. [2025-11-04T19:24:58.732080] [INFO ] [LOCAL ] HNGE: Starting deletion of derivative data. [2025-11-04T19:24:58.732135] [DEBUG] [LOCAL ] HNGE: No derivative data file found to delete. [2025-11-04T19:24:58.732179] [INFO ] [MAIN ] Update Progress: 1033/2367 symbols completed. HNGE took 6.0s. Estimated remaining: 3h 7m 9s. Rate: 7.1 symbols/min [2025-11-04T19:24:58.910088] [INFO ] [MAIN ] HNI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:24:58.936913] [DEBUG] [LOCAL ] HNI: Loaded 30250 existing records from primary storage. [2025-11-04T19:24:58.937189] [INFO ] [MAIN ] HNI: Loaded 30250 existing records. [2025-11-04T19:24:59.120534] [WARN ] [MAIN ] HNI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:24:59.120764] [INFO ] [MAIN ] HNI: Wiping existing primary data before full refresh. [2025-11-04T19:24:59.120888] [INFO ] [LOCAL ] HNI: Starting deletion of primary data. [2025-11-04T19:24:59.121563] [DEBUG] [LOCAL ] HNI: Deleted primary data file. [2025-11-04T19:24:59.121631] [INFO ] [MAIN ] HNI: Starting full primary refresh. [2025-11-04T19:24:59.367621] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-07. [2025-11-04T19:24:59.381943] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-11-04T19:24:59.530762] [INFO ] [REMOTE ] HNI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:25:00.167044] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-01. [2025-11-04T19:25:00.881950] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T19:25:00.967028] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-11. [2025-11-04T19:25:01.083948] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-11-04T19:25:01.767039] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-02. [2025-11-04T19:25:02.567063] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-08. [2025-11-04T19:25:02.644459] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-11 (1.68 seconds). [2025-11-04T19:25:02.646377] [INFO ] [REMOTE ] HMC: Full fetch complete (68320 records). [2025-11-04T19:25:02.646525] [DEBUG] [VALIDATOR ] HMC: Validating primary data. [2025-11-04T19:25:03.067986] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T19:25:03.100174] [DEBUG] [VALIDATOR ] HMC: All expected trading days are present. [2025-11-04T19:25:03.100436] [DEBUG] [VALIDATOR ] HMC: Primary data validation passed. [2025-11-04T19:25:03.102073] [INFO ] [MAIN ] HMC: Fetched primary data is valid. [2025-11-04T19:25:03.102140] [DEBUG] [LOCAL ] HMC: Saving primary data. [2025-11-04T19:25:03.103397] [DEBUG] [LOCAL ] HMC: Trimmed 68320 records to 46005 (keeping 200 days of data) [2025-11-04T19:25:03.381974] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-11-04T19:25:03.381114] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-03. [2025-11-04T19:25:03.412287] [DEBUG] [LOCAL ] HMC: Primary data saved (46005 records). [2025-11-04T19:25:03.412408] [INFO ] [MAIN ] HMC: Full history saved. [2025-11-04T19:25:03.412487] [INFO ] [MAIN ] Update Progress: 1034/2367 symbols completed. HMC took 30.0s. Estimated remaining: 3h 6m 56s. Rate: 7.1 symbols/min [2025-11-04T19:25:03.609112] [INFO ] [MAIN ] HOG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:25:03.657781] [DEBUG] [LOCAL ] HOG: Loaded 50439 existing records from primary storage. [2025-11-04T19:25:03.658129] [INFO ] [MAIN ] HOG: Loaded 50439 existing records. [2025-11-04T19:25:03.880322] [WARN ] [MAIN ] HOG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:25:03.880556] [INFO ] [MAIN ] HOG: Wiping existing primary data before full refresh. [2025-11-04T19:25:03.880635] [INFO ] [LOCAL ] HOG: Starting deletion of primary data. [2025-11-04T19:25:03.881442] [DEBUG] [LOCAL ] HOG: Deleted primary data file. [2025-11-04T19:25:03.881500] [INFO ] [MAIN ] HOG: Starting full primary refresh. [2025-11-04T19:25:04.172228] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-09. [2025-11-04T19:25:04.276218] [INFO ] [REMOTE ] HOG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:25:04.450719] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T19:25:04.887871] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-11-04T19:25:04.967098] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-04. [2025-11-04T19:25:05.767009] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-10. [2025-11-04T19:25:06.009991] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-11-04T19:25:06.521693] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-11-04T19:25:06.567072] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-05. [2025-11-04T19:25:07.367035] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-04. [2025-11-04T19:25:07.433875] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T19:25:08.167082] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-06. [2025-11-04T19:25:08.967034] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-11. [2025-11-04T19:25:09.086724] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-11-04T19:25:09.234972] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-11-04T19:25:09.649743] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T19:25:09.650962] [INFO ] [REMOTE ] HMN: Full fetch complete (32029 records). [2025-11-04T19:25:09.651033] [DEBUG] [VALIDATOR ] HMN: Validating primary data. [2025-11-04T19:25:09.772241] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-05. [2025-11-04T19:25:10.022536] [DEBUG] [VALIDATOR ] HMN: All expected trading days are present. [2025-11-04T19:25:10.022614] [DEBUG] [VALIDATOR ] HMN: Primary data validation passed. [2025-11-04T19:25:10.023187] [INFO ] [MAIN ] HMN: Fetched primary data is valid. [2025-11-04T19:25:10.023247] [DEBUG] [LOCAL ] HMN: Saving primary data. [2025-11-04T19:25:10.024032] [DEBUG] [LOCAL ] HMN: Trimmed 32029 records to 22959 (keeping 200 days of data) [2025-11-04T19:25:10.156917] [DEBUG] [LOCAL ] HMN: Primary data saved (22959 records). [2025-11-04T19:25:10.157036] [INFO ] [MAIN ] HMN: Full history saved. [2025-11-04T19:25:10.157110] [INFO ] [MAIN ] Update Progress: 1035/2367 symbols completed. HMN took 28.0s. Estimated remaining: 3h 6m 45s. Rate: 7.1 symbols/min [2025-11-04T19:25:10.328564] [INFO ] [MAIN ] HOMB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:25:10.367158] [DEBUG] [LOCAL ] HOMB: Loaded 43479 existing records from primary storage. [2025-11-04T19:25:10.367475] [INFO ] [MAIN ] HOMB: Loaded 43479 existing records. [2025-11-04T19:25:10.543635] [WARN ] [MAIN ] HOMB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:25:10.543903] [INFO ] [MAIN ] HOMB: Wiping existing primary data before full refresh. [2025-11-04T19:25:10.543951] [INFO ] [LOCAL ] HOMB: Starting deletion of primary data. [2025-11-04T19:25:10.544746] [DEBUG] [LOCAL ] HOMB: Deleted primary data file. [2025-11-04T19:25:10.544805] [INFO ] [MAIN ] HOMB: Starting full primary refresh. [2025-11-04T19:25:10.574837] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-07. [2025-11-04T19:25:10.904854] [INFO ] [REMOTE ] HOMB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:25:11.363164] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-11-04T19:25:11.373354] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-01. [2025-11-04T19:25:11.443572] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-11-04T19:25:12.167051] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-08. [2025-11-04T19:25:12.698390] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T19:25:12.967048] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-01. [2025-11-04T19:25:13.331252] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T19:25:13.767073] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-06. [2025-11-04T19:25:14.246075] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T19:25:14.567065] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-09. [2025-11-04T19:25:15.060865] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-11-04T19:25:15.367035] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-07. [2025-11-04T19:25:15.735508] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T19:25:16.167052] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-10. [2025-11-04T19:25:16.826468] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T19:25:16.967065] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-08. [2025-11-04T19:25:17.322594] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T19:25:17.767090] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-02. [2025-11-04T19:25:18.572202] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-02. [2025-11-04T19:25:18.586731] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-11-04T19:25:18.877188] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T19:25:19.367034] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-03. [2025-11-04T19:25:19.624199] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T19:25:20.167093] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-03. [2025-11-04T19:25:20.638321] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T19:25:20.967108] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-04. [2025-11-04T19:25:21.263074] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T19:25:21.767063] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-09. [2025-11-04T19:25:22.463083] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-11-04T19:25:22.567001] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-05. [2025-11-04T19:25:23.367025] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-11. [2025-11-04T19:25:23.423683] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-11-04T19:25:23.935957] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T19:25:24.167056] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-10. [2025-11-04T19:25:24.280570] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T19:25:24.282151] [INFO ] [REMOTE ] HNI: Full fetch complete (42749 records). [2025-11-04T19:25:24.282226] [DEBUG] [VALIDATOR ] HNI: Validating primary data. [2025-11-04T19:25:24.657906] [DEBUG] [VALIDATOR ] HNI: All expected trading days are present. [2025-11-04T19:25:24.657980] [DEBUG] [VALIDATOR ] HNI: Primary data validation passed. [2025-11-04T19:25:24.658689] [INFO ] [MAIN ] HNI: Fetched primary data is valid. [2025-11-04T19:25:24.658763] [DEBUG] [LOCAL ] HNI: Saving primary data. [2025-11-04T19:25:24.659622] [DEBUG] [LOCAL ] HNI: Trimmed 42749 records to 30310 (keeping 200 days of data) [2025-11-04T19:25:24.836210] [DEBUG] [LOCAL ] HNI: Primary data saved (30310 records). [2025-11-04T19:25:24.836335] [INFO ] [MAIN ] HNI: Full history saved. [2025-11-04T19:25:24.836411] [INFO ] [MAIN ] Update Progress: 1036/2367 symbols completed. HNI took 26.1s. Estimated remaining: 3h 6m 45s. Rate: 7.1 symbols/min [2025-11-04T19:25:24.979961] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-06. [2025-11-04T19:25:25.014452] [INFO ] [MAIN ] HOUS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:25:25.052986] [DEBUG] [LOCAL ] HOUS: Loaded 45965 existing records from primary storage. [2025-11-04T19:25:25.053286] [INFO ] [MAIN ] HOUS: Loaded 45965 existing records. [2025-11-04T19:25:25.231558] [WARN ] [MAIN ] HOUS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:25:25.231777] [INFO ] [MAIN ] HOUS: Wiping existing primary data before full refresh. [2025-11-04T19:25:25.231826] [INFO ] [LOCAL ] HOUS: Starting deletion of primary data. [2025-11-04T19:25:25.232522] [DEBUG] [LOCAL ] HOUS: Deleted primary data file. [2025-11-04T19:25:25.232578] [INFO ] [MAIN ] HOUS: Starting full primary refresh. [2025-11-04T19:25:25.590353] [INFO ] [REMOTE ] HOUS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:25:25.772152] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-01. [2025-11-04T19:25:25.788778] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-11-04T19:25:26.304307] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T19:25:26.567054] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-11. [2025-11-04T19:25:26.930924] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T19:25:27.367034] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-04. [2025-11-04T19:25:28.099326] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-11 (1.53 seconds). [2025-11-04T19:25:28.101416] [INFO ] [REMOTE ] HMY: Full fetch complete (77174 records). [2025-11-04T19:25:28.101515] [DEBUG] [VALIDATOR ] HMY: Validating primary data. [2025-11-04T19:25:28.176310] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-07. [2025-11-04T19:25:28.498153] [DEBUG] [VALIDATOR ] HMY: All expected trading days are present. [2025-11-04T19:25:28.498232] [DEBUG] [VALIDATOR ] HMY: Primary data validation passed. [2025-11-04T19:25:28.499129] [INFO ] [MAIN ] HMY: Fetched primary data is valid. [2025-11-04T19:25:28.499196] [DEBUG] [LOCAL ] HMY: Saving primary data. [2025-11-04T19:25:28.500520] [DEBUG] [LOCAL ] HMY: Trimmed 77174 records to 53155 (keeping 200 days of data) [2025-11-04T19:25:28.770255] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T19:25:28.873117] [DEBUG] [LOCAL ] HMY: Primary data saved (53155 records). [2025-11-04T19:25:28.873259] [INFO ] [MAIN ] HMY: Full history saved. [2025-11-04T19:25:28.873339] [INFO ] [MAIN ] Update Progress: 1037/2367 symbols completed. HMY took 37.6s. Estimated remaining: 3h 6m 31s. Rate: 7.1 symbols/min [2025-11-04T19:25:28.967032] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-05. [2025-11-04T19:25:29.482096] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T19:25:29.767043] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-02. [2025-11-04T19:25:30.095720] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T19:25:30.513622] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-11-04T19:25:30.567016] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-06. [2025-11-04T19:25:31.367059] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-08. [2025-11-04T19:25:31.458943] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T19:25:32.167080] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-07. [2025-11-04T19:25:32.594984] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T19:25:32.967022] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-09. [2025-11-04T19:25:33.553936] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-11-04T19:25:33.767076] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-03. [2025-11-04T19:25:34.394342] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T19:25:34.567086] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-10. [2025-11-04T19:25:34.799549] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-11-04T19:25:35.367089] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-04. [2025-11-04T19:25:35.855096] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T19:25:36.167145] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-11. [2025-11-04T19:25:36.591184] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T19:25:36.967035] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-05. [2025-11-04T19:25:37.505303] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-11 (1.34 seconds). [2025-11-04T19:25:37.507364] [INFO ] [REMOTE ] HOG: Full fetch complete (74279 records). [2025-11-04T19:25:37.507481] [DEBUG] [VALIDATOR ] HOG: Validating primary data. [2025-11-04T19:25:37.775060] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-08. [2025-11-04T19:25:37.894604] [DEBUG] [VALIDATOR ] HOG: All expected trading days are present. [2025-11-04T19:25:37.894706] [DEBUG] [VALIDATOR ] HOG: Primary data validation passed. [2025-11-04T19:25:37.895570] [INFO ] [MAIN ] HOG: Fetched primary data is valid. [2025-11-04T19:25:37.895610] [DEBUG] [LOCAL ] HOG: Saving primary data. [2025-11-04T19:25:37.896906] [DEBUG] [LOCAL ] HOG: Trimmed 74279 records to 50446 (keeping 200 days of data) [2025-11-04T19:25:38.154639] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T19:25:38.230521] [DEBUG] [LOCAL ] HOG: Primary data saved (50446 records). [2025-11-04T19:25:38.230682] [INFO ] [MAIN ] HOG: Full history saved. [2025-11-04T19:25:38.230772] [INFO ] [MAIN ] Update Progress: 1038/2367 symbols completed. HOG took 34.8s. Estimated remaining: 3h 6m 24s. Rate: 7.1 symbols/min [2025-11-04T19:25:38.567004] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-06. [2025-11-04T19:25:38.883960] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-11-04T19:25:39.367070] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-09. [2025-11-04T19:25:39.760129] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-11-04T19:25:40.167052] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-07. [2025-11-04T19:25:40.491181] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T19:25:40.967092] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-10. [2025-11-04T19:25:41.421497] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T19:25:41.767050] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-08. [2025-11-04T19:25:42.230620] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-11-04T19:25:42.567109] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-11. [2025-11-04T19:25:43.061470] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T19:25:43.367090] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-09. [2025-11-04T19:25:43.678331] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T19:25:43.680195] [INFO ] [REMOTE ] HOMB: Full fetch complete (64319 records). [2025-11-04T19:25:43.680286] [DEBUG] [VALIDATOR ] HOMB: Validating primary data. [2025-11-04T19:25:44.066062] [DEBUG] [VALIDATOR ] HOMB: All expected trading days are present. [2025-11-04T19:25:44.066131] [DEBUG] [VALIDATOR ] HOMB: Primary data validation passed. [2025-11-04T19:25:44.067000] [INFO ] [MAIN ] HOMB: Fetched primary data is valid. [2025-11-04T19:25:44.067064] [DEBUG] [LOCAL ] HOMB: Saving primary data. [2025-11-04T19:25:44.068389] [DEBUG] [LOCAL ] HOMB: Trimmed 64319 records to 43455 (keeping 200 days of data) [2025-11-04T19:25:44.323094] [DEBUG] [LOCAL ] HOMB: Primary data saved (43455 records). [2025-11-04T19:25:44.323230] [INFO ] [MAIN ] HOMB: Full history saved. [2025-11-04T19:25:44.323308] [INFO ] [MAIN ] Update Progress: 1039/2367 symbols completed. HOMB took 34.2s. Estimated remaining: 3h 6m 12s. Rate: 7.1 symbols/min [2025-11-04T19:25:44.703747] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T19:25:44.714595] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-10. [2025-11-04T19:25:46.009244] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T19:25:46.021450] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-11. [2025-11-04T19:25:47.146949] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:25:47.149092] [INFO ] [REMOTE ] HOUS: Full fetch complete (64996 records). [2025-11-04T19:25:47.149185] [DEBUG] [VALIDATOR ] HOUS: Validating primary data. [2025-11-04T19:25:47.620790] [DEBUG] [VALIDATOR ] HOUS: All expected trading days are present. [2025-11-04T19:25:47.621110] [DEBUG] [VALIDATOR ] HOUS: Primary data validation passed. [2025-11-04T19:25:47.622258] [INFO ] [MAIN ] HOUS: Fetched primary data is valid. [2025-11-04T19:25:47.622352] [DEBUG] [LOCAL ] HOUS: Saving primary data. [2025-11-04T19:25:47.623902] [DEBUG] [LOCAL ] HOUS: Trimmed 64996 records to 46076 (keeping 200 days of data) [2025-11-04T19:25:47.917222] [DEBUG] [LOCAL ] HOUS: Primary data saved (46076 records). [2025-11-04T19:25:47.917398] [INFO ] [MAIN ] HOUS: Full history saved. [2025-11-04T19:25:47.917512] [INFO ] [MAIN ] Update Progress: 1040/2367 symbols completed. HOUS took 23.1s. Estimated remaining: 3h 5m 58s. Rate: 7.1 symbols/min [2025-11-04T19:25:48.547895] [INFO ] [MAIN ] HP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:25:48.554727] [INFO ] [MAIN ] HPF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:25:48.684446] [INFO ] [MAIN ] HPE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:25:48.682290] [INFO ] [MAIN ] HOV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:25:48.693562] [DEBUG] [LOCAL ] HPF: Loaded 7279 existing records from primary storage. [2025-11-04T19:25:48.693915] [INFO ] [MAIN ] HPF: Loaded 7279 existing records. [2025-11-04T19:25:48.837385] [DEBUG] [LOCAL ] HOV: Loaded 13685 existing records from primary storage. [2025-11-04T19:25:48.862200] [INFO ] [MAIN ] HOV: Loaded 13685 existing records. [2025-11-04T19:25:49.051417] [DEBUG] [LOCAL ] HP: Loaded 51133 existing records from primary storage. [2025-11-04T19:25:49.046166] [WARN ] [MAIN ] HPF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:25:49.082700] [INFO ] [MAIN ] HP: Loaded 51133 existing records. [2025-11-04T19:25:49.106165] [DEBUG] [LOCAL ] HPE: Loaded 53697 existing records from primary storage. [2025-11-04T19:25:49.082891] [INFO ] [MAIN ] HPF: Wiping existing primary data before full refresh. [2025-11-04T19:25:49.120038] [INFO ] [LOCAL ] HPF: Starting deletion of primary data. [2025-11-04T19:25:49.132358] [DEBUG] [LOCAL ] HPF: Deleted primary data file. [2025-11-04T19:25:49.132292] [INFO ] [MAIN ] HPE: Loaded 53697 existing records. [2025-11-04T19:25:49.158373] [INFO ] [MAIN ] HPF: Starting full primary refresh. [2025-11-04T19:25:49.408350] [WARN ] [MAIN ] HOV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:25:49.512392] [INFO ] [MAIN ] HOV: Wiping existing primary data before full refresh. [2025-11-04T19:25:49.512667] [INFO ] [LOCAL ] HOV: Starting deletion of primary data. [2025-11-04T19:25:49.568090] [DEBUG] [LOCAL ] HOV: Deleted primary data file. [2025-11-04T19:25:49.665636] [INFO ] [MAIN ] HOV: Starting full primary refresh. [2025-11-04T19:25:49.744240] [WARN ] [MAIN ] HPE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:25:49.763964] [INFO ] [MAIN ] HPE: Wiping existing primary data before full refresh. [2025-11-04T19:25:49.773349] [INFO ] [LOCAL ] HPE: Starting deletion of primary data. [2025-11-04T19:25:49.803533] [WARN ] [MAIN ] HP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:25:49.903139] [DEBUG] [LOCAL ] HPE: Deleted primary data file. [2025-11-04T19:25:49.913418] [INFO ] [MAIN ] HP: Wiping existing primary data before full refresh. [2025-11-04T19:25:49.918710] [INFO ] [MAIN ] HPE: Starting full primary refresh. [2025-11-04T19:25:49.944066] [INFO ] [LOCAL ] HP: Starting deletion of primary data. [2025-11-04T19:25:49.960011] [DEBUG] [LOCAL ] HP: Deleted primary data file. [2025-11-04T19:25:49.988896] [INFO ] [MAIN ] HP: Starting full primary refresh. [2025-11-04T19:25:50.309093] [INFO ] [REMOTE ] HPF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:25:50.361979] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-01. [2025-11-04T19:25:50.867300] [INFO ] [REMOTE ] HOV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:25:50.884469] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-01. [2025-11-04T19:25:51.013175] [INFO ] [REMOTE ] HP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:25:51.014615] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-01. [2025-11-04T19:25:51.037294] [INFO ] [REMOTE ] HPE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:25:51.038254] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-01. [2025-11-04T19:25:51.167542] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T19:25:51.169932] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-02. [2025-11-04T19:25:51.582111] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-11-04T19:25:51.584095] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-03. [2025-11-04T19:25:51.630296] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T19:25:51.633017] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-02. [2025-11-04T19:25:51.981481] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T19:25:51.983350] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-04. [2025-11-04T19:25:52.120782] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-11-04T19:25:52.156791] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-11-04T19:25:52.166984] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-03. [2025-11-04T19:25:52.550349] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-11-04T19:25:52.600387] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-11-04T19:25:52.699965] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-11-04T19:25:52.967022] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-02. [2025-11-04T19:25:53.767074] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-04. [2025-11-04T19:25:54.243707] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-11-04T19:25:54.398537] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-11-04T19:25:54.567048] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-03. [2025-11-04T19:25:55.367114] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-02. [2025-11-04T19:25:55.790480] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T19:25:56.167011] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-04. [2025-11-04T19:25:56.729623] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T19:25:56.967083] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-03. [2025-11-04T19:25:57.352747] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T19:25:57.767059] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-05. [2025-11-04T19:25:58.327309] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-11-04T19:25:58.567031] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-05. [2025-11-04T19:25:58.712211] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-11-04T19:25:59.367056] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-05. [2025-11-04T19:26:00.007936] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T19:26:00.012686] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T19:26:00.167078] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-06. [2025-11-04T19:26:00.967048] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-04. [2025-11-04T19:26:01.653879] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-11-04T19:26:01.767136] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-07. [2025-11-04T19:26:02.567071] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-06. [2025-11-04T19:26:02.741718] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-11-04T19:26:03.223261] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T19:26:03.275236] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T19:26:03.367103] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-07. [2025-11-04T19:26:03.966054] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T19:26:04.167039] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-06. [2025-11-04T19:26:04.729039] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-11-04T19:26:04.967037] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-07. [2025-11-04T19:26:05.423316] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-11-04T19:26:05.767040] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-08. [2025-11-04T19:26:06.275077] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-11-04T19:26:06.567058] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-05. [2025-11-04T19:26:07.367067] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-08. [2025-11-04T19:26:07.993369] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-11-04T19:26:08.173086] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-08. [2025-11-04T19:26:08.179040] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-11-04T19:26:08.967117] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-09. [2025-11-04T19:26:09.521084] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-11-04T19:26:09.684501] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-11-04T19:26:09.767055] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-09. [2025-11-04T19:26:10.322424] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T19:26:10.567041] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-10. [2025-11-04T19:26:11.297766] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-11-04T19:26:11.367004] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-06. [2025-11-04T19:26:12.167061] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-11. [2025-11-04T19:26:12.837556] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T19:26:12.850900] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T19:26:12.852771] [INFO ] [REMOTE ] HOV: Full fetch complete (18928 records). [2025-11-04T19:26:12.852879] [DEBUG] [VALIDATOR ] HOV: Validating primary data. [2025-11-04T19:26:12.972121] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-07. [2025-11-04T19:26:13.210559] [DEBUG] [VALIDATOR ] HOV: All expected trading days are present. [2025-11-04T19:26:13.210631] [DEBUG] [VALIDATOR ] HOV: Primary data validation passed. [2025-11-04T19:26:13.211076] [INFO ] [MAIN ] HOV: Fetched primary data is valid. [2025-11-04T19:26:13.211122] [DEBUG] [LOCAL ] HOV: Saving primary data. [2025-11-04T19:26:13.211636] [DEBUG] [LOCAL ] HOV: Trimmed 18928 records to 13696 (keeping 200 days of data) [2025-11-04T19:26:13.291639] [DEBUG] [LOCAL ] HOV: Primary data saved (13696 records). [2025-11-04T19:26:13.291793] [INFO ] [MAIN ] HOV: Full history saved. [2025-11-04T19:26:13.291871] [INFO ] [MAIN ] Update Progress: 1041/2367 symbols completed. HOV took 25.4s. Estimated remaining: 3h 6m 11s. Rate: 7.1 symbols/min [2025-11-04T19:26:13.460375] [INFO ] [MAIN ] HPI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:26:13.470459] [DEBUG] [LOCAL ] HPI: Loaded 10543 existing records from primary storage. [2025-11-04T19:26:13.470634] [INFO ] [MAIN ] HPI: Loaded 10543 existing records. [2025-11-04T19:26:13.664866] [WARN ] [MAIN ] HPI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:26:13.665061] [INFO ] [MAIN ] HPI: Wiping existing primary data before full refresh. [2025-11-04T19:26:13.665107] [INFO ] [LOCAL ] HPI: Starting deletion of primary data. [2025-11-04T19:26:13.665419] [DEBUG] [LOCAL ] HPI: Deleted primary data file. [2025-11-04T19:26:13.665483] [INFO ] [MAIN ] HPI: Starting full primary refresh. [2025-11-04T19:26:13.772224] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-09. [2025-11-04T19:26:14.013981] [INFO ] [REMOTE ] HPI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:26:14.554601] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-11-04T19:26:14.567002] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-01. [2025-11-04T19:26:14.899169] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-11-04T19:26:15.053475] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-01 (0.49 seconds). [2025-11-04T19:26:15.367028] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-02. [2025-11-04T19:26:15.868588] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-11-04T19:26:16.167052] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-10. [2025-11-04T19:26:16.662269] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-11-04T19:26:16.967055] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-10. [2025-11-04T19:26:17.767108] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-11. [2025-11-04T19:26:18.318725] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T19:26:18.319811] [INFO ] [REMOTE ] HPF: Full fetch complete (10284 records). [2025-11-04T19:26:18.319907] [DEBUG] [VALIDATOR ] HPF: Validating primary data. [2025-11-04T19:26:18.568191] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-03. [2025-11-04T19:26:18.691955] [DEBUG] [VALIDATOR ] HPF: All expected trading days are present. [2025-11-04T19:26:18.692054] [DEBUG] [VALIDATOR ] HPF: Primary data validation passed. [2025-11-04T19:26:18.692406] [INFO ] [MAIN ] HPF: Fetched primary data is valid. [2025-11-04T19:26:18.692560] [DEBUG] [LOCAL ] HPF: Saving primary data. [2025-11-04T19:26:18.693325] [DEBUG] [LOCAL ] HPF: Trimmed 10284 records to 7295 (keeping 200 days of data) [2025-11-04T19:26:18.775014] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-11-04T19:26:18.776140] [DEBUG] [LOCAL ] HPF: Primary data saved (7295 records). [2025-11-04T19:26:18.776284] [INFO ] [MAIN ] HPF: Full history saved. [2025-11-04T19:26:18.787202] [INFO ] [MAIN ] Update Progress: 1042/2367 symbols completed. HPF took 30.8s. Estimated remaining: 3h 5m 59s. Rate: 7.1 symbols/min [2025-11-04T19:26:18.956061] [INFO ] [MAIN ] HPP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:26:18.999758] [DEBUG] [LOCAL ] HPP: Loaded 51272 existing records from primary storage. [2025-11-04T19:26:19.000075] [INFO ] [MAIN ] HPP: Loaded 51272 existing records. [2025-11-04T19:26:19.180630] [WARN ] [MAIN ] HPP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:26:19.180927] [INFO ] [MAIN ] HPP: Wiping existing primary data before full refresh. [2025-11-04T19:26:19.181087] [INFO ] [LOCAL ] HPP: Starting deletion of primary data. [2025-11-04T19:26:19.181946] [DEBUG] [LOCAL ] HPP: Deleted primary data file. [2025-11-04T19:26:19.182002] [INFO ] [MAIN ] HPP: Starting full primary refresh. [2025-11-04T19:26:19.292250] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-11-04T19:26:19.374187] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-04. [2025-11-04T19:26:19.549259] [INFO ] [REMOTE ] HPP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:26:20.100232] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-11-04T19:26:20.167077] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-01. [2025-11-04T19:26:20.967103] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-11. [2025-11-04T19:26:21.523184] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T19:26:21.767039] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-05. [2025-11-04T19:26:22.004264] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T19:26:22.006154] [INFO ] [REMOTE ] HP: Full fetch complete (74144 records). [2025-11-04T19:26:22.006239] [DEBUG] [VALIDATOR ] HP: Validating primary data. [2025-11-04T19:26:22.319226] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T19:26:22.403485] [DEBUG] [VALIDATOR ] HP: All expected trading days are present. [2025-11-04T19:26:22.403559] [DEBUG] [VALIDATOR ] HP: Primary data validation passed. [2025-11-04T19:26:22.404432] [INFO ] [MAIN ] HP: Fetched primary data is valid. [2025-11-04T19:26:22.404503] [DEBUG] [LOCAL ] HP: Saving primary data. [2025-11-04T19:26:22.405868] [DEBUG] [LOCAL ] HP: Trimmed 74144 records to 51110 (keeping 200 days of data) [2025-11-04T19:26:22.573746] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-02. [2025-11-04T19:26:22.709179] [DEBUG] [LOCAL ] HP: Primary data saved (51110 records). [2025-11-04T19:26:22.709347] [INFO ] [MAIN ] HP: Full history saved. [2025-11-04T19:26:22.709443] [INFO ] [MAIN ] Update Progress: 1043/2367 symbols completed. HP took 34.8s. Estimated remaining: 3h 5m 45s. Rate: 7.1 symbols/min [2025-11-04T19:26:22.875775] [INFO ] [MAIN ] HPQ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:26:22.921293] [DEBUG] [LOCAL ] HPQ: Loaded 53669 existing records from primary storage. [2025-11-04T19:26:22.921588] [INFO ] [MAIN ] HPQ: Loaded 53669 existing records. [2025-11-04T19:26:23.096235] [WARN ] [MAIN ] HPQ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:26:23.096457] [INFO ] [MAIN ] HPQ: Wiping existing primary data before full refresh. [2025-11-04T19:26:23.096507] [INFO ] [LOCAL ] HPQ: Starting deletion of primary data. [2025-11-04T19:26:23.097467] [DEBUG] [LOCAL ] HPQ: Deleted primary data file. [2025-11-04T19:26:23.097549] [INFO ] [MAIN ] HPQ: Starting full primary refresh. [2025-11-04T19:26:23.369451] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-06. [2025-11-04T19:26:23.448589] [INFO ] [REMOTE ] HPQ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:26:23.747890] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T19:26:23.889705] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-11-04T19:26:24.167057] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-07. [2025-11-04T19:26:24.670958] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-11-04T19:26:24.967050] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-08. [2025-11-04T19:26:25.767051] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-03. [2025-11-04T19:26:26.417306] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T19:26:26.567039] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-08. [2025-11-04T19:26:27.147519] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-11-04T19:26:27.281582] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-11-04T19:26:27.367144] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-09. [2025-11-04T19:26:28.167051] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-01. [2025-11-04T19:26:28.744978] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T19:26:28.967037] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-10. [2025-11-04T19:26:29.478241] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T19:26:29.767037] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-02. [2025-11-04T19:26:30.567013] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-09. [2025-11-04T19:26:30.639332] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-11-04T19:26:31.166228] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-11-04T19:26:31.190871] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-11-04T19:26:31.367080] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-11. [2025-11-04T19:26:32.167065] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-03. [2025-11-04T19:26:32.802713] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-11 (1.44 seconds). [2025-11-04T19:26:32.804745] [INFO ] [REMOTE ] HPE: Full fetch complete (77876 records). [2025-11-04T19:26:32.804841] [DEBUG] [VALIDATOR ] HPE: Validating primary data. [2025-11-04T19:26:32.968804] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-04. [2025-11-04T19:26:33.215505] [DEBUG] [VALIDATOR ] HPE: All expected trading days are present. [2025-11-04T19:26:33.215604] [DEBUG] [VALIDATOR ] HPE: Primary data validation passed. [2025-11-04T19:26:33.216530] [INFO ] [MAIN ] HPE: Fetched primary data is valid. [2025-11-04T19:26:33.216592] [DEBUG] [LOCAL ] HPE: Saving primary data. [2025-11-04T19:26:33.218062] [DEBUG] [LOCAL ] HPE: Trimmed 77876 records to 53697 (keeping 200 days of data) [2025-11-04T19:26:33.549673] [DEBUG] [LOCAL ] HPE: Primary data saved (53697 records). [2025-11-04T19:26:33.549844] [INFO ] [MAIN ] HPE: Full history saved. [2025-11-04T19:26:33.549928] [INFO ] [MAIN ] Update Progress: 1044/2367 symbols completed. HPE took 45.6s. Estimated remaining: 3h 5m 39s. Rate: 7.1 symbols/min [2025-11-04T19:26:33.722263] [INFO ] [MAIN ] HPS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:26:33.768487] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-10. [2025-11-04T19:26:33.769154] [DEBUG] [LOCAL ] HPS: Loaded 10137 existing records from primary storage. [2025-11-04T19:26:33.773596] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-11-04T19:26:33.785729] [INFO ] [MAIN ] HPS: Loaded 10137 existing records. [2025-11-04T19:26:33.962951] [WARN ] [MAIN ] HPS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:26:33.963164] [INFO ] [MAIN ] HPS: Wiping existing primary data before full refresh. [2025-11-04T19:26:33.963212] [INFO ] [LOCAL ] HPS: Starting deletion of primary data. [2025-11-04T19:26:33.963475] [DEBUG] [LOCAL ] HPS: Deleted primary data file. [2025-11-04T19:26:33.963514] [INFO ] [MAIN ] HPS: Starting full primary refresh. [2025-11-04T19:26:34.327185] [INFO ] [REMOTE ] HPS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:26:34.430983] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-11-04T19:26:34.430626] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-11-04T19:26:34.567009] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-11. [2025-11-04T19:26:35.097857] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-11 (0.53 seconds). [2025-11-04T19:26:35.098661] [INFO ] [REMOTE ] HPI: Full fetch complete (13813 records). [2025-11-04T19:26:35.098734] [DEBUG] [VALIDATOR ] HPI: Validating primary data. [2025-11-04T19:26:35.371973] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-01. [2025-11-04T19:26:35.464184] [DEBUG] [VALIDATOR ] HPI: All expected trading days are present. [2025-11-04T19:26:35.464262] [DEBUG] [VALIDATOR ] HPI: Primary data validation passed. [2025-11-04T19:26:35.464802] [INFO ] [MAIN ] HPI: Fetched primary data is valid. [2025-11-04T19:26:35.464892] [DEBUG] [LOCAL ] HPI: Saving primary data. [2025-11-04T19:26:35.465580] [DEBUG] [LOCAL ] HPI: Trimmed 13813 records to 10500 (keeping 200 days of data) [2025-11-04T19:26:35.532363] [DEBUG] [LOCAL ] HPI: Primary data saved (10500 records). [2025-11-04T19:26:35.532500] [INFO ] [MAIN ] HPI: Full history saved. [2025-11-04T19:26:35.532581] [INFO ] [MAIN ] Update Progress: 1045/2367 symbols completed. HPI took 22.2s. Estimated remaining: 3h 5m 23s. Rate: 7.1 symbols/min [2025-11-04T19:26:35.744068] [INFO ] [MAIN ] HQH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:26:35.770167] [DEBUG] [LOCAL ] HQH: Loaded 25566 existing records from primary storage. [2025-11-04T19:26:35.770522] [INFO ] [MAIN ] HQH: Loaded 25566 existing records. [2025-11-04T19:26:35.932165] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T19:26:35.990689] [WARN ] [MAIN ] HQH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:26:35.990920] [INFO ] [MAIN ] HQH: Wiping existing primary data before full refresh. [2025-11-04T19:26:35.990989] [INFO ] [LOCAL ] HQH: Starting deletion of primary data. [2025-11-04T19:26:35.991472] [DEBUG] [LOCAL ] HQH: Deleted primary data file. [2025-11-04T19:26:35.991528] [INFO ] [MAIN ] HQH: Starting full primary refresh. [2025-11-04T19:26:36.173418] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-05. [2025-11-04T19:26:36.340195] [INFO ] [REMOTE ] HQH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:26:36.967094] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-02. [2025-11-04T19:26:37.573233] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-11-04T19:26:37.602743] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T19:26:37.767001] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-06. [2025-11-04T19:26:38.567084] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-03. [2025-11-04T19:26:39.109354] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-11-04T19:26:39.152665] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-11-04T19:26:39.367039] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-04. [2025-11-04T19:26:39.860429] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-11-04T19:26:40.167042] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-04. [2025-11-04T19:26:40.967082] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-05. [2025-11-04T19:26:41.490534] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-11-04T19:26:41.767036] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-06. [2025-11-04T19:26:41.859689] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-11-04T19:26:42.317940] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-11-04T19:26:42.567114] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-07. [2025-11-04T19:26:43.367073] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-01. [2025-11-04T19:26:43.986603] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-11-04T19:26:44.167056] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-07. [2025-11-04T19:26:44.226367] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T19:26:44.761639] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-11-04T19:26:44.967077] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-02. [2025-11-04T19:26:45.767027] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-08. [2025-11-04T19:26:45.803854] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-11-04T19:26:46.212182] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-11-04T19:26:46.567066] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-05. [2025-11-04T19:26:47.367050] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-08. [2025-11-04T19:26:48.016826] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T19:26:48.167085] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-06. [2025-11-04T19:26:48.783458] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T19:26:48.967011] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-09. [2025-11-04T19:26:49.572090] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-11-04T19:26:49.767094] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-07. [2025-11-04T19:26:50.200631] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T19:26:50.567072] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-10. [2025-11-04T19:26:51.212690] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T19:26:51.367064] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-08. [2025-11-04T19:26:51.866457] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-11-04T19:26:52.167063] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-09. [2025-11-04T19:26:52.591581] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-11-04T19:26:52.948346] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-11-04T19:26:52.967014] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-09. [2025-11-04T19:26:53.767046] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-03. [2025-11-04T19:26:54.405969] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T19:26:54.560777] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T19:26:54.566995] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-04. [2025-11-04T19:26:55.373340] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-11. [2025-11-04T19:26:55.379232] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-11-04T19:26:56.167044] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-05. [2025-11-04T19:26:56.693170] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-11 (1.32 seconds). [2025-11-04T19:26:56.695267] [INFO ] [REMOTE ] HPP: Full fetch complete (73438 records). [2025-11-04T19:26:56.695364] [DEBUG] [VALIDATOR ] HPP: Validating primary data. [2025-11-04T19:26:56.973357] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-10. [2025-11-04T19:26:57.098280] [DEBUG] [VALIDATOR ] HPP: All expected trading days are present. [2025-11-04T19:26:57.098366] [DEBUG] [VALIDATOR ] HPP: Primary data validation passed. [2025-11-04T19:26:57.099278] [INFO ] [MAIN ] HPP: Fetched primary data is valid. [2025-11-04T19:26:57.099342] [DEBUG] [LOCAL ] HPP: Saving primary data. [2025-11-04T19:26:57.100763] [DEBUG] [LOCAL ] HPP: Trimmed 73438 records to 51296 (keeping 200 days of data) [2025-11-04T19:26:57.344460] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-11-04T19:26:57.414813] [DEBUG] [LOCAL ] HPP: Primary data saved (51296 records). [2025-11-04T19:26:57.414963] [INFO ] [MAIN ] HPP: Full history saved. [2025-11-04T19:26:57.415052] [INFO ] [MAIN ] Update Progress: 1046/2367 symbols completed. HPP took 38.6s. Estimated remaining: 3h 5m 31s. Rate: 7.1 symbols/min [2025-11-04T19:26:57.560401] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-11-04T19:26:57.608254] [INFO ] [MAIN ] HQL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:26:57.630215] [DEBUG] [LOCAL ] HQL: Loaded 17676 existing records from primary storage. [2025-11-04T19:26:57.630609] [INFO ] [MAIN ] HQL: Loaded 17676 existing records. [2025-11-04T19:26:57.772263] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-11. [2025-11-04T19:26:57.827491] [WARN ] [MAIN ] HQL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:26:57.827744] [INFO ] [MAIN ] HQL: Wiping existing primary data before full refresh. [2025-11-04T19:26:57.827799] [INFO ] [LOCAL ] HQL: Starting deletion of primary data. [2025-11-04T19:26:57.828206] [DEBUG] [LOCAL ] HQL: Deleted primary data file. [2025-11-04T19:26:57.828268] [INFO ] [MAIN ] HQL: Starting full primary refresh. [2025-11-04T19:26:58.184201] [INFO ] [REMOTE ] HQL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:26:58.301328] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-11 (0.53 seconds). [2025-11-04T19:26:58.302228] [INFO ] [REMOTE ] HPS: Full fetch complete (14876 records). [2025-11-04T19:26:58.302294] [DEBUG] [VALIDATOR ] HPS: Validating primary data. [2025-11-04T19:26:58.574792] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-10. [2025-11-04T19:26:58.660111] [DEBUG] [VALIDATOR ] HPS: All expected trading days are present. [2025-11-04T19:26:58.660201] [DEBUG] [VALIDATOR ] HPS: Primary data validation passed. [2025-11-04T19:26:58.660565] [INFO ] [MAIN ] HPS: Fetched primary data is valid. [2025-11-04T19:26:58.660683] [DEBUG] [LOCAL ] HPS: Saving primary data. [2025-11-04T19:26:58.661260] [DEBUG] [LOCAL ] HPS: Trimmed 14876 records to 10161 (keeping 200 days of data) [2025-11-04T19:26:58.722151] [DEBUG] [LOCAL ] HPS: Primary data saved (10161 records). [2025-11-04T19:26:58.722297] [INFO ] [MAIN ] HPS: Full history saved. [2025-11-04T19:26:58.722379] [INFO ] [MAIN ] Update Progress: 1047/2367 symbols completed. HPS took 25.2s. Estimated remaining: 3h 5m 14s. Rate: 7.1 symbols/min [2025-11-04T19:26:58.889045] [INFO ] [MAIN ] HR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:26:58.937066] [DEBUG] [LOCAL ] HR: Loaded 52099 existing records from primary storage. [2025-11-04T19:26:58.937409] [INFO ] [MAIN ] HR: Loaded 52099 existing records. [2025-11-04T19:26:59.114814] [WARN ] [MAIN ] HR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:26:59.115040] [INFO ] [MAIN ] HR: Wiping existing primary data before full refresh. [2025-11-04T19:26:59.115089] [INFO ] [LOCAL ] HR: Starting deletion of primary data. [2025-11-04T19:26:59.115332] [DEBUG] [LOCAL ] HR: Deleted primary data file. [2025-11-04T19:26:59.115388] [INFO ] [MAIN ] HR: Starting full primary refresh. [2025-11-04T19:26:59.376849] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-06. [2025-11-04T19:26:59.465458] [INFO ] [REMOTE ] HR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:27:00.167088] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-01. [2025-11-04T19:27:00.231508] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-11-04T19:27:00.371226] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-11-04T19:27:00.967108] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-01. [2025-11-04T19:27:01.638078] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T19:27:01.663804] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-11-04T19:27:01.767079] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-11. [2025-11-04T19:27:02.023304] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-11 (0.26 seconds). [2025-11-04T19:27:02.025353] [INFO ] [REMOTE ] HPQ: Full fetch complete (77843 records). [2025-11-04T19:27:02.025480] [DEBUG] [VALIDATOR ] HPQ: Validating primary data. [2025-11-04T19:27:02.416147] [DEBUG] [VALIDATOR ] HPQ: All expected trading days are present. [2025-11-04T19:27:02.416270] [DEBUG] [VALIDATOR ] HPQ: Primary data validation passed. [2025-11-04T19:27:02.417309] [INFO ] [MAIN ] HPQ: Fetched primary data is valid. [2025-11-04T19:27:02.417376] [DEBUG] [LOCAL ] HPQ: Saving primary data. [2025-11-04T19:27:02.418861] [DEBUG] [LOCAL ] HPQ: Trimmed 77843 records to 53669 (keeping 200 days of data) [2025-11-04T19:27:02.589849] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-02. [2025-11-04T19:27:02.741158] [DEBUG] [LOCAL ] HPQ: Primary data saved (53669 records). [2025-11-04T19:27:02.741296] [INFO ] [MAIN ] HPQ: Full history saved. [2025-11-04T19:27:02.741377] [INFO ] [MAIN ] Update Progress: 1048/2367 symbols completed. HPQ took 40.0s. Estimated remaining: 3h 5m 0s. Rate: 7.1 symbols/min [2025-11-04T19:27:02.904440] [INFO ] [MAIN ] HRB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:27:02.946112] [DEBUG] [LOCAL ] HRB: Loaded 47854 existing records from primary storage. [2025-11-04T19:27:02.946427] [INFO ] [MAIN ] HRB: Loaded 47854 existing records. [2025-11-04T19:27:03.120870] [WARN ] [MAIN ] HRB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:27:03.121167] [INFO ] [MAIN ] HRB: Wiping existing primary data before full refresh. [2025-11-04T19:27:03.121222] [INFO ] [LOCAL ] HRB: Starting deletion of primary data. [2025-11-04T19:27:03.121605] [DEBUG] [LOCAL ] HRB: Deleted primary data file. [2025-11-04T19:27:03.121699] [INFO ] [MAIN ] HRB: Starting full primary refresh. [2025-11-04T19:27:03.371280] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-07. [2025-11-04T19:27:03.496553] [INFO ] [REMOTE ] HRB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:27:03.497960] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T19:27:04.167063] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-01. [2025-11-04T19:27:04.534815] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-11-04T19:27:04.967085] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-02. [2025-11-04T19:27:05.379261] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T19:27:05.767076] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-03. [2025-11-04T19:27:06.230040] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T19:27:06.477789] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T19:27:06.567031] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-04. [2025-11-04T19:27:07.278583] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-11-04T19:27:07.367053] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-02. [2025-11-04T19:27:08.167060] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-08. [2025-11-04T19:27:08.604927] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-11-04T19:27:08.967015] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-03. [2025-11-04T19:27:09.048403] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-11-04T19:27:09.767034] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-05. [2025-11-04T19:27:10.342076] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T19:27:10.565132] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-11-04T19:27:10.568629] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-09. [2025-11-04T19:27:11.367024] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-06. [2025-11-04T19:27:11.425518] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-11-04T19:27:12.075749] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T19:27:12.167058] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-07. [2025-11-04T19:27:12.889838] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-11-04T19:27:12.967052] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-10. [2025-11-04T19:27:13.767020] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-08. [2025-11-04T19:27:13.936552] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T19:27:14.332903] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-11-04T19:27:14.567067] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-04. [2025-11-04T19:27:15.367086] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-03. [2025-11-04T19:27:15.990887] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T19:27:16.167060] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-05. [2025-11-04T19:27:16.803517] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T19:27:16.967025] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-04. [2025-11-04T19:27:17.448895] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T19:27:17.767046] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-11. [2025-11-04T19:27:18.386867] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T19:27:18.548659] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T19:27:18.549992] [INFO ] [REMOTE ] HQH: Full fetch complete (36691 records). [2025-11-04T19:27:18.550068] [DEBUG] [VALIDATOR ] HQH: Validating primary data. [2025-11-04T19:27:18.572212] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-09. [2025-11-04T19:27:18.920802] [DEBUG] [VALIDATOR ] HQH: All expected trading days are present. [2025-11-04T19:27:18.920882] [DEBUG] [VALIDATOR ] HQH: Primary data validation passed. [2025-11-04T19:27:18.921761] [INFO ] [MAIN ] HQH: Fetched primary data is valid. [2025-11-04T19:27:18.921822] [DEBUG] [LOCAL ] HQH: Saving primary data. [2025-11-04T19:27:18.922692] [DEBUG] [LOCAL ] HQH: Trimmed 36691 records to 25593 (keeping 200 days of data) [2025-11-04T19:27:19.078290] [DEBUG] [LOCAL ] HQH: Primary data saved (25593 records). [2025-11-04T19:27:19.078424] [INFO ] [MAIN ] HQH: Full history saved. [2025-11-04T19:27:19.078502] [INFO ] [MAIN ] Update Progress: 1049/2367 symbols completed. HQH took 43.5s. Estimated remaining: 3h 5m 2s. Rate: 7.1 symbols/min [2025-11-04T19:27:19.243509] [INFO ] [MAIN ] HRI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:27:19.274297] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-11-04T19:27:19.287854] [DEBUG] [LOCAL ] HRI: Loaded 32069 existing records from primary storage. [2025-11-04T19:27:19.288357] [INFO ] [MAIN ] HRI: Loaded 32069 existing records. [2025-11-04T19:27:19.372140] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-10. [2025-11-04T19:27:19.464380] [WARN ] [MAIN ] HRI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:27:19.464584] [INFO ] [MAIN ] HRI: Wiping existing primary data before full refresh. [2025-11-04T19:27:19.464629] [INFO ] [LOCAL ] HRI: Starting deletion of primary data. [2025-11-04T19:27:19.465257] [DEBUG] [LOCAL ] HRI: Deleted primary data file. [2025-11-04T19:27:19.465315] [INFO ] [MAIN ] HRI: Starting full primary refresh. [2025-11-04T19:27:19.813715] [INFO ] [REMOTE ] HRI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:27:20.167055] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-05. [2025-11-04T19:27:20.335738] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-11-04T19:27:20.967040] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-01. [2025-11-04T19:27:21.548266] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-11-04T19:27:21.767059] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-06. [2025-11-04T19:27:21.900187] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-11-04T19:27:22.567050] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-11. [2025-11-04T19:27:23.104276] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T19:27:23.312210] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-11 (0.75 seconds). [2025-11-04T19:27:23.313454] [INFO ] [REMOTE ] HQL: Full fetch complete (25710 records). [2025-11-04T19:27:23.313522] [DEBUG] [VALIDATOR ] HQL: Validating primary data. [2025-11-04T19:27:23.375364] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-06. [2025-11-04T19:27:23.679097] [DEBUG] [VALIDATOR ] HQL: All expected trading days are present. [2025-11-04T19:27:23.679191] [DEBUG] [VALIDATOR ] HQL: Primary data validation passed. [2025-11-04T19:27:23.679701] [INFO ] [MAIN ] HQL: Fetched primary data is valid. [2025-11-04T19:27:23.679760] [DEBUG] [LOCAL ] HQL: Saving primary data. [2025-11-04T19:27:23.680361] [DEBUG] [LOCAL ] HQL: Trimmed 25710 records to 17777 (keeping 200 days of data) [2025-11-04T19:27:23.783735] [DEBUG] [LOCAL ] HQL: Primary data saved (17777 records). [2025-11-04T19:27:23.783839] [INFO ] [MAIN ] HQL: Full history saved. [2025-11-04T19:27:23.783918] [INFO ] [MAIN ] Update Progress: 1050/2367 symbols completed. HQL took 26.4s. Estimated remaining: 3h 4m 48s. Rate: 7.1 symbols/min [2025-11-04T19:27:23.961428] [INFO ] [MAIN ] HRL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:27:24.006363] [DEBUG] [LOCAL ] HRL: Loaded 53149 existing records from primary storage. [2025-11-04T19:27:24.006703] [INFO ] [MAIN ] HRL: Loaded 53149 existing records. [2025-11-04T19:27:24.172146] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-02. [2025-11-04T19:27:24.229197] [WARN ] [MAIN ] HRL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:27:24.229398] [INFO ] [MAIN ] HRL: Wiping existing primary data before full refresh. [2025-11-04T19:27:24.229444] [INFO ] [LOCAL ] HRL: Starting deletion of primary data. [2025-11-04T19:27:24.230392] [DEBUG] [LOCAL ] HRL: Deleted primary data file. [2025-11-04T19:27:24.230448] [INFO ] [MAIN ] HRL: Starting full primary refresh. [2025-11-04T19:27:24.573787] [INFO ] [REMOTE ] HRL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:27:24.606976] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-11-04T19:27:24.967054] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-01. [2025-11-04T19:27:25.145195] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T19:27:25.767040] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-07. [2025-11-04T19:27:26.405718] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T19:27:26.567045] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-07. [2025-11-04T19:27:27.156973] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-11-04T19:27:27.367042] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-08. [2025-11-04T19:27:27.811989] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-11-04T19:27:28.167060] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-03. [2025-11-04T19:27:28.540775] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T19:27:28.967048] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-08. [2025-11-04T19:27:29.308022] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T19:27:29.767033] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-09. [2025-11-04T19:27:30.357551] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T19:27:30.567040] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-04. [2025-11-04T19:27:31.131243] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-11-04T19:27:31.367028] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-10. [2025-11-04T19:27:31.592058] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-11-04T19:27:32.167057] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-05. [2025-11-04T19:27:32.820061] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T19:27:32.967057] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-09. [2025-11-04T19:27:33.276143] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T19:27:33.767022] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-11. [2025-11-04T19:27:34.305271] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-11-04T19:27:34.567066] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-02. [2025-11-04T19:27:34.989312] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T19:27:34.991287] [INFO ] [REMOTE ] HR: Full fetch complete (75222 records). [2025-11-04T19:27:34.991377] [DEBUG] [VALIDATOR ] HR: Validating primary data. [2025-11-04T19:27:35.370230] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-06. [2025-11-04T19:27:35.370551] [DEBUG] [VALIDATOR ] HR: All expected trading days are present. [2025-11-04T19:27:35.371631] [DEBUG] [VALIDATOR ] HR: Primary data validation passed. [2025-11-04T19:27:35.372531] [INFO ] [MAIN ] HR: Fetched primary data is valid. [2025-11-04T19:27:35.372699] [DEBUG] [LOCAL ] HR: Saving primary data. [2025-11-04T19:27:35.374122] [DEBUG] [LOCAL ] HR: Trimmed 75222 records to 52101 (keeping 200 days of data) [2025-11-04T19:27:35.680722] [DEBUG] [LOCAL ] HR: Primary data saved (52101 records). [2025-11-04T19:27:35.680843] [INFO ] [MAIN ] HR: Full history saved. [2025-11-04T19:27:35.680919] [INFO ] [MAIN ] Update Progress: 1051/2367 symbols completed. HR took 37.0s. Estimated remaining: 3h 4m 44s. Rate: 7.1 symbols/min [2025-11-04T19:27:35.848541] [INFO ] [MAIN ] HRTG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:27:35.878603] [DEBUG] [LOCAL ] HRTG: Loaded 34179 existing records from primary storage. [2025-11-04T19:27:35.878852] [INFO ] [MAIN ] HRTG: Loaded 34179 existing records. [2025-11-04T19:27:36.057675] [WARN ] [MAIN ] HRTG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:27:36.057907] [INFO ] [MAIN ] HRTG: Wiping existing primary data before full refresh. [2025-11-04T19:27:36.058001] [INFO ] [LOCAL ] HRTG: Starting deletion of primary data. [2025-11-04T19:27:36.058596] [DEBUG] [LOCAL ] HRTG: Deleted primary data file. [2025-11-04T19:27:36.058692] [INFO ] [MAIN ] HRTG: Starting full primary refresh. [2025-11-04T19:27:36.168416] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-10. [2025-11-04T19:27:36.236173] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-11-04T19:27:36.455529] [INFO ] [REMOTE ] HRTG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:27:36.480483] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T19:27:36.967027] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-03. [2025-11-04T19:27:37.571417] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T19:27:37.767034] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-01. [2025-11-04T19:27:38.411825] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T19:27:38.567014] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-07. [2025-11-04T19:27:38.642108] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T19:27:39.367054] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-02. [2025-11-04T19:27:39.645491] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-11-04T19:27:40.088215] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-11-04T19:27:40.167060] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-03. [2025-11-04T19:27:40.959496] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T19:27:40.967002] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-04. [2025-11-04T19:27:41.767043] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-08. [2025-11-04T19:27:42.265146] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T19:27:42.567108] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-04. [2025-11-04T19:27:42.709209] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-11-04T19:27:43.367131] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-11. [2025-11-04T19:27:43.743108] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T19:27:44.167082] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-05. [2025-11-04T19:27:44.567459] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T19:27:44.569309] [INFO ] [REMOTE ] HRB: Full fetch complete (69873 records). [2025-11-04T19:27:44.569399] [DEBUG] [VALIDATOR ] HRB: Validating primary data. [2025-11-04T19:27:44.963136] [DEBUG] [VALIDATOR ] HRB: All expected trading days are present. [2025-11-04T19:27:44.963209] [DEBUG] [VALIDATOR ] HRB: Primary data validation passed. [2025-11-04T19:27:44.964049] [INFO ] [MAIN ] HRB: Fetched primary data is valid. [2025-11-04T19:27:44.964238] [DEBUG] [LOCAL ] HRB: Saving primary data. [2025-11-04T19:27:44.965587] [DEBUG] [LOCAL ] HRB: Trimmed 69873 records to 47805 (keeping 200 days of data) [2025-11-04T19:27:45.019302] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-09. [2025-11-04T19:27:45.264190] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-11-04T19:27:45.283695] [DEBUG] [LOCAL ] HRB: Primary data saved (47805 records). [2025-11-04T19:27:45.283777] [INFO ] [MAIN ] HRB: Full history saved. [2025-11-04T19:27:45.283844] [INFO ] [MAIN ] Update Progress: 1052/2367 symbols completed. HRB took 42.5s. Estimated remaining: 3h 4m 37s. Rate: 7.1 symbols/min [2025-11-04T19:27:45.448865] [INFO ] [MAIN ] HSBC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:27:45.493369] [DEBUG] [LOCAL ] HSBC: Loaded 50916 existing records from primary storage. [2025-11-04T19:27:45.493671] [INFO ] [MAIN ] HSBC: Loaded 50916 existing records. [2025-11-04T19:27:45.667762] [WARN ] [MAIN ] HSBC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:27:45.667955] [INFO ] [MAIN ] HSBC: Wiping existing primary data before full refresh. [2025-11-04T19:27:45.668003] [INFO ] [LOCAL ] HSBC: Starting deletion of primary data. [2025-11-04T19:27:45.668871] [DEBUG] [LOCAL ] HSBC: Deleted primary data file. [2025-11-04T19:27:45.668927] [INFO ] [MAIN ] HSBC: Starting full primary refresh. [2025-11-04T19:27:45.768098] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-06. [2025-11-04T19:27:46.027845] [INFO ] [REMOTE ] HSBC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:27:46.045795] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-11-04T19:27:46.567067] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-10. [2025-11-04T19:27:46.905441] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-11-04T19:27:47.367137] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-05. [2025-11-04T19:27:47.518210] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T19:27:48.167110] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-11. [2025-11-04T19:27:48.775468] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T19:27:48.967090] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-01. [2025-11-04T19:27:49.067336] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T19:27:49.068682] [INFO ] [REMOTE ] HRI: Full fetch complete (47095 records). [2025-11-04T19:27:49.068783] [DEBUG] [VALIDATOR ] HRI: Validating primary data. [2025-11-04T19:27:49.440756] [DEBUG] [VALIDATOR ] HRI: All expected trading days are present. [2025-11-04T19:27:49.440860] [DEBUG] [VALIDATOR ] HRI: Primary data validation passed. [2025-11-04T19:27:49.441540] [INFO ] [MAIN ] HRI: Fetched primary data is valid. [2025-11-04T19:27:49.441584] [DEBUG] [LOCAL ] HRI: Saving primary data. [2025-11-04T19:27:49.442619] [DEBUG] [LOCAL ] HRI: Trimmed 47095 records to 31994 (keeping 200 days of data) [2025-11-04T19:27:49.637077] [DEBUG] [LOCAL ] HRI: Primary data saved (31994 records). [2025-11-04T19:27:49.637218] [INFO ] [MAIN ] HRI: Full history saved. [2025-11-04T19:27:49.637297] [INFO ] [MAIN ] Update Progress: 1053/2367 symbols completed. HRI took 30.6s. Estimated remaining: 3h 4m 24s. Rate: 7.1 symbols/min [2025-11-04T19:27:49.781110] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-07. [2025-11-04T19:27:49.816109] [INFO ] [MAIN ] HSHP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:27:49.829370] [DEBUG] [LOCAL ] HSHP: Loaded 13694 existing records from primary storage. [2025-11-04T19:27:49.829578] [INFO ] [MAIN ] HSHP: Loaded 13694 existing records. [2025-11-04T19:27:50.002220] [WARN ] [MAIN ] HSHP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:27:50.002422] [INFO ] [MAIN ] HSHP: Wiping existing primary data before full refresh. [2025-11-04T19:27:50.002472] [INFO ] [LOCAL ] HSHP: Starting deletion of primary data. [2025-11-04T19:27:50.002815] [DEBUG] [LOCAL ] HSHP: Deleted primary data file. [2025-11-04T19:27:50.002871] [INFO ] [MAIN ] HSHP: Starting full primary refresh. [2025-11-04T19:27:50.377402] [INFO ] [REMOTE ] HSHP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:27:50.405801] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T19:27:50.567070] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-01. [2025-11-04T19:27:50.874706] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-11-04T19:27:51.358897] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-11-04T19:27:51.367008] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-02. [2025-11-04T19:27:52.018784] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-11-04T19:27:52.167110] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-08. [2025-11-04T19:27:52.967082] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-02. [2025-11-04T19:27:53.182935] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-11-04T19:27:53.767138] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-06. [2025-11-04T19:27:54.441406] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-11-04T19:27:54.567097] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-03. [2025-11-04T19:27:55.084977] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-11-04T19:27:55.331502] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T19:27:55.367115] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-03. [2025-11-04T19:27:56.167059] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-04. [2025-11-04T19:27:56.889080] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-11-04T19:27:56.967051] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-05. [2025-11-04T19:27:57.184991] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-11-04T19:27:57.511617] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-11-04T19:27:57.767044] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-06. [2025-11-04T19:27:58.561948] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T19:27:58.566986] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-07. [2025-11-04T19:27:59.188717] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-11-04T19:27:59.367156] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-07. [2025-11-04T19:28:00.167098] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-08. [2025-11-04T19:28:00.852363] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-11-04T19:28:00.967046] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-09. [2025-11-04T19:28:01.164994] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-11-04T19:28:01.774755] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-04. [2025-11-04T19:28:01.778600] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-11-04T19:28:02.567058] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-08. [2025-11-04T19:28:03.329242] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-11-04T19:28:03.366996] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-05. [2025-11-04T19:28:04.153142] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-11-04T19:28:04.167002] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-09. [2025-11-04T19:28:04.811902] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-11-04T19:28:04.967236] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-06. [2025-11-04T19:28:05.597183] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-11-04T19:28:05.767053] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-09. [2025-11-04T19:28:06.270466] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T19:28:06.567031] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-07. [2025-11-04T19:28:06.851477] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T19:28:07.367036] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-10. [2025-11-04T19:28:08.167014] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-10. [2025-11-04T19:28:08.417093] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-11-04T19:28:08.452785] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-11-04T19:28:08.848306] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-11-04T19:28:08.967006] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-11. [2025-11-04T19:28:09.767031] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-10. [2025-11-04T19:28:09.927023] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-11 (0.96 seconds). [2025-11-04T19:28:09.928402] [INFO ] [REMOTE ] HRTG: Full fetch complete (44765 records). [2025-11-04T19:28:09.928479] [DEBUG] [VALIDATOR ] HRTG: Validating primary data. [2025-11-04T19:28:10.316061] [DEBUG] [VALIDATOR ] HRTG: All expected trading days are present. [2025-11-04T19:28:10.316158] [DEBUG] [VALIDATOR ] HRTG: Primary data validation passed. [2025-11-04T19:28:10.316880] [INFO ] [MAIN ] HRTG: Fetched primary data is valid. [2025-11-04T19:28:10.316944] [DEBUG] [LOCAL ] HRTG: Saving primary data. [2025-11-04T19:28:10.317932] [DEBUG] [LOCAL ] HRTG: Trimmed 44765 records to 34077 (keeping 200 days of data) [2025-11-04T19:28:10.508105] [DEBUG] [LOCAL ] HRTG: Primary data saved (34077 records). [2025-11-04T19:28:10.508229] [INFO ] [MAIN ] HRTG: Full history saved. [2025-11-04T19:28:10.508306] [INFO ] [MAIN ] Update Progress: 1054/2367 symbols completed. HRTG took 34.8s. Estimated remaining: 3h 4m 31s. Rate: 7.1 symbols/min [2025-11-04T19:28:10.572170] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-11. [2025-11-04T19:28:10.680442] [INFO ] [MAIN ] HSY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:28:10.723509] [DEBUG] [LOCAL ] HSY: Loaded 50347 existing records from primary storage. [2025-11-04T19:28:10.723845] [INFO ] [MAIN ] HSY: Loaded 50347 existing records. [2025-11-04T19:28:10.904022] [WARN ] [MAIN ] HSY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:28:10.904252] [INFO ] [MAIN ] HSY: Wiping existing primary data before full refresh. [2025-11-04T19:28:10.904308] [INFO ] [LOCAL ] HSY: Starting deletion of primary data. [2025-11-04T19:28:10.905239] [DEBUG] [LOCAL ] HSY: Deleted primary data file. [2025-11-04T19:28:10.905297] [INFO ] [MAIN ] HSY: Starting full primary refresh. [2025-11-04T19:28:11.279565] [INFO ] [REMOTE ] HSY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:28:11.282307] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T19:28:11.283770] [INFO ] [REMOTE ] HSHP: Full fetch complete (22683 records). [2025-11-04T19:28:11.283900] [DEBUG] [VALIDATOR ] HSHP: Validating primary data. [2025-11-04T19:28:11.377344] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-01. [2025-11-04T19:28:11.659912] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-11-04T19:28:11.698686] [DEBUG] [VALIDATOR ] HSHP: All expected trading days are present. [2025-11-04T19:28:11.698772] [DEBUG] [VALIDATOR ] HSHP: Primary data validation passed. [2025-11-04T19:28:11.699240] [INFO ] [MAIN ] HSHP: Fetched primary data is valid. [2025-11-04T19:28:11.699278] [DEBUG] [LOCAL ] HSHP: Saving primary data. [2025-11-04T19:28:11.699923] [DEBUG] [LOCAL ] HSHP: Trimmed 22683 records to 13823 (keeping 200 days of data) [2025-11-04T19:28:11.779443] [DEBUG] [LOCAL ] HSHP: Primary data saved (13823 records). [2025-11-04T19:28:11.779569] [INFO ] [MAIN ] HSHP: Full history saved. [2025-11-04T19:28:11.779670] [INFO ] [MAIN ] Update Progress: 1055/2367 symbols completed. HSHP took 22.1s. Estimated remaining: 3h 4m 14s. Rate: 7.1 symbols/min [2025-11-04T19:28:11.949845] [INFO ] [MAIN ] HTB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:28:11.950024] [DEBUG] [LOCAL ] HTB: No existing data file found in primary storage. [2025-11-04T19:28:11.950061] [WARN ] [MAIN ] HTB: No existing data found, performing full refresh instead. [2025-11-04T19:28:11.950170] [INFO ] [MAIN ] HTB: Wiping existing primary data before full refresh. [2025-11-04T19:28:11.950210] [INFO ] [LOCAL ] HTB: Starting deletion of primary data. [2025-11-04T19:28:11.950250] [DEBUG] [LOCAL ] HTB: No primary data file found to delete. [2025-11-04T19:28:11.950282] [INFO ] [MAIN ] HTB: Starting full primary refresh. [2025-11-04T19:28:12.172241] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-11. [2025-11-04T19:28:12.301970] [INFO ] [REMOTE ] HTB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:28:12.967018] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-01. [2025-11-04T19:28:12.999473] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-11-04T19:28:13.140191] [DEBUG] [REMOTE ] HTB: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T19:28:13.140461] [WARN ] [REMOTE ] HTB: No data returned for month 2025-01 [2025-11-04T19:28:13.140548] [FATAL] [REMOTE ] HTB: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:28:13.140615] [FATAL] [MAIN ] HTB: Failed to fetch primary data. [2025-11-04T19:28:13.140700] [WARN ] [MAIN ] HTB: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:28:13.140784] [INFO ] [LOCAL ] HTB: Starting deletion of primary data. [2025-11-04T19:28:13.140846] [DEBUG] [LOCAL ] HTB: No primary data file found to delete. [2025-11-04T19:28:13.140882] [INFO ] [LOCAL ] HTB: Starting deletion of derivative data. [2025-11-04T19:28:13.140920] [DEBUG] [LOCAL ] HTB: No derivative data file found to delete. [2025-11-04T19:28:13.140963] [INFO ] [MAIN ] Update Progress: 1056/2367 symbols completed. HTB took 1.4s. Estimated remaining: 3h 3m 57s. Rate: 7.1 symbols/min [2025-11-04T19:28:13.310139] [INFO ] [MAIN ] HTD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:28:13.323219] [DEBUG] [LOCAL ] HTD: Loaded 13973 existing records from primary storage. [2025-11-04T19:28:13.323431] [INFO ] [MAIN ] HTD: Loaded 13973 existing records. [2025-11-04T19:28:13.496254] [WARN ] [MAIN ] HTD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:28:13.496452] [INFO ] [MAIN ] HTD: Wiping existing primary data before full refresh. [2025-11-04T19:28:13.496499] [INFO ] [LOCAL ] HTD: Starting deletion of primary data. [2025-11-04T19:28:13.496922] [DEBUG] [LOCAL ] HTD: Deleted primary data file. [2025-11-04T19:28:13.496981] [INFO ] [MAIN ] HTD: Starting full primary refresh. [2025-11-04T19:28:13.771607] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-02. [2025-11-04T19:28:13.779409] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-11 (1.61 seconds). [2025-11-04T19:28:13.835505] [INFO ] [REMOTE ] HRL: Full fetch complete (77057 records). [2025-11-04T19:28:13.853506] [DEBUG] [VALIDATOR ] HRL: Validating primary data. [2025-11-04T19:28:13.858589] [INFO ] [REMOTE ] HTD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:28:14.267386] [DEBUG] [VALIDATOR ] HRL: All expected trading days are present. [2025-11-04T19:28:14.267460] [DEBUG] [VALIDATOR ] HRL: Primary data validation passed. [2025-11-04T19:28:14.268438] [INFO ] [MAIN ] HRL: Fetched primary data is valid. [2025-11-04T19:28:14.268499] [DEBUG] [LOCAL ] HRL: Saving primary data. [2025-11-04T19:28:14.269914] [DEBUG] [LOCAL ] HRL: Trimmed 77057 records to 53153 (keeping 200 days of data) [2025-11-04T19:28:14.568893] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-01. [2025-11-04T19:28:14.590178] [DEBUG] [LOCAL ] HRL: Primary data saved (53153 records). [2025-11-04T19:28:14.590494] [INFO ] [MAIN ] HRL: Full history saved. [2025-11-04T19:28:14.590697] [INFO ] [MAIN ] Update Progress: 1057/2367 symbols completed. HRL took 50.8s. Estimated remaining: 3h 3m 39s. Rate: 7.1 symbols/min [2025-11-04T19:28:14.766709] [INFO ] [MAIN ] HTFC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:28:14.766883] [DEBUG] [LOCAL ] HTFC: No existing data file found in primary storage. [2025-11-04T19:28:14.766939] [WARN ] [MAIN ] HTFC: No existing data found, performing full refresh instead. [2025-11-04T19:28:14.767047] [INFO ] [MAIN ] HTFC: Wiping existing primary data before full refresh. [2025-11-04T19:28:14.767088] [INFO ] [LOCAL ] HTFC: Starting deletion of primary data. [2025-11-04T19:28:14.767129] [DEBUG] [LOCAL ] HTFC: No primary data file found to delete. [2025-11-04T19:28:14.767161] [INFO ] [MAIN ] HTFC: Starting full primary refresh. [2025-11-04T19:28:15.149488] [INFO ] [REMOTE ] HTFC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:28:15.184998] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-11-04T19:28:15.345357] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-11-04T19:28:15.367004] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-03. [2025-11-04T19:28:16.167032] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-08. [2025-11-04T19:28:16.915014] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T19:28:16.967027] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-04. [2025-11-04T19:28:17.697951] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-11-04T19:28:17.767007] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-09. [2025-11-04T19:28:18.534792] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-11-04T19:28:18.566994] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-01. [2025-11-04T19:28:18.818240] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-01 (0.25 seconds). [2025-11-04T19:28:19.273091] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-11-04T19:28:19.367015] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-10. [2025-11-04T19:28:20.167046] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-02. [2025-11-04T19:28:20.675830] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T19:28:20.967017] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-03. [2025-11-04T19:28:21.045320] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-11-04T19:28:21.536759] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T19:28:21.767054] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-04. [2025-11-04T19:28:22.447064] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T19:28:22.567060] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-11. [2025-11-04T19:28:23.367036] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-02. [2025-11-04T19:28:23.580603] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-02 (0.21 seconds). [2025-11-04T19:28:24.115488] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-11 (1.55 seconds). [2025-11-04T19:28:24.117424] [INFO ] [REMOTE ] HSBC: Full fetch complete (74298 records). [2025-11-04T19:28:24.117517] [DEBUG] [VALIDATOR ] HSBC: Validating primary data. [2025-11-04T19:28:24.175009] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-05. [2025-11-04T19:28:24.525801] [DEBUG] [VALIDATOR ] HSBC: All expected trading days are present. [2025-11-04T19:28:24.525914] [DEBUG] [VALIDATOR ] HSBC: Primary data validation passed. [2025-11-04T19:28:24.526822] [INFO ] [MAIN ] HSBC: Fetched primary data is valid. [2025-11-04T19:28:24.526886] [DEBUG] [LOCAL ] HSBC: Saving primary data. [2025-11-04T19:28:24.528255] [DEBUG] [LOCAL ] HSBC: Trimmed 74298 records to 50902 (keeping 200 days of data) [2025-11-04T19:28:24.831464] [DEBUG] [LOCAL ] HSBC: Primary data saved (50902 records). [2025-11-04T19:28:24.831607] [INFO ] [MAIN ] HSBC: Full history saved. [2025-11-04T19:28:24.831711] [INFO ] [MAIN ] Update Progress: 1058/2367 symbols completed. HSBC took 39.5s. Estimated remaining: 3h 3m 33s. Rate: 7.1 symbols/min [2025-11-04T19:28:24.972244] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-03. [2025-11-04T19:28:25.036296] [INFO ] [MAIN ] HTGC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:28:25.083818] [DEBUG] [LOCAL ] HTGC: Loaded 47480 existing records from primary storage. [2025-11-04T19:28:25.084172] [INFO ] [MAIN ] HTGC: Loaded 47480 existing records. [2025-11-04T19:28:25.277120] [WARN ] [MAIN ] HTGC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:28:25.277917] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-11-04T19:28:25.279092] [INFO ] [MAIN ] HTGC: Wiping existing primary data before full refresh. [2025-11-04T19:28:25.279174] [INFO ] [LOCAL ] HTGC: Starting deletion of primary data. [2025-11-04T19:28:25.280103] [DEBUG] [LOCAL ] HTGC: Deleted primary data file. [2025-11-04T19:28:25.280193] [INFO ] [MAIN ] HTGC: Starting full primary refresh. [2025-11-04T19:28:25.673014] [INFO ] [REMOTE ] HTGC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:28:25.704638] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-11-04T19:28:25.767002] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-06. [2025-11-04T19:28:26.567038] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-04. [2025-11-04T19:28:26.858788] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-04 (0.29 seconds). [2025-11-04T19:28:27.222466] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T19:28:27.367003] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-07. [2025-11-04T19:28:28.167037] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-01. [2025-11-04T19:28:28.921248] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-11-04T19:28:28.967036] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-05. [2025-11-04T19:28:29.205935] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-11-04T19:28:29.239874] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-11-04T19:28:29.767041] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-05. [2025-11-04T19:28:30.332849] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-11-04T19:28:30.567096] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-08. [2025-11-04T19:28:31.367080] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-02. [2025-11-04T19:28:31.981983] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-11-04T19:28:32.167067] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-06. [2025-11-04T19:28:32.664189] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-11-04T19:28:32.665538] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T19:28:32.967061] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-07. [2025-11-04T19:28:33.547307] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-11-04T19:28:33.767091] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-06. [2025-11-04T19:28:33.989623] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-06 (0.22 seconds). [2025-11-04T19:28:34.567048] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-03. [2025-11-04T19:28:35.367089] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-07. [2025-11-04T19:28:35.547020] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-07 (0.18 seconds). [2025-11-04T19:28:35.984138] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-11-04T19:28:36.167050] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-08. [2025-11-04T19:28:36.781012] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-11-04T19:28:36.967122] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-08. [2025-11-04T19:28:37.153064] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-08 (0.19 seconds). [2025-11-04T19:28:37.767054] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-09. [2025-11-04T19:28:38.396425] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:28:38.567041] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-09. [2025-11-04T19:28:39.367028] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-04. [2025-11-04T19:28:39.708502] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T19:28:40.167045] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-10. [2025-11-04T19:28:40.786192] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-11-04T19:28:40.967056] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-05. [2025-11-04T19:28:41.712096] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-11-04T19:28:41.767057] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-10. [2025-11-04T19:28:42.372077] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-11-04T19:28:42.418313] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T19:28:42.567055] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-09. [2025-11-04T19:28:42.789497] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-09 (0.22 seconds). [2025-11-04T19:28:43.367046] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-10. [2025-11-04T19:28:43.607881] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-10 (0.24 seconds). [2025-11-04T19:28:44.167031] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-11. [2025-11-04T19:28:44.504232] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T19:28:44.506248] [INFO ] [REMOTE ] HSY: Full fetch complete (73551 records). [2025-11-04T19:28:44.506366] [DEBUG] [VALIDATOR ] HSY: Validating primary data. [2025-11-04T19:28:44.974430] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-11. [2025-11-04T19:28:44.975461] [DEBUG] [VALIDATOR ] HSY: All expected trading days are present. [2025-11-04T19:28:44.976576] [DEBUG] [VALIDATOR ] HSY: Primary data validation passed. [2025-11-04T19:28:44.977590] [INFO ] [MAIN ] HSY: Fetched primary data is valid. [2025-11-04T19:28:44.977802] [DEBUG] [LOCAL ] HSY: Saving primary data. [2025-11-04T19:28:44.979405] [DEBUG] [LOCAL ] HSY: Trimmed 73551 records to 50383 (keeping 200 days of data) [2025-11-04T19:28:45.331049] [DEBUG] [LOCAL ] HSY: Primary data saved (50383 records). [2025-11-04T19:28:45.331222] [INFO ] [MAIN ] HSY: Full history saved. [2025-11-04T19:28:45.331324] [INFO ] [MAIN ] Update Progress: 1059/2367 symbols completed. HSY took 34.8s. Estimated remaining: 3h 3m 40s. Rate: 7.1 symbols/min [2025-11-04T19:28:45.508292] [INFO ] [MAIN ] HTH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:28:45.536629] [DEBUG] [LOCAL ] HTH: Loaded 32465 existing records from primary storage. [2025-11-04T19:28:45.536955] [INFO ] [MAIN ] HTH: Loaded 32465 existing records. [2025-11-04T19:28:45.713599] [WARN ] [MAIN ] HTH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:28:45.715834] [INFO ] [MAIN ] HTH: Wiping existing primary data before full refresh. [2025-11-04T19:28:45.716025] [INFO ] [LOCAL ] HTH: Starting deletion of primary data. [2025-11-04T19:28:45.715966] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-11 (0.74 seconds). [2025-11-04T19:28:45.717207] [INFO ] [REMOTE ] HTD: Full fetch complete (19426 records). [2025-11-04T19:28:45.717278] [DEBUG] [VALIDATOR ] HTD: Validating primary data. [2025-11-04T19:28:45.722955] [DEBUG] [LOCAL ] HTH: Deleted primary data file. [2025-11-04T19:28:45.769400] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-11. [2025-11-04T19:28:45.769556] [INFO ] [MAIN ] HTH: Starting full primary refresh. [2025-11-04T19:28:46.308406] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T19:28:46.324042] [INFO ] [REMOTE ] HTFC: Full fetch complete (833 records). [2025-11-04T19:28:46.335231] [DEBUG] [VALIDATOR ] HTFC: Validating primary data. [2025-11-04T19:28:46.560111] [DEBUG] [VALIDATOR ] HTD: All expected trading days are present. [2025-11-04T19:28:46.592928] [DEBUG] [VALIDATOR ] HTD: Primary data validation passed. [2025-11-04T19:28:46.599620] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-06. [2025-11-04T19:28:46.592856] [INFO ] [REMOTE ] HTH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:28:46.610949] [INFO ] [MAIN ] HTD: Fetched primary data is valid. [2025-11-04T19:28:46.622586] [DEBUG] [LOCAL ] HTD: Saving primary data. [2025-11-04T19:28:46.627387] [DEBUG] [LOCAL ] HTD: Trimmed 19426 records to 13954 (keeping 200 days of data) [2025-11-04T19:28:46.747949] [WARN ] [VALIDATOR ] HTFC: Validation failed - insufficient historical data. Have 183 unique days, need 200 days [2025-11-04T19:28:46.765849] [FATAL] [MAIN ] HTFC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:28:46.768666] [WARN ] [MAIN ] HTFC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:28:46.774697] [INFO ] [LOCAL ] HTFC: Starting deletion of primary data. [2025-11-04T19:28:46.777793] [DEBUG] [LOCAL ] HTFC: No primary data file found to delete. [2025-11-04T19:28:46.779920] [INFO ] [LOCAL ] HTFC: Starting deletion of derivative data. [2025-11-04T19:28:46.785588] [DEBUG] [LOCAL ] HTFC: No derivative data file found to delete. [2025-11-04T19:28:46.789918] [INFO ] [MAIN ] Update Progress: 1060/2367 symbols completed. HTFC took 32.2s. Estimated remaining: 3h 3m 23s. Rate: 7.1 symbols/min [2025-11-04T19:28:46.791253] [DEBUG] [LOCAL ] HTD: Primary data saved (13954 records). [2025-11-04T19:28:46.802836] [INFO ] [MAIN ] HTD: Full history saved. [2025-11-04T19:28:46.833894] [INFO ] [MAIN ] Update Progress: 1061/2367 symbols completed. HTD took 33.7s. Estimated remaining: 3h 3m 4s. Rate: 7.1 symbols/min [2025-11-04T19:28:47.126104] [INFO ] [MAIN ] HUBB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:28:47.124737] [INFO ] [MAIN ] HUBS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:28:47.176327] [DEBUG] [LOCAL ] HUBB: Loaded 35612 existing records from primary storage. [2025-11-04T19:28:47.177495] [INFO ] [MAIN ] HUBB: Loaded 35612 existing records. [2025-11-04T19:28:47.210193] [DEBUG] [LOCAL ] HUBS: Loaded 39514 existing records from primary storage. [2025-11-04T19:28:47.248211] [INFO ] [MAIN ] HUBS: Loaded 39514 existing records. [2025-11-04T19:28:47.369139] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-01. [2025-11-04T19:28:47.529485] [WARN ] [MAIN ] HUBB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:28:47.549507] [INFO ] [MAIN ] HUBB: Wiping existing primary data before full refresh. [2025-11-04T19:28:47.549592] [INFO ] [LOCAL ] HUBB: Starting deletion of primary data. [2025-11-04T19:28:47.549444] [WARN ] [MAIN ] HUBS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:28:47.549804] [INFO ] [MAIN ] HUBS: Wiping existing primary data before full refresh. [2025-11-04T19:28:47.549849] [INFO ] [LOCAL ] HUBS: Starting deletion of primary data. [2025-11-04T19:28:47.550452] [DEBUG] [LOCAL ] HUBS: Deleted primary data file. [2025-11-04T19:28:47.550519] [DEBUG] [LOCAL ] HUBB: Deleted primary data file. [2025-11-04T19:28:47.550575] [INFO ] [MAIN ] HUBS: Starting full primary refresh. [2025-11-04T19:28:47.581436] [INFO ] [MAIN ] HUBB: Starting full primary refresh. [2025-11-04T19:28:48.169386] [INFO ] [REMOTE ] HUBS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:28:48.222794] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-01. [2025-11-04T19:28:48.359620] [INFO ] [REMOTE ] HUBB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:28:48.362470] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-11-04T19:28:48.391816] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T19:28:48.967041] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-07. [2025-11-04T19:28:49.479406] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T19:28:49.767086] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-02. [2025-11-04T19:28:50.177873] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-11-04T19:28:50.567067] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-01. [2025-11-04T19:28:50.791134] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T19:28:51.367106] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-03. [2025-11-04T19:28:51.634948] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-11-04T19:28:52.172123] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-08. [2025-11-04T19:28:52.182729] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-11-04T19:28:52.967043] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-02. [2025-11-04T19:28:53.431563] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T19:28:53.767098] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-09. [2025-11-04T19:28:54.189407] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T19:28:54.567060] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-02. [2025-11-04T19:28:55.071995] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T19:28:55.367046] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-10. [2025-11-04T19:28:55.704637] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T19:28:56.167052] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-04. [2025-11-04T19:28:56.958178] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-11-04T19:28:56.968633] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-03. [2025-11-04T19:28:57.151637] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-11-04T19:28:57.767092] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-11. [2025-11-04T19:28:58.107254] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T19:28:58.567024] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-04. [2025-11-04T19:28:58.874392] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T19:28:58.876300] [INFO ] [REMOTE ] HTGC: Full fetch complete (69013 records). [2025-11-04T19:28:58.876393] [DEBUG] [VALIDATOR ] HTGC: Validating primary data. [2025-11-04T19:28:59.280496] [DEBUG] [VALIDATOR ] HTGC: All expected trading days are present. [2025-11-04T19:28:59.280573] [DEBUG] [VALIDATOR ] HTGC: Primary data validation passed. [2025-11-04T19:28:59.281412] [INFO ] [MAIN ] HTGC: Fetched primary data is valid. [2025-11-04T19:28:59.281474] [DEBUG] [LOCAL ] HTGC: Saving primary data. [2025-11-04T19:28:59.282730] [DEBUG] [LOCAL ] HTGC: Trimmed 69013 records to 47500 (keeping 200 days of data) [2025-11-04T19:28:59.371796] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-03. [2025-11-04T19:28:59.564340] [DEBUG] [LOCAL ] HTGC: Primary data saved (47500 records). [2025-11-04T19:28:59.564459] [INFO ] [MAIN ] HTGC: Full history saved. [2025-11-04T19:28:59.564533] [INFO ] [MAIN ] Update Progress: 1062/2367 symbols completed. HTGC took 34.7s. Estimated remaining: 3h 3m 1s. Rate: 7.1 symbols/min [2025-11-04T19:28:59.738745] [INFO ] [MAIN ] HUM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:28:59.779904] [DEBUG] [LOCAL ] HUM: Loaded 48954 existing records from primary storage. [2025-11-04T19:28:59.780192] [INFO ] [MAIN ] HUM: Loaded 48954 existing records. [2025-11-04T19:28:59.966104] [WARN ] [MAIN ] HUM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:28:59.966457] [INFO ] [MAIN ] HUM: Wiping existing primary data before full refresh. [2025-11-04T19:28:59.966561] [INFO ] [LOCAL ] HUM: Starting deletion of primary data. [2025-11-04T19:28:59.967694] [DEBUG] [LOCAL ] HUM: Deleted primary data file. [2025-11-04T19:28:59.967758] [INFO ] [MAIN ] HUM: Starting full primary refresh. [2025-11-04T19:29:00.170300] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-05. [2025-11-04T19:29:00.146048] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-11-04T19:29:00.363801] [INFO ] [REMOTE ] HUM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:29:00.776386] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T19:29:00.967033] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-05. [2025-11-04T19:29:01.344524] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T19:29:01.767086] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-01. [2025-11-04T19:29:02.327620] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T19:29:02.567082] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-06. [2025-11-04T19:29:03.251695] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-11-04T19:29:03.366999] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-02. [2025-11-04T19:29:03.506217] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-11-04T19:29:04.167055] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-06. [2025-11-04T19:29:04.683937] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-11-04T19:29:04.967055] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-04. [2025-11-04T19:29:05.252613] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T19:29:05.767065] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-07. [2025-11-04T19:29:06.346102] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T19:29:06.567033] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-05. [2025-11-04T19:29:06.703566] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-11-04T19:29:07.367033] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-03. [2025-11-04T19:29:07.846378] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T19:29:08.167081] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-08. [2025-11-04T19:29:08.751391] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T19:29:08.967039] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-07. [2025-11-04T19:29:09.378248] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T19:29:09.767053] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-06. [2025-11-04T19:29:10.219995] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T19:29:10.567006] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-08. [2025-11-04T19:29:10.715855] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-11-04T19:29:11.367086] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-09. [2025-11-04T19:29:11.822885] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T19:29:12.167088] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-09. [2025-11-04T19:29:12.350754] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T19:29:12.967071] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-04. [2025-11-04T19:29:13.394634] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T19:29:13.767041] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-10. [2025-11-04T19:29:14.475443] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-11-04T19:29:14.567053] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-05. [2025-11-04T19:29:14.588359] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-11-04T19:29:15.367121] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-11. [2025-11-04T19:29:16.109251] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T19:29:16.167017] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-06. [2025-11-04T19:29:16.499043] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:29:16.500808] [INFO ] [REMOTE ] HTH: Full fetch complete (47470 records). [2025-11-04T19:29:16.500913] [DEBUG] [VALIDATOR ] HTH: Validating primary data. [2025-11-04T19:29:16.888449] [DEBUG] [VALIDATOR ] HTH: All expected trading days are present. [2025-11-04T19:29:16.888535] [DEBUG] [VALIDATOR ] HTH: Primary data validation passed. [2025-11-04T19:29:16.889222] [INFO ] [MAIN ] HTH: Fetched primary data is valid. [2025-11-04T19:29:16.889283] [DEBUG] [LOCAL ] HTH: Saving primary data. [2025-11-04T19:29:16.890326] [DEBUG] [LOCAL ] HTH: Trimmed 47470 records to 32555 (keeping 200 days of data) [2025-11-04T19:29:16.974374] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-07. [2025-11-04T19:29:17.079226] [DEBUG] [LOCAL ] HTH: Primary data saved (32555 records). [2025-11-04T19:29:17.079350] [INFO ] [MAIN ] HTH: Full history saved. [2025-11-04T19:29:17.079424] [INFO ] [MAIN ] Update Progress: 1063/2367 symbols completed. HTH took 31.7s. Estimated remaining: 3h 3m 4s. Rate: 7.1 symbols/min [2025-11-04T19:29:17.245751] [INFO ] [MAIN ] HUN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:29:17.293147] [DEBUG] [LOCAL ] HUN: Loaded 53214 existing records from primary storage. [2025-11-04T19:29:17.293443] [INFO ] [MAIN ] HUN: Loaded 53214 existing records. [2025-11-04T19:29:17.473052] [WARN ] [MAIN ] HUN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:29:17.473301] [INFO ] [MAIN ] HUN: Wiping existing primary data before full refresh. [2025-11-04T19:29:17.473377] [INFO ] [LOCAL ] HUN: Starting deletion of primary data. [2025-11-04T19:29:17.474501] [DEBUG] [LOCAL ] HUN: Deleted primary data file. [2025-11-04T19:29:17.474559] [INFO ] [MAIN ] HUN: Starting full primary refresh. [2025-11-04T19:29:17.735359] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-11-04T19:29:17.774065] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-10. [2025-11-04T19:29:17.890887] [INFO ] [REMOTE ] HUN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:29:18.041903] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T19:29:18.567036] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-08. [2025-11-04T19:29:19.023126] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-11-04T19:29:19.367009] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-11. [2025-11-04T19:29:19.624902] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-11-04T19:29:20.167040] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-09. [2025-11-04T19:29:20.629771] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-11 (1.26 seconds). [2025-11-04T19:29:20.631595] [INFO ] [REMOTE ] HUBS: Full fetch complete (57579 records). [2025-11-04T19:29:20.631717] [DEBUG] [VALIDATOR ] HUBS: Validating primary data. [2025-11-04T19:29:20.971087] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-01. [2025-11-04T19:29:21.014175] [DEBUG] [VALIDATOR ] HUBS: All expected trading days are present. [2025-11-04T19:29:21.014269] [DEBUG] [VALIDATOR ] HUBS: Primary data validation passed. [2025-11-04T19:29:21.015030] [INFO ] [MAIN ] HUBS: Fetched primary data is valid. [2025-11-04T19:29:21.015091] [DEBUG] [LOCAL ] HUBS: Saving primary data. [2025-11-04T19:29:21.016202] [DEBUG] [LOCAL ] HUBS: Trimmed 57579 records to 39625 (keeping 200 days of data) [2025-11-04T19:29:21.271510] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T19:29:21.302432] [DEBUG] [LOCAL ] HUBS: Primary data saved (39625 records). [2025-11-04T19:29:21.302735] [INFO ] [MAIN ] HUBS: Full history saved. [2025-11-04T19:29:21.302869] [INFO ] [MAIN ] Update Progress: 1064/2367 symbols completed. HUBS took 34.4s. Estimated remaining: 3h 2m 50s. Rate: 7.1 symbols/min [2025-11-04T19:29:21.482238] [INFO ] [MAIN ] HUYA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:29:21.517367] [DEBUG] [LOCAL ] HUYA: Loaded 43564 existing records from primary storage. [2025-11-04T19:29:21.517673] [INFO ] [MAIN ] HUYA: Loaded 43564 existing records. [2025-11-04T19:29:21.693172] [WARN ] [MAIN ] HUYA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:29:21.693369] [INFO ] [MAIN ] HUYA: Wiping existing primary data before full refresh. [2025-11-04T19:29:21.693415] [INFO ] [LOCAL ] HUYA: Starting deletion of primary data. [2025-11-04T19:29:21.694138] [DEBUG] [LOCAL ] HUYA: Deleted primary data file. [2025-11-04T19:29:21.694195] [INFO ] [MAIN ] HUYA: Starting full primary refresh. [2025-11-04T19:29:21.772213] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-07. [2025-11-04T19:29:22.042988] [INFO ] [REMOTE ] HUYA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:29:22.264378] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-11-04T19:29:22.567041] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-01. [2025-11-04T19:29:23.244069] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-11-04T19:29:23.367055] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-08. [2025-11-04T19:29:23.865231] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-11-04T19:29:24.167084] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-10. [2025-11-04T19:29:24.805106] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-11-04T19:29:24.967032] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-09. [2025-11-04T19:29:25.306084] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-11-04T19:29:25.767047] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-02. [2025-11-04T19:29:26.315244] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T19:29:26.567076] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-10. [2025-11-04T19:29:26.851411] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T19:29:27.367043] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-03. [2025-11-04T19:29:28.133824] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-11-04T19:29:28.167002] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-11. [2025-11-04T19:29:28.611012] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T19:29:28.967031] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-04. [2025-11-04T19:29:29.430936] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-11 (1.26 seconds). [2025-11-04T19:29:29.432845] [INFO ] [REMOTE ] HUM: Full fetch complete (71108 records). [2025-11-04T19:29:29.432931] [DEBUG] [VALIDATOR ] HUM: Validating primary data. [2025-11-04T19:29:29.776550] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-11. [2025-11-04T19:29:29.823385] [DEBUG] [VALIDATOR ] HUM: All expected trading days are present. [2025-11-04T19:29:29.823470] [DEBUG] [VALIDATOR ] HUM: Primary data validation passed. [2025-11-04T19:29:29.824367] [INFO ] [MAIN ] HUM: Fetched primary data is valid. [2025-11-04T19:29:29.824430] [DEBUG] [LOCAL ] HUM: Saving primary data. [2025-11-04T19:29:29.825807] [DEBUG] [LOCAL ] HUM: Trimmed 71108 records to 48943 (keeping 200 days of data) [2025-11-04T19:29:30.123434] [DEBUG] [LOCAL ] HUM: Primary data saved (48943 records). [2025-11-04T19:29:30.123578] [INFO ] [MAIN ] HUM: Full history saved. [2025-11-04T19:29:30.123679] [INFO ] [MAIN ] Update Progress: 1065/2367 symbols completed. HUM took 30.6s. Estimated remaining: 3h 2m 42s. Rate: 7.1 symbols/min [2025-11-04T19:29:30.301759] [INFO ] [MAIN ] HVT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:29:30.327489] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:29:30.347032] [DEBUG] [LOCAL ] HVT: Loaded 14497 existing records from primary storage. [2025-11-04T19:29:30.347267] [INFO ] [MAIN ] HVT: Loaded 14497 existing records. [2025-11-04T19:29:30.523263] [WARN ] [MAIN ] HVT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:29:30.523501] [INFO ] [MAIN ] HVT: Wiping existing primary data before full refresh. [2025-11-04T19:29:30.523547] [INFO ] [LOCAL ] HVT: Starting deletion of primary data. [2025-11-04T19:29:30.523890] [DEBUG] [LOCAL ] HVT: Deleted primary data file. [2025-11-04T19:29:30.523947] [INFO ] [MAIN ] HVT: Starting full primary refresh. [2025-11-04T19:29:30.577240] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-05. [2025-11-04T19:29:30.871097] [INFO ] [REMOTE ] HVT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:29:31.030588] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T19:29:31.032477] [INFO ] [REMOTE ] HUBB: Full fetch complete (54590 records). [2025-11-04T19:29:31.032577] [DEBUG] [VALIDATOR ] HUBB: Validating primary data. [2025-11-04T19:29:31.374291] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-01. [2025-11-04T19:29:31.418366] [DEBUG] [VALIDATOR ] HUBB: All expected trading days are present. [2025-11-04T19:29:31.418444] [DEBUG] [VALIDATOR ] HUBB: Primary data validation passed. [2025-11-04T19:29:31.419190] [INFO ] [MAIN ] HUBB: Fetched primary data is valid. [2025-11-04T19:29:31.419252] [DEBUG] [LOCAL ] HUBB: Saving primary data. [2025-11-04T19:29:31.420271] [DEBUG] [LOCAL ] HUBB: Trimmed 54590 records to 35729 (keeping 200 days of data) [2025-11-04T19:29:31.644282] [DEBUG] [LOCAL ] HUBB: Primary data saved (35729 records). [2025-11-04T19:29:31.644503] [INFO ] [MAIN ] HUBB: Full history saved. [2025-11-04T19:29:31.644669] [INFO ] [MAIN ] Update Progress: 1066/2367 symbols completed. HUBB took 44.9s. Estimated remaining: 3h 2m 25s. Rate: 7.1 symbols/min [2025-11-04T19:29:31.790667] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T19:29:31.845968] [INFO ] [MAIN ] HWM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:29:31.893153] [DEBUG] [LOCAL ] HWM: Loaded 52614 existing records from primary storage. [2025-11-04T19:29:31.893458] [INFO ] [MAIN ] HWM: Loaded 52614 existing records. [2025-11-04T19:29:32.071009] [WARN ] [MAIN ] HWM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:29:32.071383] [INFO ] [MAIN ] HWM: Wiping existing primary data before full refresh. [2025-11-04T19:29:32.071475] [INFO ] [LOCAL ] HWM: Starting deletion of primary data. [2025-11-04T19:29:32.072568] [DEBUG] [LOCAL ] HWM: Deleted primary data file. [2025-11-04T19:29:32.072634] [INFO ] [MAIN ] HWM: Starting full primary refresh. [2025-11-04T19:29:32.170433] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-06. [2025-11-04T19:29:32.242388] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T19:29:32.436280] [INFO ] [REMOTE ] HWM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:29:32.967089] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-02. [2025-11-04T19:29:33.327231] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T19:29:33.767064] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-01. [2025-11-04T19:29:34.136409] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T19:29:34.567027] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-07. [2025-11-04T19:29:35.367025] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-02. [2025-11-04T19:29:35.578232] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-11-04T19:29:35.833422] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T19:29:36.074420] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T19:29:36.167066] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-08. [2025-11-04T19:29:36.967036] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-03. [2025-11-04T19:29:37.370529] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T19:29:37.728954] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T19:29:37.767034] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-04. [2025-11-04T19:29:38.567053] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-02. [2025-11-04T19:29:38.609587] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-11-04T19:29:39.367104] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-03. [2025-11-04T19:29:40.092170] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-11-04T19:29:40.167031] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-03. [2025-11-04T19:29:40.664943] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T19:29:40.967160] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-09. [2025-11-04T19:29:41.766234] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-11-04T19:29:41.776464] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-05. [2025-11-04T19:29:42.165530] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T19:29:42.484845] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-11-04T19:29:42.567042] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-04. [2025-11-04T19:29:43.367020] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-10. [2025-11-04T19:29:43.971392] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T19:29:44.167037] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-04. [2025-11-04T19:29:44.661265] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-11-04T19:29:44.967046] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-06. [2025-11-04T19:29:45.657444] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-11-04T19:29:45.765012] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-11-04T19:29:45.775249] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-07. [2025-11-04T19:29:46.414193] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-11-04T19:29:46.567008] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-05. [2025-11-04T19:29:47.367062] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-11. [2025-11-04T19:29:47.941525] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-11-04T19:29:48.167063] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-06. [2025-11-04T19:29:48.379843] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-11 (1.01 seconds). [2025-11-04T19:29:48.381510] [INFO ] [REMOTE ] HUYA: Full fetch complete (63852 records). [2025-11-04T19:29:48.381587] [DEBUG] [VALIDATOR ] HUYA: Validating primary data. [2025-11-04T19:29:48.758337] [DEBUG] [VALIDATOR ] HUYA: All expected trading days are present. [2025-11-04T19:29:48.758434] [DEBUG] [VALIDATOR ] HUYA: Primary data validation passed. [2025-11-04T19:29:48.759211] [INFO ] [MAIN ] HUYA: Fetched primary data is valid. [2025-11-04T19:29:48.759272] [DEBUG] [LOCAL ] HUYA: Saving primary data. [2025-11-04T19:29:48.760435] [DEBUG] [LOCAL ] HUYA: Trimmed 63852 records to 43494 (keeping 200 days of data) [2025-11-04T19:29:48.976473] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-08. [2025-11-04T19:29:48.995824] [DEBUG] [LOCAL ] HUYA: Primary data saved (43494 records). [2025-11-04T19:29:48.995901] [INFO ] [MAIN ] HUYA: Full history saved. [2025-11-04T19:29:48.995982] [INFO ] [MAIN ] Update Progress: 1067/2367 symbols completed. HUYA took 27.7s. Estimated remaining: 3h 2m 28s. Rate: 7.1 symbols/min [2025-11-04T19:29:49.160082] [INFO ] [MAIN ] HXL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:29:49.197032] [DEBUG] [LOCAL ] HXL: Loaded 45787 existing records from primary storage. [2025-11-04T19:29:49.197310] [INFO ] [MAIN ] HXL: Loaded 45787 existing records. [2025-11-04T19:29:49.372478] [WARN ] [MAIN ] HXL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:29:49.372834] [INFO ] [MAIN ] HXL: Wiping existing primary data before full refresh. [2025-11-04T19:29:49.372908] [INFO ] [LOCAL ] HXL: Starting deletion of primary data. [2025-11-04T19:29:49.373808] [DEBUG] [LOCAL ] HXL: Deleted primary data file. [2025-11-04T19:29:49.373869] [INFO ] [MAIN ] HXL: Starting full primary refresh. [2025-11-04T19:29:49.662177] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-11-04T19:29:49.766696] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-11-04T19:29:49.776818] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-09. [2025-11-04T19:29:49.776734] [INFO ] [REMOTE ] HXL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:29:50.319503] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T19:29:50.567084] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-05. [2025-11-04T19:29:51.367041] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-01. [2025-11-04T19:29:52.167063] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-10. [2025-11-04T19:29:52.278381] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-11-04T19:29:52.615502] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T19:29:52.825286] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-11-04T19:29:52.967034] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-11. [2025-11-04T19:29:53.534234] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T19:29:53.535521] [INFO ] [REMOTE ] HVT: Full fetch complete (23088 records). [2025-11-04T19:29:53.535594] [DEBUG] [VALIDATOR ] HVT: Validating primary data. [2025-11-04T19:29:53.776601] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-07. [2025-11-04T19:29:53.995459] [DEBUG] [VALIDATOR ] HVT: All expected trading days are present. [2025-11-04T19:29:53.995622] [DEBUG] [VALIDATOR ] HVT: Primary data validation passed. [2025-11-04T19:29:53.996334] [INFO ] [MAIN ] HVT: Fetched primary data is valid. [2025-11-04T19:29:53.996405] [DEBUG] [LOCAL ] HVT: Saving primary data. [2025-11-04T19:29:53.997249] [DEBUG] [LOCAL ] HVT: Trimmed 23088 records to 14527 (keeping 200 days of data) [2025-11-04T19:29:54.091628] [DEBUG] [LOCAL ] HVT: Primary data saved (14527 records). [2025-11-04T19:29:54.091803] [INFO ] [MAIN ] HVT: Full history saved. [2025-11-04T19:29:54.091897] [INFO ] [MAIN ] Update Progress: 1068/2367 symbols completed. HVT took 24.0s. Estimated remaining: 3h 2m 15s. Rate: 7.1 symbols/min [2025-11-04T19:29:54.301159] [INFO ] [MAIN ] HY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:29:54.312919] [DEBUG] [LOCAL ] HY: Loaded 9422 existing records from primary storage. [2025-11-04T19:29:54.313196] [INFO ] [MAIN ] HY: Loaded 9422 existing records. [2025-11-04T19:29:54.536588] [WARN ] [MAIN ] HY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:29:54.536874] [INFO ] [MAIN ] HY: Wiping existing primary data before full refresh. [2025-11-04T19:29:54.536936] [INFO ] [LOCAL ] HY: Starting deletion of primary data. [2025-11-04T19:29:54.537195] [DEBUG] [LOCAL ] HY: Deleted primary data file. [2025-11-04T19:29:54.537236] [INFO ] [MAIN ] HY: Starting full primary refresh. [2025-11-04T19:29:54.574165] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-06. [2025-11-04T19:29:54.953466] [INFO ] [REMOTE ] HY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:29:55.228729] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T19:29:55.367006] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-08. [2025-11-04T19:29:55.958906] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-11-04T19:29:56.167034] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-01. [2025-11-04T19:29:56.561638] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-11-04T19:29:56.752996] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T19:29:56.967008] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-07. [2025-11-04T19:29:57.767107] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-09. [2025-11-04T19:29:58.567064] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-02. [2025-11-04T19:29:58.651670] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-11-04T19:29:59.054024] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T19:29:59.072729] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T19:29:59.367048] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-10. [2025-11-04T19:30:00.167081] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-03. [2025-11-04T19:30:00.771162] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-11-04T19:30:00.909355] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-11-04T19:30:00.966999] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-11. [2025-11-04T19:30:01.767046] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-02. [2025-11-04T19:30:02.567102] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-08. [2025-11-04T19:30:02.666950] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-11 (1.70 seconds). [2025-11-04T19:30:02.669381] [INFO ] [REMOTE ] HUN: Full fetch complete (77048 records). [2025-11-04T19:30:02.669544] [DEBUG] [VALIDATOR ] HUN: Validating primary data. [2025-11-04T19:30:03.167071] [DEBUG] [VALIDATOR ] HUN: All expected trading days are present. [2025-11-04T19:30:03.167302] [DEBUG] [VALIDATOR ] HUN: Primary data validation passed. [2025-11-04T19:30:03.168777] [INFO ] [MAIN ] HUN: Fetched primary data is valid. [2025-11-04T19:30:03.168859] [DEBUG] [LOCAL ] HUN: Saving primary data. [2025-11-04T19:30:03.185490] [DEBUG] [LOCAL ] HUN: Trimmed 77048 records to 53216 (keeping 200 days of data) [2025-11-04T19:30:03.199205] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-11-04T19:30:03.420092] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-04. [2025-11-04T19:30:03.603145] [DEBUG] [LOCAL ] HUN: Primary data saved (53216 records). [2025-11-04T19:30:03.603326] [INFO ] [MAIN ] HUN: Full history saved. [2025-11-04T19:30:03.603453] [INFO ] [MAIN ] Update Progress: 1069/2367 symbols completed. HUN took 46.5s. Estimated remaining: 3h 2m 8s. Rate: 7.1 symbols/min [2025-11-04T19:30:03.831355] [INFO ] [MAIN ] HYAC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:30:03.831556] [DEBUG] [LOCAL ] HYAC: No existing data file found in primary storage. [2025-11-04T19:30:03.831600] [WARN ] [MAIN ] HYAC: No existing data found, performing full refresh instead. [2025-11-04T19:30:03.831739] [INFO ] [MAIN ] HYAC: Wiping existing primary data before full refresh. [2025-11-04T19:30:03.831783] [INFO ] [LOCAL ] HYAC: Starting deletion of primary data. [2025-11-04T19:30:03.831827] [DEBUG] [LOCAL ] HYAC: No primary data file found to delete. [2025-11-04T19:30:03.831860] [INFO ] [MAIN ] HYAC: Starting full primary refresh. [2025-11-04T19:30:04.165494] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T19:30:04.171003] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-03. [2025-11-04T19:30:04.239378] [INFO ] [REMOTE ] HYAC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:30:04.281680] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-11-04T19:30:04.967010] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-01. [2025-11-04T19:30:05.214195] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-01 (0.25 seconds). [2025-11-04T19:30:05.484115] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T19:30:05.767070] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-02. [2025-11-04T19:30:06.002674] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-02 (0.24 seconds). [2025-11-04T19:30:06.567064] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-05. [2025-11-04T19:30:07.117308] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T19:30:07.367077] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-09. [2025-11-04T19:30:08.167047] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-03. [2025-11-04T19:30:08.362152] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-03 (0.20 seconds). [2025-11-04T19:30:08.967014] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-04. [2025-11-04T19:30:09.118315] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-11-04T19:30:09.207037] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-04 (0.24 seconds). [2025-11-04T19:30:09.767080] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-05. [2025-11-04T19:30:10.007924] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-11-04T19:30:10.567106] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-04. [2025-11-04T19:30:11.367090] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-10. [2025-11-04T19:30:12.163989] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-11-04T19:30:12.173448] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-06. [2025-11-04T19:30:12.598363] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-11-04T19:30:12.967008] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-05. [2025-11-04T19:30:13.220558] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-11-04T19:30:13.767102] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-11. [2025-11-04T19:30:14.411599] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-11-04T19:30:14.567085] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-06. [2025-11-04T19:30:15.367050] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-07. [2025-11-04T19:30:15.465392] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-11 (1.70 seconds). [2025-11-04T19:30:15.467483] [INFO ] [REMOTE ] HWM: Full fetch complete (76567 records). [2025-11-04T19:30:15.467575] [DEBUG] [VALIDATOR ] HWM: Validating primary data. [2025-11-04T19:30:15.909176] [DEBUG] [VALIDATOR ] HWM: All expected trading days are present. [2025-11-04T19:30:15.915529] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-11-04T19:30:15.914806] [DEBUG] [VALIDATOR ] HWM: Primary data validation passed. [2025-11-04T19:30:15.916589] [INFO ] [MAIN ] HWM: Fetched primary data is valid. [2025-11-04T19:30:15.919160] [DEBUG] [LOCAL ] HWM: Saving primary data. [2025-11-04T19:30:15.920638] [DEBUG] [LOCAL ] HWM: Trimmed 76567 records to 52606 (keeping 200 days of data) [2025-11-04T19:30:15.946598] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-11-04T19:30:16.167130] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-07. [2025-11-04T19:30:16.271797] [DEBUG] [LOCAL ] HWM: Primary data saved (52606 records). [2025-11-04T19:30:16.271921] [INFO ] [MAIN ] HWM: Full history saved. [2025-11-04T19:30:16.272011] [INFO ] [MAIN ] Update Progress: 1070/2367 symbols completed. HWM took 44.6s. Estimated remaining: 3h 2m 5s. Rate: 7.1 symbols/min [2025-11-04T19:30:16.436283] [INFO ] [MAIN ] HYBX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:30:16.436405] [DEBUG] [LOCAL ] HYBX: No existing data file found in primary storage. [2025-11-04T19:30:16.436441] [WARN ] [MAIN ] HYBX: No existing data found, performing full refresh instead. [2025-11-04T19:30:16.436573] [INFO ] [MAIN ] HYBX: Wiping existing primary data before full refresh. [2025-11-04T19:30:16.436639] [INFO ] [LOCAL ] HYBX: Starting deletion of primary data. [2025-11-04T19:30:16.436750] [DEBUG] [LOCAL ] HYBX: No primary data file found to delete. [2025-11-04T19:30:16.436785] [INFO ] [MAIN ] HYBX: Starting full primary refresh. [2025-11-04T19:30:16.782243] [INFO ] [REMOTE ] HYBX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:30:16.967028] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-01. [2025-11-04T19:30:17.191953] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-01 (0.22 seconds). [2025-11-04T19:30:17.523422] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T19:30:17.767114] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-08. [2025-11-04T19:30:18.310586] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-11-04T19:30:18.567081] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-02. [2025-11-04T19:30:18.756988] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-02 (0.19 seconds). [2025-11-04T19:30:19.367098] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-08. [2025-11-04T19:30:20.167035] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-03. [2025-11-04T19:30:20.363847] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-03 (0.20 seconds). [2025-11-04T19:30:20.603750] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-11-04T19:30:20.967084] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-09. [2025-11-04T19:30:21.767060] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-04. [2025-11-04T19:30:21.993084] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-04 (0.23 seconds). [2025-11-04T19:30:22.160892] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T19:30:22.567034] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-09. [2025-11-04T19:30:23.192765] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:30:23.367076] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-10. [2025-11-04T19:30:23.829763] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-11-04T19:30:24.167074] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-06. [2025-11-04T19:30:24.482605] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-06 (0.32 seconds). [2025-11-04T19:30:24.967081] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-05. [2025-11-04T19:30:25.196573] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-05 (0.23 seconds). [2025-11-04T19:30:25.767049] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-10. [2025-11-04T19:30:26.567038] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-07. [2025-11-04T19:30:26.920594] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-11-04T19:30:27.156112] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T19:30:27.367074] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-08. [2025-11-04T19:30:27.550969] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-08 (0.18 seconds). [2025-11-04T19:30:28.167037] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-09. [2025-11-04T19:30:28.350053] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-09 (0.18 seconds). [2025-11-04T19:30:28.967052] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-11. [2025-11-04T19:30:29.485893] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-11 (0.52 seconds). [2025-11-04T19:30:29.486819] [INFO ] [REMOTE ] HY: Full fetch complete (14608 records). [2025-11-04T19:30:29.486887] [DEBUG] [VALIDATOR ] HY: Validating primary data. [2025-11-04T19:30:29.770833] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-10. [2025-11-04T19:30:29.850264] [DEBUG] [VALIDATOR ] HY: All expected trading days are present. [2025-11-04T19:30:29.850335] [DEBUG] [VALIDATOR ] HY: Primary data validation passed. [2025-11-04T19:30:29.850637] [INFO ] [MAIN ] HY: Fetched primary data is valid. [2025-11-04T19:30:29.850716] [DEBUG] [LOCAL ] HY: Saving primary data. [2025-11-04T19:30:29.851164] [DEBUG] [LOCAL ] HY: Trimmed 14608 records to 9435 (keeping 200 days of data) [2025-11-04T19:30:29.906735] [DEBUG] [LOCAL ] HY: Primary data saved (9435 records). [2025-11-04T19:30:29.906854] [INFO ] [MAIN ] HY: Full history saved. [2025-11-04T19:30:29.906928] [INFO ] [MAIN ] Update Progress: 1071/2367 symbols completed. HY took 35.8s. Estimated remaining: 3h 2m 3s. Rate: 7.1 symbols/min [2025-11-04T19:30:30.076815] [INFO ] [MAIN ] HYI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:30:30.079176] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-10 (0.31 seconds). [2025-11-04T19:30:30.090444] [DEBUG] [LOCAL ] HYI: Loaded 9599 existing records from primary storage. [2025-11-04T19:30:30.090680] [INFO ] [MAIN ] HYI: Loaded 9599 existing records. [2025-11-04T19:30:30.266098] [WARN ] [MAIN ] HYI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:30:30.266291] [INFO ] [MAIN ] HYI: Wiping existing primary data before full refresh. [2025-11-04T19:30:30.266338] [INFO ] [LOCAL ] HYI: Starting deletion of primary data. [2025-11-04T19:30:30.266618] [DEBUG] [LOCAL ] HYI: Deleted primary data file. [2025-11-04T19:30:30.266736] [INFO ] [MAIN ] HYI: Starting full primary refresh. [2025-11-04T19:30:30.572212] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-06. [2025-11-04T19:30:30.625189] [INFO ] [REMOTE ] HYI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:30:30.847086] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-11-04T19:30:31.367023] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-07. [2025-11-04T19:30:31.612595] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-07 (0.25 seconds). [2025-11-04T19:30:32.167047] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-11. [2025-11-04T19:30:32.505297] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T19:30:32.505975] [INFO ] [REMOTE ] HYAC: Full fetch complete (895 records). [2025-11-04T19:30:32.506044] [DEBUG] [VALIDATOR ] HYAC: Validating primary data. [2025-11-04T19:30:32.679882] [WARN ] [VALIDATOR ] HYAC: Validation failed - insufficient historical data. Have 164 unique days, need 200 days [2025-11-04T19:30:32.680069] [FATAL] [MAIN ] HYAC: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:30:32.680156] [WARN ] [MAIN ] HYAC: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:30:32.680250] [INFO ] [LOCAL ] HYAC: Starting deletion of primary data. [2025-11-04T19:30:32.680331] [DEBUG] [LOCAL ] HYAC: No primary data file found to delete. [2025-11-04T19:30:32.680368] [INFO ] [LOCAL ] HYAC: Starting deletion of derivative data. [2025-11-04T19:30:32.680409] [DEBUG] [LOCAL ] HYAC: No derivative data file found to delete. [2025-11-04T19:30:32.680519] [INFO ] [MAIN ] Update Progress: 1072/2367 symbols completed. HYAC took 29.1s. Estimated remaining: 3h 1m 48s. Rate: 7.1 symbols/min [2025-11-04T19:30:32.852352] [INFO ] [MAIN ] HYT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:30:32.880162] [DEBUG] [LOCAL ] HYT: Loaded 32603 existing records from primary storage. [2025-11-04T19:30:32.880422] [INFO ] [MAIN ] HYT: Loaded 32603 existing records. [2025-11-04T19:30:32.972219] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-11. [2025-11-04T19:30:33.066701] [WARN ] [MAIN ] HYT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:30:33.066963] [INFO ] [MAIN ] HYT: Wiping existing primary data before full refresh. [2025-11-04T19:30:33.067022] [INFO ] [LOCAL ] HYT: Starting deletion of primary data. [2025-11-04T19:30:33.067555] [DEBUG] [LOCAL ] HYT: Deleted primary data file. [2025-11-04T19:30:33.067612] [INFO ] [MAIN ] HYT: Starting full primary refresh. [2025-11-04T19:30:33.418714] [INFO ] [REMOTE ] HYT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:30:33.767076] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-01. [2025-11-04T19:30:34.214947] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T19:30:34.217592] [INFO ] [REMOTE ] HXL: Full fetch complete (66844 records). [2025-11-04T19:30:34.217753] [DEBUG] [VALIDATOR ] HXL: Validating primary data. [2025-11-04T19:30:34.575073] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-01. [2025-11-04T19:30:34.611914] [DEBUG] [VALIDATOR ] HXL: All expected trading days are present. [2025-11-04T19:30:34.611996] [DEBUG] [VALIDATOR ] HXL: Primary data validation passed. [2025-11-04T19:30:34.612884] [INFO ] [MAIN ] HXL: Fetched primary data is valid. [2025-11-04T19:30:34.612946] [DEBUG] [LOCAL ] HXL: Saving primary data. [2025-11-04T19:30:34.614229] [DEBUG] [LOCAL ] HXL: Trimmed 66844 records to 45790 (keeping 200 days of data) [2025-11-04T19:30:34.844931] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T19:30:34.917184] [DEBUG] [LOCAL ] HXL: Primary data saved (45790 records). [2025-11-04T19:30:34.917320] [INFO ] [MAIN ] HXL: Full history saved. [2025-11-04T19:30:34.917444] [INFO ] [MAIN ] Update Progress: 1073/2367 symbols completed. HXL took 45.9s. Estimated remaining: 3h 1m 32s. Rate: 7.1 symbols/min [2025-11-04T19:30:35.085614] [INFO ] [MAIN ] HYTR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:30:35.089822] [DEBUG] [LOCAL ] HYTR: Loaded 3079 existing records from primary storage. [2025-11-04T19:30:35.089961] [INFO ] [MAIN ] HYTR: Loaded 3079 existing records. [2025-11-04T19:30:35.244919] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-11-04T19:30:35.276148] [WARN ] [MAIN ] HYTR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:30:35.276349] [INFO ] [MAIN ] HYTR: Wiping existing primary data before full refresh. [2025-11-04T19:30:35.276398] [INFO ] [LOCAL ] HYTR: Starting deletion of primary data. [2025-11-04T19:30:35.276597] [DEBUG] [LOCAL ] HYTR: Deleted primary data file. [2025-11-04T19:30:35.276725] [INFO ] [MAIN ] HYTR: Starting full primary refresh. [2025-11-04T19:30:35.372294] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-02. [2025-11-04T19:30:35.625504] [INFO ] [REMOTE ] HYTR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:30:35.932504] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-11-04T19:30:36.167033] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-02. [2025-11-04T19:30:36.967039] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-08. [2025-11-04T19:30:37.189621] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-08 (0.22 seconds). [2025-11-04T19:30:37.215066] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T19:30:37.767072] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-03. [2025-11-04T19:30:38.341500] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T19:30:38.567058] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-04. [2025-11-04T19:30:39.159193] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-11-04T19:30:39.367040] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-09. [2025-11-04T19:30:39.599063] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-09 (0.23 seconds). [2025-11-04T19:30:40.167073] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-05. [2025-11-04T19:30:40.660333] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-11-04T19:30:40.967039] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-01. [2025-11-04T19:30:41.299730] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-11-04T19:30:41.767077] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-10. [2025-11-04T19:30:42.005420] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-10 (0.24 seconds). [2025-11-04T19:30:42.567046] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-06. [2025-11-04T19:30:43.088048] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-11-04T19:30:43.367040] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-11. [2025-11-04T19:30:43.694849] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-11 (0.33 seconds). [2025-11-04T19:30:43.695451] [INFO ] [REMOTE ] HYBX: Full fetch complete (682 records). [2025-11-04T19:30:43.695516] [DEBUG] [VALIDATOR ] HYBX: Validating primary data. [2025-11-04T19:30:43.872668] [WARN ] [VALIDATOR ] HYBX: Validation failed - insufficient historical data. Have 179 unique days, need 200 days [2025-11-04T19:30:43.872877] [FATAL] [MAIN ] HYBX: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:30:43.872954] [WARN ] [MAIN ] HYBX: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:30:43.873046] [INFO ] [LOCAL ] HYBX: Starting deletion of primary data. [2025-11-04T19:30:43.873110] [DEBUG] [LOCAL ] HYBX: No primary data file found to delete. [2025-11-04T19:30:43.873146] [INFO ] [LOCAL ] HYBX: Starting deletion of derivative data. [2025-11-04T19:30:43.873184] [DEBUG] [LOCAL ] HYBX: No derivative data file found to delete. [2025-11-04T19:30:43.873239] [INFO ] [MAIN ] Update Progress: 1074/2367 symbols completed. HYBX took 27.6s. Estimated remaining: 3h 1m 24s. Rate: 7.1 symbols/min [2025-11-04T19:30:44.039430] [INFO ] [MAIN ] HZO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:30:44.064050] [DEBUG] [LOCAL ] HZO: Loaded 27635 existing records from primary storage. [2025-11-04T19:30:44.064307] [INFO ] [MAIN ] HZO: Loaded 27635 existing records. [2025-11-04T19:30:44.172191] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-07. [2025-11-04T19:30:44.280243] [WARN ] [MAIN ] HZO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:30:44.280441] [INFO ] [MAIN ] HZO: Wiping existing primary data before full refresh. [2025-11-04T19:30:44.280487] [INFO ] [LOCAL ] HZO: Starting deletion of primary data. [2025-11-04T19:30:44.280970] [DEBUG] [LOCAL ] HZO: Deleted primary data file. [2025-11-04T19:30:44.281026] [INFO ] [MAIN ] HZO: Starting full primary refresh. [2025-11-04T19:30:44.636734] [INFO ] [REMOTE ] HZO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:30:44.768198] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-11-04T19:30:44.967047] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-01. [2025-11-04T19:30:45.767108] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-02. [2025-11-04T19:30:45.777022] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T19:30:46.089349] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T19:30:46.567049] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-03. [2025-11-04T19:30:47.367056] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-02. [2025-11-04T19:30:47.666956] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T19:30:48.105562] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-11-04T19:30:48.167012] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-03. [2025-11-04T19:30:48.967055] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-04. [2025-11-04T19:30:49.022218] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-11-04T19:30:49.767035] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-03. [2025-11-04T19:30:50.043545] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-11-04T19:30:50.224413] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-11-04T19:30:50.567038] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-04. [2025-11-04T19:30:50.886784] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-04 (0.32 seconds). [2025-11-04T19:30:51.367039] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-05. [2025-11-04T19:30:51.811942] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-11-04T19:30:52.167042] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-06. [2025-11-04T19:30:52.495939] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-11-04T19:30:52.967040] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-07. [2025-11-04T19:30:53.320519] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-11-04T19:30:53.767080] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-05. [2025-11-04T19:30:54.567056] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-08. [2025-11-04T19:30:54.623716] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-11-04T19:30:54.891441] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-11-04T19:30:55.367050] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-06. [2025-11-04T19:30:56.163724] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-11-04T19:30:56.170379] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-04. [2025-11-04T19:30:56.967046] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-07. [2025-11-04T19:30:57.154589] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-11-04T19:30:57.767059] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-09. [2025-11-04T19:30:57.994295] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T19:30:58.046324] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-09 (0.28 seconds). [2025-11-04T19:30:58.567036] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-08. [2025-11-04T19:30:59.367019] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-08. [2025-11-04T19:30:59.740560] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T19:30:59.807792] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T19:31:00.167051] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-10. [2025-11-04T19:31:00.536138] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-11-04T19:31:00.967041] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-11. [2025-11-04T19:31:01.437398] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-11 (0.47 seconds). [2025-11-04T19:31:01.438284] [INFO ] [REMOTE ] HYTR: Full fetch complete (4753 records). [2025-11-04T19:31:01.438364] [DEBUG] [VALIDATOR ] HYTR: Validating primary data. [2025-11-04T19:31:01.772127] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-09. [2025-11-04T19:31:01.863002] [DEBUG] [VALIDATOR ] HYTR: All expected trading days are present. [2025-11-04T19:31:01.863083] [DEBUG] [VALIDATOR ] HYTR: Primary data validation passed. [2025-11-04T19:31:01.863429] [INFO ] [MAIN ] HYTR: Fetched primary data is valid. [2025-11-04T19:31:01.863467] [DEBUG] [LOCAL ] HYTR: Saving primary data. [2025-11-04T19:31:01.863868] [DEBUG] [LOCAL ] HYTR: Trimmed 4753 records to 3083 (keeping 200 days of data) [2025-11-04T19:31:01.883111] [DEBUG] [LOCAL ] HYTR: Primary data saved (3083 records). [2025-11-04T19:31:01.883199] [INFO ] [MAIN ] HYTR: Full history saved. [2025-11-04T19:31:01.883270] [INFO ] [MAIN ] Update Progress: 1075/2367 symbols completed. HYTR took 27.0s. Estimated remaining: 3h 1m 27s. Rate: 7.1 symbols/min [2025-11-04T19:31:02.071017] [INFO ] [MAIN ] IAE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:31:02.078371] [DEBUG] [LOCAL ] IAE: Loaded 6534 existing records from primary storage. [2025-11-04T19:31:02.078580] [INFO ] [MAIN ] IAE: Loaded 6534 existing records. [2025-11-04T19:31:02.262101] [WARN ] [MAIN ] IAE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:31:02.262773] [INFO ] [MAIN ] IAE: Wiping existing primary data before full refresh. [2025-11-04T19:31:02.262907] [INFO ] [LOCAL ] IAE: Starting deletion of primary data. [2025-11-04T19:31:02.263401] [DEBUG] [LOCAL ] IAE: Deleted primary data file. [2025-11-04T19:31:02.263626] [INFO ] [MAIN ] IAE: Starting full primary refresh. [2025-11-04T19:31:02.397509] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:31:02.574168] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-09. [2025-11-04T19:31:02.661216] [INFO ] [REMOTE ] IAE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:31:03.367069] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-01. [2025-11-04T19:31:03.640633] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-11-04T19:31:03.787493] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-11-04T19:31:04.167062] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-05. [2025-11-04T19:31:04.967038] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-10. [2025-11-04T19:31:05.127297] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T19:31:05.767065] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-02. [2025-11-04T19:31:06.122621] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-11-04T19:31:06.141008] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-11-04T19:31:06.567034] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-03. [2025-11-04T19:31:06.927086] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-11-04T19:31:07.367044] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-11. [2025-11-04T19:31:08.167052] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-06. [2025-11-04T19:31:08.301934] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T19:31:08.303305] [INFO ] [REMOTE ] HYT: Full fetch complete (47349 records). [2025-11-04T19:31:08.303383] [DEBUG] [VALIDATOR ] HYT: Validating primary data. [2025-11-04T19:31:08.671614] [DEBUG] [VALIDATOR ] HYT: All expected trading days are present. [2025-11-04T19:31:08.671722] [DEBUG] [VALIDATOR ] HYT: Primary data validation passed. [2025-11-04T19:31:08.672407] [INFO ] [MAIN ] HYT: Fetched primary data is valid. [2025-11-04T19:31:08.672449] [DEBUG] [LOCAL ] HYT: Saving primary data. [2025-11-04T19:31:08.673494] [DEBUG] [LOCAL ] HYT: Trimmed 47349 records to 32574 (keeping 200 days of data) [2025-11-04T19:31:08.863125] [DEBUG] [LOCAL ] HYT: Primary data saved (32574 records). [2025-11-04T19:31:08.863257] [INFO ] [MAIN ] HYT: Full history saved. [2025-11-04T19:31:08.863335] [INFO ] [MAIN ] Update Progress: 1076/2367 symbols completed. HYT took 36.2s. Estimated remaining: 3h 1m 17s. Rate: 7.1 symbols/min [2025-11-04T19:31:08.968432] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-10. [2025-11-04T19:31:09.040445] [INFO ] [MAIN ] IAG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:31:09.058893] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T19:31:09.108738] [DEBUG] [LOCAL ] IAG: Loaded 53491 existing records from primary storage. [2025-11-04T19:31:09.109109] [INFO ] [MAIN ] IAG: Loaded 53491 existing records. [2025-11-04T19:31:09.282064] [WARN ] [MAIN ] IAG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:31:09.282275] [INFO ] [MAIN ] IAG: Wiping existing primary data before full refresh. [2025-11-04T19:31:09.282337] [INFO ] [LOCAL ] IAG: Starting deletion of primary data. [2025-11-04T19:31:09.283205] [DEBUG] [LOCAL ] IAG: Deleted primary data file. [2025-11-04T19:31:09.283267] [INFO ] [MAIN ] IAG: Starting full primary refresh. [2025-11-04T19:31:09.630941] [INFO ] [REMOTE ] IAG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:31:09.694801] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-11-04T19:31:09.767007] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-11. [2025-11-04T19:31:10.317233] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T19:31:10.318148] [INFO ] [REMOTE ] HYI: Full fetch complete (14206 records). [2025-11-04T19:31:10.318238] [DEBUG] [VALIDATOR ] HYI: Validating primary data. [2025-11-04T19:31:10.572111] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-01. [2025-11-04T19:31:10.674952] [DEBUG] [VALIDATOR ] HYI: All expected trading days are present. [2025-11-04T19:31:10.675028] [DEBUG] [VALIDATOR ] HYI: Primary data validation passed. [2025-11-04T19:31:10.675342] [INFO ] [MAIN ] HYI: Fetched primary data is valid. [2025-11-04T19:31:10.675384] [DEBUG] [LOCAL ] HYI: Saving primary data. [2025-11-04T19:31:10.675917] [DEBUG] [LOCAL ] HYI: Trimmed 14206 records to 9655 (keeping 200 days of data) [2025-11-04T19:31:10.735353] [DEBUG] [LOCAL ] HYI: Primary data saved (9655 records). [2025-11-04T19:31:10.735517] [INFO ] [MAIN ] HYI: Full history saved. [2025-11-04T19:31:10.735609] [INFO ] [MAIN ] Update Progress: 1077/2367 symbols completed. HYI took 40.8s. Estimated remaining: 3h 1m 1s. Rate: 7.1 symbols/min [2025-11-04T19:31:11.367047] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-04. [2025-11-04T19:31:11.729749] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T19:31:11.924705] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T19:31:12.167024] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-07. [2025-11-04T19:31:12.967015] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-05. [2025-11-04T19:31:13.079918] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-11-04T19:31:13.421572] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-11-04T19:31:13.767034] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-02. [2025-11-04T19:31:14.567044] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-06. [2025-11-04T19:31:14.986900] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-11-04T19:31:15.121694] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T19:31:15.367069] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-07. [2025-11-04T19:31:15.796871] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-11-04T19:31:16.167055] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-08. [2025-11-04T19:31:16.594940] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-11-04T19:31:16.967125] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-09. [2025-11-04T19:31:17.503463] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T19:31:17.767034] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-10. [2025-11-04T19:31:18.314495] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-11-04T19:31:18.567045] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-11. [2025-11-04T19:31:19.075611] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-11 (0.51 seconds). [2025-11-04T19:31:19.076725] [INFO ] [REMOTE ] IAE: Full fetch complete (9216 records). [2025-11-04T19:31:19.076808] [DEBUG] [VALIDATOR ] IAE: Validating primary data. [2025-11-04T19:31:19.372215] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-03. [2025-11-04T19:31:19.445876] [DEBUG] [VALIDATOR ] IAE: All expected trading days are present. [2025-11-04T19:31:19.445959] [DEBUG] [VALIDATOR ] IAE: Primary data validation passed. [2025-11-04T19:31:19.446248] [INFO ] [MAIN ] IAE: Fetched primary data is valid. [2025-11-04T19:31:19.446286] [DEBUG] [LOCAL ] IAE: Saving primary data. [2025-11-04T19:31:19.449693] [DEBUG] [LOCAL ] IAE: Trimmed 9216 records to 6551 (keeping 200 days of data) [2025-11-04T19:31:19.487545] [DEBUG] [LOCAL ] IAE: Primary data saved (6551 records). [2025-11-04T19:31:19.487670] [INFO ] [MAIN ] IAE: Full history saved. [2025-11-04T19:31:19.487785] [INFO ] [MAIN ] Update Progress: 1078/2367 symbols completed. IAE took 17.6s. Estimated remaining: 3h 0m 53s. Rate: 7.1 symbols/min [2025-11-04T19:31:20.167038] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-08. [2025-11-04T19:31:20.921530] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T19:31:20.967009] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-04. [2025-11-04T19:31:21.014131] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-11-04T19:31:21.767042] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-09. [2025-11-04T19:31:22.436454] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-11-04T19:31:22.567007] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-05. [2025-11-04T19:31:22.593477] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-11-04T19:31:23.367043] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-10. [2025-11-04T19:31:23.982810] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-11-04T19:31:24.167059] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-06. [2025-11-04T19:31:24.224782] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-11-04T19:31:24.967112] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-11. [2025-11-04T19:31:25.500146] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T19:31:25.767078] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-07. [2025-11-04T19:31:25.831415] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T19:31:25.832838] [INFO ] [REMOTE ] HZO: Full fetch complete (41585 records). [2025-11-04T19:31:25.832953] [DEBUG] [VALIDATOR ] HZO: Validating primary data. [2025-11-04T19:31:26.246100] [DEBUG] [VALIDATOR ] HZO: All expected trading days are present. [2025-11-04T19:31:26.246178] [DEBUG] [VALIDATOR ] HZO: Primary data validation passed. [2025-11-04T19:31:26.246855] [INFO ] [MAIN ] HZO: Fetched primary data is valid. [2025-11-04T19:31:26.246918] [DEBUG] [LOCAL ] HZO: Saving primary data. [2025-11-04T19:31:26.247832] [DEBUG] [LOCAL ] HZO: Trimmed 41585 records to 27540 (keeping 200 days of data) [2025-11-04T19:31:26.403078] [DEBUG] [LOCAL ] HZO: Primary data saved (27540 records). [2025-11-04T19:31:26.403202] [INFO ] [MAIN ] HZO: Full history saved. [2025-11-04T19:31:26.403277] [INFO ] [MAIN ] Update Progress: 1079/2367 symbols completed. HZO took 42.5s. Estimated remaining: 3h 0m 42s. Rate: 7.1 symbols/min [2025-11-04T19:31:27.294108] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T19:31:27.305840] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-08. [2025-11-04T19:31:28.531221] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T19:31:28.543888] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-09. [2025-11-04T19:31:29.917969] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-11-04T19:31:29.929533] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-10. [2025-11-04T19:31:31.501617] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-11-04T19:31:31.514177] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-11. [2025-11-04T19:31:33.049683] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-11 (1.54 seconds). [2025-11-04T19:31:33.051661] [INFO ] [REMOTE ] IAG: Full fetch complete (77554 records). [2025-11-04T19:31:33.051786] [DEBUG] [VALIDATOR ] IAG: Validating primary data. [2025-11-04T19:31:33.479529] [DEBUG] [VALIDATOR ] IAG: All expected trading days are present. [2025-11-04T19:31:33.479609] [DEBUG] [VALIDATOR ] IAG: Primary data validation passed. [2025-11-04T19:31:33.480620] [INFO ] [MAIN ] IAG: Fetched primary data is valid. [2025-11-04T19:31:33.480723] [DEBUG] [LOCAL ] IAG: Saving primary data. [2025-11-04T19:31:33.482211] [DEBUG] [LOCAL ] IAG: Trimmed 77554 records to 53493 (keeping 200 days of data) [2025-11-04T19:31:33.791253] [DEBUG] [LOCAL ] IAG: Primary data saved (53493 records). [2025-11-04T19:31:33.791387] [INFO ] [MAIN ] IAG: Full history saved. [2025-11-04T19:31:33.791464] [INFO ] [MAIN ] Update Progress: 1080/2367 symbols completed. IAG took 24.9s. Estimated remaining: 3h 0m 33s. Rate: 7.1 symbols/min [2025-11-04T19:31:34.455638] [INFO ] [MAIN ] IBTA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:31:34.497249] [INFO ] [MAIN ] IBM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:31:34.519258] [INFO ] [MAIN ] IBP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:31:34.544861] [INFO ] [MAIN ] IBN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:31:34.580151] [DEBUG] [LOCAL ] IBP: Loaded 28107 existing records from primary storage. [2025-11-04T19:31:34.580574] [INFO ] [MAIN ] IBP: Loaded 28107 existing records. [2025-11-04T19:31:34.691580] [DEBUG] [LOCAL ] IBTA: Loaded 35017 existing records from primary storage. [2025-11-04T19:31:34.729212] [DEBUG] [LOCAL ] IBM: Loaded 53631 existing records from primary storage. [2025-11-04T19:31:34.745473] [INFO ] [MAIN ] IBM: Loaded 53631 existing records. [2025-11-04T19:31:34.758367] [INFO ] [MAIN ] IBTA: Loaded 35017 existing records. [2025-11-04T19:31:34.833985] [DEBUG] [LOCAL ] IBN: Loaded 52765 existing records from primary storage. [2025-11-04T19:31:34.942260] [INFO ] [MAIN ] IBN: Loaded 52765 existing records. [2025-11-04T19:31:35.029094] [WARN ] [MAIN ] IBP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:31:35.120229] [INFO ] [MAIN ] IBP: Wiping existing primary data before full refresh. [2025-11-04T19:31:35.166588] [INFO ] [LOCAL ] IBP: Starting deletion of primary data. [2025-11-04T19:31:35.261722] [DEBUG] [LOCAL ] IBP: Deleted primary data file. [2025-11-04T19:31:35.296796] [INFO ] [MAIN ] IBP: Starting full primary refresh. [2025-11-04T19:31:35.307937] [WARN ] [MAIN ] IBTA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:31:35.353432] [INFO ] [MAIN ] IBTA: Wiping existing primary data before full refresh. [2025-11-04T19:31:35.464561] [WARN ] [MAIN ] IBM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:31:35.526117] [INFO ] [MAIN ] IBM: Wiping existing primary data before full refresh. [2025-11-04T19:31:35.504272] [INFO ] [LOCAL ] IBTA: Starting deletion of primary data. [2025-11-04T19:31:35.520864] [WARN ] [MAIN ] IBN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:31:35.549781] [INFO ] [MAIN ] IBN: Wiping existing primary data before full refresh. [2025-11-04T19:31:35.536700] [INFO ] [LOCAL ] IBM: Starting deletion of primary data. [2025-11-04T19:31:35.550883] [DEBUG] [LOCAL ] IBM: Deleted primary data file. [2025-11-04T19:31:35.550965] [INFO ] [MAIN ] IBM: Starting full primary refresh. [2025-11-04T19:31:35.556779] [DEBUG] [LOCAL ] IBTA: Deleted primary data file. [2025-11-04T19:31:35.556932] [INFO ] [LOCAL ] IBN: Starting deletion of primary data. [2025-11-04T19:31:35.599024] [DEBUG] [LOCAL ] IBN: Deleted primary data file. [2025-11-04T19:31:35.593761] [INFO ] [MAIN ] IBTA: Starting full primary refresh. [2025-11-04T19:31:35.615153] [INFO ] [MAIN ] IBN: Starting full primary refresh. [2025-11-04T19:31:36.675931] [INFO ] [REMOTE ] IBP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:31:36.755323] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-01. [2025-11-04T19:31:36.832811] [INFO ] [REMOTE ] IBM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:31:36.858095] [INFO ] [REMOTE ] IBTA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:31:36.854803] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-01. [2025-11-04T19:31:36.886505] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-01. [2025-11-04T19:31:36.942816] [INFO ] [REMOTE ] IBN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:31:36.943542] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-01. [2025-11-04T19:31:37.634844] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-11-04T19:31:37.639905] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-02. [2025-11-04T19:31:37.839144] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T19:31:37.844515] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-02. [2025-11-04T19:31:38.266133] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T19:31:38.276171] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-02. [2025-11-04T19:31:38.474365] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-11-04T19:31:38.484777] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-02. [2025-11-04T19:31:38.502903] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-11-04T19:31:38.508626] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-03. [2025-11-04T19:31:38.599581] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T19:31:38.604456] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-03. [2025-11-04T19:31:39.584952] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T19:31:39.595594] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-03. [2025-11-04T19:31:39.614486] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T19:31:39.621541] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-04. [2025-11-04T19:31:39.882545] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-11-04T19:31:39.892373] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-04. [2025-11-04T19:31:39.995472] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-11-04T19:31:40.167006] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-03. [2025-11-04T19:31:40.486732] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-11-04T19:31:40.967065] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-05. [2025-11-04T19:31:41.117839] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T19:31:41.247106] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:31:41.690820] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T19:31:41.721046] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-11-04T19:31:41.767001] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-06. [2025-11-04T19:31:42.567122] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-05. [2025-11-04T19:31:42.652389] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T19:31:43.367049] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-04. [2025-11-04T19:31:43.620402] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T19:31:44.167092] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-04. [2025-11-04T19:31:44.586060] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T19:31:44.967078] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-07. [2025-11-04T19:31:45.767071] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-05. [2025-11-04T19:31:45.966595] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-11-04T19:31:46.004488] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T19:31:46.567046] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-08. [2025-11-04T19:31:47.175355] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T19:31:47.367113] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-06. [2025-11-04T19:31:47.742382] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-11-04T19:31:48.162236] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-11-04T19:31:48.168703] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-05. [2025-11-04T19:31:48.967078] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-09. [2025-11-04T19:31:49.767039] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-07. [2025-11-04T19:31:49.842847] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T19:31:49.966723] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-11-04T19:31:50.567031] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-06. [2025-11-04T19:31:50.675807] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-11-04T19:31:51.367051] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-06. [2025-11-04T19:31:51.687002] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T19:31:52.167093] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-10. [2025-11-04T19:31:52.967052] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-08. [2025-11-04T19:31:53.187154] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-11-04T19:31:53.191879] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T19:31:53.767044] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-07. [2025-11-04T19:31:54.105537] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T19:31:54.567039] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-09. [2025-11-04T19:31:55.367060] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-07. [2025-11-04T19:31:55.558438] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T19:31:55.779950] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-11-04T19:31:56.167084] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-10. [2025-11-04T19:31:56.595346] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-11-04T19:31:56.967037] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-08. [2025-11-04T19:31:57.268782] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T19:31:57.767059] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-11. [2025-11-04T19:31:58.567055] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-11. [2025-11-04T19:31:58.674398] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-11 (0.91 seconds). [2025-11-04T19:31:58.675917] [INFO ] [REMOTE ] IBTA: Full fetch complete (51360 records). [2025-11-04T19:31:58.675996] [DEBUG] [VALIDATOR ] IBTA: Validating primary data. [2025-11-04T19:31:59.065866] [DEBUG] [VALIDATOR ] IBTA: All expected trading days are present. [2025-11-04T19:31:59.066001] [DEBUG] [VALIDATOR ] IBTA: Primary data validation passed. [2025-11-04T19:31:59.069773] [INFO ] [MAIN ] IBTA: Fetched primary data is valid. [2025-11-04T19:31:59.069883] [DEBUG] [LOCAL ] IBTA: Saving primary data. [2025-11-04T19:31:59.071330] [DEBUG] [LOCAL ] IBTA: Trimmed 51360 records to 34929 (keeping 200 days of data) [2025-11-04T19:31:59.197833] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-08 (2.23 seconds). [2025-11-04T19:31:59.308133] [DEBUG] [LOCAL ] IBTA: Primary data saved (34929 records). [2025-11-04T19:31:59.308268] [INFO ] [MAIN ] IBTA: Full history saved. [2025-11-04T19:31:59.308348] [INFO ] [MAIN ] Update Progress: 1081/2367 symbols completed. IBTA took 25.5s. Estimated remaining: 3h 0m 45s. Rate: 7.1 symbols/min [2025-11-04T19:31:59.372546] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-09. [2025-11-04T19:31:59.438248] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T19:31:59.478117] [INFO ] [REMOTE ] IBP: Full fetch complete (41398 records). [2025-11-04T19:31:59.497911] [DEBUG] [VALIDATOR ] IBP: Validating primary data. [2025-11-04T19:31:59.497789] [INFO ] [MAIN ] ICE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:31:59.921430] [DEBUG] [LOCAL ] ICE: Loaded 53012 existing records from primary storage. [2025-11-04T19:31:59.921864] [DEBUG] [VALIDATOR ] IBP: All expected trading days are present. [2025-11-04T19:31:59.922029] [INFO ] [MAIN ] ICE: Loaded 53012 existing records. [2025-11-04T19:31:59.953685] [DEBUG] [VALIDATOR ] IBP: Primary data validation passed. [2025-11-04T19:31:59.965802] [INFO ] [MAIN ] IBP: Fetched primary data is valid. [2025-11-04T19:31:59.965881] [DEBUG] [LOCAL ] IBP: Saving primary data. [2025-11-04T19:32:00.024824] [DEBUG] [LOCAL ] IBP: Trimmed 41398 records to 28289 (keeping 200 days of data) [2025-11-04T19:32:00.194931] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-08. [2025-11-04T19:32:00.199501] [WARN ] [MAIN ] ICE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:32:00.201817] [INFO ] [MAIN ] ICE: Wiping existing primary data before full refresh. [2025-11-04T19:32:00.202212] [INFO ] [LOCAL ] ICE: Starting deletion of primary data. [2025-11-04T19:32:00.206264] [DEBUG] [LOCAL ] ICE: Deleted primary data file. [2025-11-04T19:32:00.207877] [INFO ] [MAIN ] ICE: Starting full primary refresh. [2025-11-04T19:32:00.571951] [DEBUG] [LOCAL ] IBP: Primary data saved (28289 records). [2025-11-04T19:32:00.572128] [INFO ] [MAIN ] IBP: Full history saved. [2025-11-04T19:32:00.572230] [INFO ] [MAIN ] Update Progress: 1082/2367 symbols completed. IBP took 26.8s. Estimated remaining: 3h 0m 28s. Rate: 7.1 symbols/min [2025-11-04T19:32:00.769864] [INFO ] [REMOTE ] ICE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:32:00.828267] [INFO ] [MAIN ] ICL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:32:00.859990] [DEBUG] [LOCAL ] ICL: Loaded 37401 existing records from primary storage. [2025-11-04T19:32:00.860260] [INFO ] [MAIN ] ICL: Loaded 37401 existing records. [2025-11-04T19:32:00.972227] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-01. [2025-11-04T19:32:01.040164] [WARN ] [MAIN ] ICL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:32:01.040365] [INFO ] [MAIN ] ICL: Wiping existing primary data before full refresh. [2025-11-04T19:32:01.040412] [INFO ] [LOCAL ] ICL: Starting deletion of primary data. [2025-11-04T19:32:01.041059] [DEBUG] [LOCAL ] ICL: Deleted primary data file. [2025-11-04T19:32:01.041115] [INFO ] [MAIN ] ICL: Starting full primary refresh. [2025-11-04T19:32:01.393742] [INFO ] [REMOTE ] ICL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:32:01.472125] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-11-04T19:32:01.779932] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-01. [2025-11-04T19:32:01.786138] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-11-04T19:32:02.469901] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T19:32:02.567050] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-09. [2025-11-04T19:32:02.804474] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T19:32:03.367072] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-02. [2025-11-04T19:32:03.990797] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T19:32:04.167068] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-10. [2025-11-04T19:32:04.739834] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-11-04T19:32:04.967098] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-03. [2025-11-04T19:32:05.749186] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-11-04T19:32:05.767034] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-11. [2025-11-04T19:32:06.232239] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T19:32:06.567070] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-04. [2025-11-04T19:32:07.014321] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T19:32:07.016338] [INFO ] [REMOTE ] IBN: Full fetch complete (76663 records). [2025-11-04T19:32:07.016438] [DEBUG] [VALIDATOR ] IBN: Validating primary data. [2025-11-04T19:32:07.367294] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-10. [2025-11-04T19:32:07.406816] [DEBUG] [VALIDATOR ] IBN: All expected trading days are present. [2025-11-04T19:32:07.406902] [DEBUG] [VALIDATOR ] IBN: Primary data validation passed. [2025-11-04T19:32:07.408000] [INFO ] [MAIN ] IBN: Fetched primary data is valid. [2025-11-04T19:32:07.408066] [DEBUG] [LOCAL ] IBN: Saving primary data. [2025-11-04T19:32:07.409765] [DEBUG] [LOCAL ] IBN: Trimmed 76663 records to 52759 (keeping 200 days of data) [2025-11-04T19:32:07.721715] [DEBUG] [LOCAL ] IBN: Primary data saved (52759 records). [2025-11-04T19:32:07.721889] [INFO ] [MAIN ] IBN: Full history saved. [2025-11-04T19:32:07.721993] [INFO ] [MAIN ] Update Progress: 1083/2367 symbols completed. IBN took 33.9s. Estimated remaining: 3h 0m 18s. Rate: 7.1 symbols/min [2025-11-04T19:32:07.892184] [INFO ] [MAIN ] IDA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:32:07.922485] [DEBUG] [LOCAL ] IDA: Loaded 32540 existing records from primary storage. [2025-11-04T19:32:07.922787] [INFO ] [MAIN ] IDA: Loaded 32540 existing records. [2025-11-04T19:32:08.115862] [WARN ] [MAIN ] IDA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:32:08.116171] [INFO ] [MAIN ] IDA: Wiping existing primary data before full refresh. [2025-11-04T19:32:08.116239] [INFO ] [LOCAL ] IDA: Starting deletion of primary data. [2025-11-04T19:32:08.117179] [DEBUG] [LOCAL ] IDA: Deleted primary data file. [2025-11-04T19:32:08.117315] [INFO ] [MAIN ] IDA: Starting full primary refresh. [2025-11-04T19:32:08.173846] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-02. [2025-11-04T19:32:08.484737] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-11-04T19:32:08.561298] [INFO ] [REMOTE ] IDA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:32:08.967042] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-01. [2025-11-04T19:32:09.250627] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T19:32:09.767046] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-03. [2025-11-04T19:32:09.851486] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-10 (2.48 seconds). [2025-11-04T19:32:09.895703] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-11-04T19:32:10.567042] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-05. [2025-11-04T19:32:10.888286] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T19:32:11.367093] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-11. [2025-11-04T19:32:11.764192] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-11 (0.40 seconds). [2025-11-04T19:32:11.767408] [INFO ] [REMOTE ] IBM: Full fetch complete (77750 records). [2025-11-04T19:32:11.767560] [DEBUG] [VALIDATOR ] IBM: Validating primary data. [2025-11-04T19:32:12.167913] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-02. [2025-11-04T19:32:12.168980] [DEBUG] [VALIDATOR ] IBM: All expected trading days are present. [2025-11-04T19:32:12.169053] [DEBUG] [VALIDATOR ] IBM: Primary data validation passed. [2025-11-04T19:32:12.170056] [INFO ] [MAIN ] IBM: Fetched primary data is valid. [2025-11-04T19:32:12.170215] [DEBUG] [LOCAL ] IBM: Saving primary data. [2025-11-04T19:32:12.171859] [DEBUG] [LOCAL ] IBM: Trimmed 77750 records to 53631 (keeping 200 days of data) [2025-11-04T19:32:12.295704] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-11-04T19:32:12.538730] [DEBUG] [LOCAL ] IBM: Primary data saved (53631 records). [2025-11-04T19:32:12.538879] [INFO ] [MAIN ] IBM: Full history saved. [2025-11-04T19:32:12.539639] [INFO ] [MAIN ] Update Progress: 1084/2367 symbols completed. IBM took 38.7s. Estimated remaining: 3h 0m 5s. Rate: 7.1 symbols/min [2025-11-04T19:32:12.710769] [INFO ] [MAIN ] IDE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:32:12.721335] [DEBUG] [LOCAL ] IDE: Loaded 9165 existing records from primary storage. [2025-11-04T19:32:12.721555] [INFO ] [MAIN ] IDE: Loaded 9165 existing records. [2025-11-04T19:32:12.899906] [WARN ] [MAIN ] IDE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:32:12.900111] [INFO ] [MAIN ] IDE: Wiping existing primary data before full refresh. [2025-11-04T19:32:12.900157] [INFO ] [LOCAL ] IDE: Starting deletion of primary data. [2025-11-04T19:32:12.900414] [DEBUG] [LOCAL ] IDE: Deleted primary data file. [2025-11-04T19:32:12.900470] [INFO ] [MAIN ] IDE: Starting full primary refresh. [2025-11-04T19:32:12.967800] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-06. [2025-11-04T19:32:13.301247] [INFO ] [REMOTE ] IDE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:32:13.313541] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T19:32:13.767047] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-01. [2025-11-04T19:32:14.207131] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-11-04T19:32:14.484421] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-11-04T19:32:14.567052] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-04. [2025-11-04T19:32:15.367046] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-07. [2025-11-04T19:32:15.702635] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T19:32:16.167021] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-05. [2025-11-04T19:32:16.910287] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-11-04T19:32:16.966999] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-08. [2025-11-04T19:32:17.458353] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T19:32:17.767066] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-06. [2025-11-04T19:32:18.340507] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-11-04T19:32:18.567039] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-09. [2025-11-04T19:32:18.796611] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T19:32:19.367042] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-02. [2025-11-04T19:32:19.844710] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T19:32:20.043905] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-11-04T19:32:20.167044] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-03. [2025-11-04T19:32:20.586748] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T19:32:20.967039] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-10. [2025-11-04T19:32:21.767031] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-07. [2025-11-04T19:32:22.567055] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-04. [2025-11-04T19:32:22.652163] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T19:32:22.688803] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T19:32:23.162342] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-11-04T19:32:23.367035] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-05. [2025-11-04T19:32:23.837524] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-11-04T19:32:24.167122] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-06. [2025-11-04T19:32:24.675180] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T19:32:24.967103] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-03. [2025-11-04T19:32:25.767087] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-08. [2025-11-04T19:32:25.831767] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-11-04T19:32:26.567083] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-11. [2025-11-04T19:32:26.922632] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T19:32:27.367145] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-07. [2025-11-04T19:32:27.868030] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-11-04T19:32:27.903556] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-11 (1.34 seconds). [2025-11-04T19:32:27.905629] [INFO ] [REMOTE ] ICE: Full fetch complete (77038 records). [2025-11-04T19:32:27.905800] [DEBUG] [VALIDATOR ] ICE: Validating primary data. [2025-11-04T19:32:28.172369] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-08. [2025-11-04T19:32:28.307761] [DEBUG] [VALIDATOR ] ICE: All expected trading days are present. [2025-11-04T19:32:28.307856] [DEBUG] [VALIDATOR ] ICE: Primary data validation passed. [2025-11-04T19:32:28.308809] [INFO ] [MAIN ] ICE: Fetched primary data is valid. [2025-11-04T19:32:28.308874] [DEBUG] [LOCAL ] ICE: Saving primary data. [2025-11-04T19:32:28.310330] [DEBUG] [LOCAL ] ICE: Trimmed 77038 records to 53015 (keeping 200 days of data) [2025-11-04T19:32:28.644275] [DEBUG] [LOCAL ] ICE: Primary data saved (53015 records). [2025-11-04T19:32:28.644418] [INFO ] [MAIN ] ICE: Full history saved. [2025-11-04T19:32:28.644500] [INFO ] [MAIN ] Update Progress: 1085/2367 symbols completed. ICE took 29.3s. Estimated remaining: 3h 0m 6s. Rate: 7.1 symbols/min [2025-11-04T19:32:28.840319] [INFO ] [MAIN ] IDT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:32:28.842042] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T19:32:28.863412] [DEBUG] [LOCAL ] IDT: Loaded 18591 existing records from primary storage. [2025-11-04T19:32:28.863707] [INFO ] [MAIN ] IDT: Loaded 18591 existing records. [2025-11-04T19:32:28.972260] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-04. [2025-11-04T19:32:29.081327] [WARN ] [MAIN ] IDT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:32:29.081547] [INFO ] [MAIN ] IDT: Wiping existing primary data before full refresh. [2025-11-04T19:32:29.081594] [INFO ] [LOCAL ] IDT: Starting deletion of primary data. [2025-11-04T19:32:29.081994] [DEBUG] [LOCAL ] IDT: Deleted primary data file. [2025-11-04T19:32:29.082051] [INFO ] [MAIN ] IDT: Starting full primary refresh. [2025-11-04T19:32:29.434991] [INFO ] [REMOTE ] IDT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:32:29.767046] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-01. [2025-11-04T19:32:30.094875] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T19:32:30.226462] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-11-04T19:32:30.567069] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-09. [2025-11-04T19:32:31.367047] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-09. [2025-11-04T19:32:31.588394] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T19:32:31.841729] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-11-04T19:32:32.167112] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-10. [2025-11-04T19:32:32.751866] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-11-04T19:32:32.967036] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-11. [2025-11-04T19:32:33.467731] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-11 (0.50 seconds). [2025-11-04T19:32:33.468515] [INFO ] [REMOTE ] IDE: Full fetch complete (12548 records). [2025-11-04T19:32:33.468583] [DEBUG] [VALIDATOR ] IDE: Validating primary data. [2025-11-04T19:32:33.772178] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-10. [2025-11-04T19:32:33.828677] [DEBUG] [VALIDATOR ] IDE: All expected trading days are present. [2025-11-04T19:32:33.828754] [DEBUG] [VALIDATOR ] IDE: Primary data validation passed. [2025-11-04T19:32:33.829061] [INFO ] [MAIN ] IDE: Fetched primary data is valid. [2025-11-04T19:32:33.829140] [DEBUG] [LOCAL ] IDE: Saving primary data. [2025-11-04T19:32:33.829937] [DEBUG] [LOCAL ] IDE: Trimmed 12548 records to 9180 (keeping 200 days of data) [2025-11-04T19:32:33.894605] [DEBUG] [LOCAL ] IDE: Primary data saved (9180 records). [2025-11-04T19:32:33.894804] [INFO ] [MAIN ] IDE: Full history saved. [2025-11-04T19:32:33.894930] [INFO ] [MAIN ] Update Progress: 1086/2367 symbols completed. IDE took 21.4s. Estimated remaining: 2h 59m 54s. Rate: 7.1 symbols/min [2025-11-04T19:32:34.070671] [INFO ] [MAIN ] IEX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:32:34.107258] [DEBUG] [LOCAL ] IEX: Loaded 38044 existing records from primary storage. [2025-11-04T19:32:34.107541] [INFO ] [MAIN ] IEX: Loaded 38044 existing records. [2025-11-04T19:32:34.308969] [WARN ] [MAIN ] IEX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:32:34.309179] [INFO ] [MAIN ] IEX: Wiping existing primary data before full refresh. [2025-11-04T19:32:34.309227] [INFO ] [LOCAL ] IEX: Starting deletion of primary data. [2025-11-04T19:32:34.310062] [DEBUG] [LOCAL ] IEX: Deleted primary data file. [2025-11-04T19:32:34.310124] [INFO ] [MAIN ] IEX: Starting full primary refresh. [2025-11-04T19:32:34.572401] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-02. [2025-11-04T19:32:34.678765] [INFO ] [REMOTE ] IEX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:32:34.871756] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T19:32:35.028379] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-11-04T19:32:35.367060] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-01. [2025-11-04T19:32:36.167064] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-03. [2025-11-04T19:32:36.448167] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T19:32:36.766231] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-11-04T19:32:36.967046] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-04. [2025-11-04T19:32:37.641452] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-11-04T19:32:37.767034] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-05. [2025-11-04T19:32:38.452396] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-11-04T19:32:38.567084] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-06. [2025-11-04T19:32:39.368550] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-11. [2025-11-04T19:32:39.373861] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T19:32:40.167084] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-07. [2025-11-04T19:32:40.418302] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T19:32:40.420504] [INFO ] [REMOTE ] ICL: Full fetch complete (55471 records). [2025-11-04T19:32:40.420619] [DEBUG] [VALIDATOR ] ICL: Validating primary data. [2025-11-04T19:32:40.804145] [DEBUG] [VALIDATOR ] ICL: All expected trading days are present. [2025-11-04T19:32:40.804245] [DEBUG] [VALIDATOR ] ICL: Primary data validation passed. [2025-11-04T19:32:40.805049] [INFO ] [MAIN ] ICL: Fetched primary data is valid. [2025-11-04T19:32:40.805111] [DEBUG] [LOCAL ] ICL: Saving primary data. [2025-11-04T19:32:40.806204] [DEBUG] [LOCAL ] ICL: Trimmed 55471 records to 37407 (keeping 200 days of data) [2025-11-04T19:32:40.967023] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-05. [2025-11-04T19:32:40.977359] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-11-04T19:32:41.033031] [DEBUG] [LOCAL ] ICL: Primary data saved (37407 records). [2025-11-04T19:32:41.033166] [INFO ] [MAIN ] ICL: Full history saved. [2025-11-04T19:32:41.033245] [INFO ] [MAIN ] Update Progress: 1087/2367 symbols completed. ICL took 40.5s. Estimated remaining: 2h 59m 44s. Rate: 7.1 symbols/min [2025-11-04T19:32:41.196229] [INFO ] [MAIN ] IFF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:32:41.196396] [DEBUG] [LOCAL ] IFF: No existing data file found in primary storage. [2025-11-04T19:32:41.196434] [WARN ] [MAIN ] IFF: No existing data found, performing full refresh instead. [2025-11-04T19:32:41.196550] [INFO ] [MAIN ] IFF: Wiping existing primary data before full refresh. [2025-11-04T19:32:41.196592] [INFO ] [LOCAL ] IFF: Starting deletion of primary data. [2025-11-04T19:32:41.196631] [DEBUG] [LOCAL ] IFF: No primary data file found to delete. [2025-11-04T19:32:41.196715] [INFO ] [MAIN ] IFF: Starting full primary refresh. [2025-11-04T19:32:41.542776] [INFO ] [REMOTE ] IFF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:32:41.767044] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-02. [2025-11-04T19:32:42.158793] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T19:32:42.567040] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-08. [2025-11-04T19:32:42.834384] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T19:32:43.251570] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-11-04T19:32:43.367054] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-06. [2025-11-04T19:32:44.167015] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-09. [2025-11-04T19:32:44.231910] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T19:32:44.883062] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-11-04T19:32:44.967016] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-07. [2025-11-04T19:32:45.767045] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-10. [2025-11-04T19:32:46.006878] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T19:32:46.523112] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-11-04T19:32:46.567054] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-08. [2025-11-04T19:32:47.367057] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-01. [2025-11-04T19:32:47.579139] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-11-04T19:32:48.167095] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-03. [2025-11-04T19:32:48.820610] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T19:32:48.967018] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-02. [2025-11-04T19:32:49.236679] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T19:32:49.767109] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-11. [2025-11-04T19:32:49.995492] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T19:32:50.515352] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-11 (0.75 seconds). [2025-11-04T19:32:50.516719] [INFO ] [REMOTE ] IDT: Full fetch complete (24434 records). [2025-11-04T19:32:50.516798] [DEBUG] [VALIDATOR ] IDT: Validating primary data. [2025-11-04T19:32:50.577476] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-04. [2025-11-04T19:32:50.956799] [DEBUG] [VALIDATOR ] IDT: All expected trading days are present. [2025-11-04T19:32:50.956883] [DEBUG] [VALIDATOR ] IDT: Primary data validation passed. [2025-11-04T19:32:50.957573] [INFO ] [MAIN ] IDT: Fetched primary data is valid. [2025-11-04T19:32:50.957637] [DEBUG] [LOCAL ] IDT: Saving primary data. [2025-11-04T19:32:50.958340] [DEBUG] [LOCAL ] IDT: Trimmed 24434 records to 18633 (keeping 200 days of data) [2025-11-04T19:32:51.086415] [DEBUG] [LOCAL ] IDT: Primary data saved (18633 records). [2025-11-04T19:32:51.086578] [INFO ] [MAIN ] IDT: Full history saved. [2025-11-04T19:32:51.086732] [INFO ] [MAIN ] Update Progress: 1088/2367 symbols completed. IDT took 22.4s. Estimated remaining: 2h 59m 37s. Rate: 7.1 symbols/min [2025-11-04T19:32:51.290816] [INFO ] [MAIN ] IFN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:32:51.310781] [DEBUG] [LOCAL ] IFN: Loaded 18072 existing records from primary storage. [2025-11-04T19:32:51.311041] [INFO ] [MAIN ] IFN: Loaded 18072 existing records. [2025-11-04T19:32:51.372209] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-03. [2025-11-04T19:32:51.527414] [WARN ] [MAIN ] IFN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:32:51.527685] [INFO ] [MAIN ] IFN: Wiping existing primary data before full refresh. [2025-11-04T19:32:51.527745] [INFO ] [LOCAL ] IFN: Starting deletion of primary data. [2025-11-04T19:32:51.528171] [DEBUG] [LOCAL ] IFN: Deleted primary data file. [2025-11-04T19:32:51.528250] [INFO ] [MAIN ] IFN: Starting full primary refresh. [2025-11-04T19:32:51.892164] [INFO ] [REMOTE ] IFN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:32:51.937450] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:32:52.167080] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-05. [2025-11-04T19:32:52.831199] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T19:32:52.967036] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-04. [2025-11-04T19:32:53.273967] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T19:32:53.767037] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-01. [2025-11-04T19:32:54.470934] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-11-04T19:32:54.575161] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-09. [2025-11-04T19:32:54.579135] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T19:32:55.367013] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-05. [2025-11-04T19:32:55.519891] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-11-04T19:32:56.167057] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-06. [2025-11-04T19:32:56.567277] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T19:32:56.967037] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-02. [2025-11-04T19:32:57.330774] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T19:32:57.725137] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T19:32:57.767009] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-03. [2025-11-04T19:32:58.380235] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-11-04T19:32:58.567039] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-04. [2025-11-04T19:32:59.367021] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-10. [2025-11-04T19:32:59.398105] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-11-04T19:33:00.167030] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-05. [2025-11-04T19:33:00.580743] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-11-04T19:33:00.967041] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-07. [2025-11-04T19:33:01.158980] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T19:33:01.767060] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-06. [2025-11-04T19:33:02.259027] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-11-04T19:33:02.567061] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-06. [2025-11-04T19:33:03.099633] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T19:33:03.118222] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-11-04T19:33:03.367024] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-11. [2025-11-04T19:33:04.167068] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-07. [2025-11-04T19:33:04.257381] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T19:33:04.258933] [INFO ] [REMOTE ] IDA: Full fetch complete (45373 records). [2025-11-04T19:33:04.259037] [DEBUG] [VALIDATOR ] IDA: Validating primary data. [2025-11-04T19:33:04.646252] [DEBUG] [VALIDATOR ] IDA: All expected trading days are present. [2025-11-04T19:33:04.646345] [DEBUG] [VALIDATOR ] IDA: Primary data validation passed. [2025-11-04T19:33:04.647069] [INFO ] [MAIN ] IDA: Fetched primary data is valid. [2025-11-04T19:33:04.647164] [DEBUG] [LOCAL ] IDA: Saving primary data. [2025-11-04T19:33:04.648141] [DEBUG] [LOCAL ] IDA: Trimmed 45373 records to 32527 (keeping 200 days of data) [2025-11-04T19:33:04.845808] [DEBUG] [LOCAL ] IDA: Primary data saved (32527 records). [2025-11-04T19:33:04.845945] [INFO ] [MAIN ] IDA: Full history saved. [2025-11-04T19:33:04.846025] [INFO ] [MAIN ] Update Progress: 1089/2367 symbols completed. IDA took 57.1s. Estimated remaining: 2h 59m 35s. Rate: 7.1 symbols/min [2025-11-04T19:33:04.972215] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-08. [2025-11-04T19:33:05.017665] [INFO ] [MAIN ] IFS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:33:05.038391] [DEBUG] [LOCAL ] IFS: Loaded 21527 existing records from primary storage. [2025-11-04T19:33:05.038668] [INFO ] [MAIN ] IFS: Loaded 21527 existing records. [2025-11-04T19:33:05.218237] [WARN ] [MAIN ] IFS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:33:05.218477] [INFO ] [MAIN ] IFS: Wiping existing primary data before full refresh. [2025-11-04T19:33:05.218526] [INFO ] [LOCAL ] IFS: Starting deletion of primary data. [2025-11-04T19:33:05.218964] [DEBUG] [LOCAL ] IFS: Deleted primary data file. [2025-11-04T19:33:05.219020] [INFO ] [MAIN ] IFS: Starting full primary refresh. [2025-11-04T19:33:05.598725] [INFO ] [REMOTE ] IFS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:33:05.642671] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-11-04T19:33:05.767006] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-01. [2025-11-04T19:33:06.035589] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-11-04T19:33:06.351904] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-11-04T19:33:06.567057] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-02. [2025-11-04T19:33:07.367020] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-09. [2025-11-04T19:33:07.413090] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T19:33:08.167105] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-03. [2025-11-04T19:33:08.611869] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-11-04T19:33:08.928892] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T19:33:08.967054] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-04. [2025-11-04T19:33:09.662711] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T19:33:09.767076] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-07. [2025-11-04T19:33:10.560924] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-11-04T19:33:10.566988] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-08. [2025-11-04T19:33:11.292979] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-11-04T19:33:11.367022] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-09. [2025-11-04T19:33:12.116626] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-11-04T19:33:12.167115] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-08. [2025-11-04T19:33:12.967078] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-05. [2025-11-04T19:33:13.594231] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-11-04T19:33:13.678931] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-11-04T19:33:13.767001] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-06. [2025-11-04T19:33:14.504392] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-11-04T19:33:14.567054] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-07. [2025-11-04T19:33:15.331995] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-11-04T19:33:15.367046] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-10. [2025-11-04T19:33:16.159425] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T19:33:16.166989] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-11. [2025-11-04T19:33:16.967041] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-09. [2025-11-04T19:33:17.110102] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T19:33:17.111522] [INFO ] [REMOTE ] IFN: Full fetch complete (27077 records). [2025-11-04T19:33:17.111609] [DEBUG] [VALIDATOR ] IFN: Validating primary data. [2025-11-04T19:33:17.559005] [DEBUG] [VALIDATOR ] IFN: All expected trading days are present. [2025-11-04T19:33:17.559118] [DEBUG] [VALIDATOR ] IFN: Primary data validation passed. [2025-11-04T19:33:17.560010] [INFO ] [MAIN ] IFN: Fetched primary data is valid. [2025-11-04T19:33:17.560075] [DEBUG] [LOCAL ] IFN: Saving primary data. [2025-11-04T19:33:17.561109] [DEBUG] [LOCAL ] IFN: Trimmed 27077 records to 18044 (keeping 200 days of data) [2025-11-04T19:33:17.681357] [DEBUG] [LOCAL ] IFN: Primary data saved (18044 records). [2025-11-04T19:33:17.681553] [INFO ] [MAIN ] IFN: Full history saved. [2025-11-04T19:33:17.681682] [INFO ] [MAIN ] Update Progress: 1090/2367 symbols completed. IFN took 26.6s. Estimated remaining: 2h 59m 32s. Rate: 7.1 symbols/min [2025-11-04T19:33:17.772251] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-10. [2025-11-04T19:33:17.867941] [INFO ] [MAIN ] IGA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:33:17.868116] [DEBUG] [LOCAL ] IGA: No existing data file found in primary storage. [2025-11-04T19:33:17.868157] [WARN ] [MAIN ] IGA: No existing data found, performing full refresh instead. [2025-11-04T19:33:17.868273] [INFO ] [MAIN ] IGA: Wiping existing primary data before full refresh. [2025-11-04T19:33:17.868316] [INFO ] [LOCAL ] IGA: Starting deletion of primary data. [2025-11-04T19:33:17.868357] [DEBUG] [LOCAL ] IGA: No primary data file found to delete. [2025-11-04T19:33:17.868390] [INFO ] [MAIN ] IGA: Starting full primary refresh. [2025-11-04T19:33:18.256557] [INFO ] [REMOTE ] IGA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:33:18.422965] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-11-04T19:33:18.567055] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-10. [2025-11-04T19:33:19.001121] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T19:33:19.367059] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-11. [2025-11-04T19:33:19.884560] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T19:33:20.167069] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-11. [2025-11-04T19:33:20.326748] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-11 (0.96 seconds). [2025-11-04T19:33:20.328608] [INFO ] [REMOTE ] IEX: Full fetch complete (54227 records). [2025-11-04T19:33:20.328784] [DEBUG] [VALIDATOR ] IEX: Validating primary data. [2025-11-04T19:33:20.703046] [DEBUG] [VALIDATOR ] IEX: All expected trading days are present. [2025-11-04T19:33:20.703127] [DEBUG] [VALIDATOR ] IEX: Primary data validation passed. [2025-11-04T19:33:20.703898] [INFO ] [MAIN ] IEX: Fetched primary data is valid. [2025-11-04T19:33:20.704004] [DEBUG] [LOCAL ] IEX: Saving primary data. [2025-11-04T19:33:20.705100] [DEBUG] [LOCAL ] IEX: Trimmed 54227 records to 38087 (keeping 200 days of data) [2025-11-04T19:33:20.940593] [DEBUG] [LOCAL ] IEX: Primary data saved (38087 records). [2025-11-04T19:33:20.940771] [INFO ] [MAIN ] IEX: Full history saved. [2025-11-04T19:33:20.940852] [INFO ] [MAIN ] Update Progress: 1091/2367 symbols completed. IEX took 47.0s. Estimated remaining: 2h 59m 17s. Rate: 7.1 symbols/min [2025-11-04T19:33:20.972317] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-08. [2025-11-04T19:33:21.114149] [INFO ] [MAIN ] IGD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:33:21.131259] [DEBUG] [LOCAL ] IGD: Loaded 19263 existing records from primary storage. [2025-11-04T19:33:21.131488] [INFO ] [MAIN ] IGD: Loaded 19263 existing records. [2025-11-04T19:33:21.307287] [WARN ] [MAIN ] IGD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:33:21.307510] [INFO ] [MAIN ] IGD: Wiping existing primary data before full refresh. [2025-11-04T19:33:21.307558] [INFO ] [LOCAL ] IGD: Starting deletion of primary data. [2025-11-04T19:33:21.307973] [DEBUG] [LOCAL ] IGD: Deleted primary data file. [2025-11-04T19:33:21.308030] [INFO ] [MAIN ] IGD: Starting full primary refresh. [2025-11-04T19:33:21.619755] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-11 (1.45 seconds). [2025-11-04T19:33:21.665848] [INFO ] [REMOTE ] IFF: Full fetch complete (73907 records). [2025-11-04T19:33:21.672449] [DEBUG] [VALIDATOR ] IFF: Validating primary data. [2025-11-04T19:33:21.714531] [INFO ] [REMOTE ] IGD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:33:21.771237] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-01. [2025-11-04T19:33:22.047089] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T19:33:22.124311] [DEBUG] [VALIDATOR ] IFF: All expected trading days are present. [2025-11-04T19:33:22.124385] [DEBUG] [VALIDATOR ] IFF: Primary data validation passed. [2025-11-04T19:33:22.125263] [INFO ] [MAIN ] IFF: Fetched primary data is valid. [2025-11-04T19:33:22.125324] [DEBUG] [LOCAL ] IFF: Saving primary data. [2025-11-04T19:33:22.126850] [DEBUG] [LOCAL ] IFF: Trimmed 73907 records to 51242 (keeping 200 days of data) [2025-11-04T19:33:22.437687] [DEBUG] [LOCAL ] IFF: Primary data saved (51242 records). [2025-11-04T19:33:22.437835] [INFO ] [MAIN ] IFF: Full history saved. [2025-11-04T19:33:22.437910] [INFO ] [MAIN ] Update Progress: 1092/2367 symbols completed. IFF took 41.4s. Estimated remaining: 2h 59m 1s. Rate: 7.1 symbols/min [2025-11-04T19:33:22.576308] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-09. [2025-11-04T19:33:22.609444] [INFO ] [MAIN ] IGI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:33:22.615574] [DEBUG] [LOCAL ] IGI: Loaded 4159 existing records from primary storage. [2025-11-04T19:33:22.615846] [INFO ] [MAIN ] IGI: Loaded 4159 existing records. [2025-11-04T19:33:22.796053] [WARN ] [MAIN ] IGI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:33:22.796457] [INFO ] [MAIN ] IGI: Wiping existing primary data before full refresh. [2025-11-04T19:33:22.796620] [INFO ] [LOCAL ] IGI: Starting deletion of primary data. [2025-11-04T19:33:22.796983] [DEBUG] [LOCAL ] IGI: Deleted primary data file. [2025-11-04T19:33:22.797097] [INFO ] [MAIN ] IGI: Starting full primary refresh. [2025-11-04T19:33:22.913546] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T19:33:23.172918] [INFO ] [REMOTE ] IGI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:33:23.367020] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-02. [2025-11-04T19:33:23.387412] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-11-04T19:33:24.129054] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T19:33:24.167047] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-03. [2025-11-04T19:33:24.933720] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-11-04T19:33:24.967015] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-04. [2025-11-04T19:33:25.655566] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-11-04T19:33:25.767040] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-01. [2025-11-04T19:33:26.228260] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-11-04T19:33:26.567026] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-05. [2025-11-04T19:33:27.268014] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-11-04T19:33:27.367019] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-06. [2025-11-04T19:33:28.167053] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-02. [2025-11-04T19:33:28.205775] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-11-04T19:33:28.527223] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-11-04T19:33:28.967049] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-01. [2025-11-04T19:33:29.356326] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-11-04T19:33:29.767094] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-07. [2025-11-04T19:33:30.567011] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-02. [2025-11-04T19:33:30.685219] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T19:33:30.859995] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-11-04T19:33:31.367038] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-03. [2025-11-04T19:33:31.778220] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-11-04T19:33:32.167051] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-04. [2025-11-04T19:33:32.526302] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-11-04T19:33:32.967095] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-03. [2025-11-04T19:33:33.538924] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T19:33:33.767048] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-08. [2025-11-04T19:33:34.508601] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-11-04T19:33:34.567040] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-09. [2025-11-04T19:33:35.246370] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T19:33:35.367049] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-10. [2025-11-04T19:33:36.156780] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T19:33:36.167001] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-11. [2025-11-04T19:33:36.782321] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T19:33:36.783444] [INFO ] [REMOTE ] IFS: Full fetch complete (32404 records). [2025-11-04T19:33:36.783514] [DEBUG] [VALIDATOR ] IFS: Validating primary data. [2025-11-04T19:33:36.972181] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-04. [2025-11-04T19:33:37.157753] [DEBUG] [VALIDATOR ] IFS: All expected trading days are present. [2025-11-04T19:33:37.157831] [DEBUG] [VALIDATOR ] IFS: Primary data validation passed. [2025-11-04T19:33:37.158451] [INFO ] [MAIN ] IFS: Fetched primary data is valid. [2025-11-04T19:33:37.158490] [DEBUG] [LOCAL ] IFS: Saving primary data. [2025-11-04T19:33:37.159206] [DEBUG] [LOCAL ] IFS: Trimmed 32404 records to 21576 (keeping 200 days of data) [2025-11-04T19:33:37.278794] [DEBUG] [LOCAL ] IFS: Primary data saved (21576 records). [2025-11-04T19:33:37.278908] [INFO ] [MAIN ] IFS: Full history saved. [2025-11-04T19:33:37.278980] [INFO ] [MAIN ] Update Progress: 1093/2367 symbols completed. IFS took 32.4s. Estimated remaining: 2h 58m 60s. Rate: 7.1 symbols/min [2025-11-04T19:33:37.445407] [INFO ] [MAIN ] IGR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:33:37.474667] [DEBUG] [LOCAL ] IGR: Loaded 29933 existing records from primary storage. [2025-11-04T19:33:37.474920] [INFO ] [MAIN ] IGR: Loaded 29933 existing records. [2025-11-04T19:33:37.654360] [WARN ] [MAIN ] IGR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:33:37.654695] [INFO ] [MAIN ] IGR: Wiping existing primary data before full refresh. [2025-11-04T19:33:37.654754] [INFO ] [LOCAL ] IGR: Starting deletion of primary data. [2025-11-04T19:33:37.655269] [DEBUG] [LOCAL ] IGR: Deleted primary data file. [2025-11-04T19:33:37.655326] [INFO ] [MAIN ] IGR: Starting full primary refresh. [2025-11-04T19:33:37.770634] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-05. [2025-11-04T19:33:37.818685] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-11-04T19:33:38.026586] [INFO ] [REMOTE ] IGR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:33:38.157095] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-11-04T19:33:38.567049] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-10. [2025-11-04T19:33:39.367002] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-06. [2025-11-04T19:33:39.539744] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T19:33:39.710309] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-11-04T19:33:40.167123] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-11. [2025-11-04T19:33:40.967052] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-01. [2025-11-04T19:33:41.025969] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T19:33:41.027154] [INFO ] [REMOTE ] IGD: Full fetch complete (29368 records). [2025-11-04T19:33:41.027224] [DEBUG] [VALIDATOR ] IGD: Validating primary data. [2025-11-04T19:33:41.417223] [DEBUG] [VALIDATOR ] IGD: All expected trading days are present. [2025-11-04T19:33:41.417300] [DEBUG] [VALIDATOR ] IGD: Primary data validation passed. [2025-11-04T19:33:41.417930] [INFO ] [MAIN ] IGD: Fetched primary data is valid. [2025-11-04T19:33:41.417996] [DEBUG] [LOCAL ] IGD: Saving primary data. [2025-11-04T19:33:41.418714] [DEBUG] [LOCAL ] IGD: Trimmed 29368 records to 19318 (keeping 200 days of data) [2025-11-04T19:33:41.527680] [DEBUG] [LOCAL ] IGD: Primary data saved (19318 records). [2025-11-04T19:33:41.527812] [INFO ] [MAIN ] IGD: Full history saved. [2025-11-04T19:33:41.527889] [INFO ] [MAIN ] Update Progress: 1094/2367 symbols completed. IGD took 20.6s. Estimated remaining: 2h 58m 47s. Rate: 7.1 symbols/min [2025-11-04T19:33:41.713885] [INFO ] [MAIN ] IH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:33:41.717836] [DEBUG] [LOCAL ] IH: Loaded 2659 existing records from primary storage. [2025-11-04T19:33:41.717984] [INFO ] [MAIN ] IH: Loaded 2659 existing records. [2025-11-04T19:33:41.776350] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-05. [2025-11-04T19:33:41.916832] [WARN ] [MAIN ] IH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:33:41.917022] [INFO ] [MAIN ] IH: Wiping existing primary data before full refresh. [2025-11-04T19:33:41.917070] [INFO ] [LOCAL ] IH: Starting deletion of primary data. [2025-11-04T19:33:41.917218] [DEBUG] [LOCAL ] IH: Deleted primary data file. [2025-11-04T19:33:41.917292] [INFO ] [MAIN ] IH: Starting full primary refresh. [2025-11-04T19:33:42.297221] [INFO ] [REMOTE ] IH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:33:42.292819] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T19:33:42.328937] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-11-04T19:33:42.567125] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-06. [2025-11-04T19:33:43.047294] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-11-04T19:33:43.367059] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-07. [2025-11-04T19:33:43.845519] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-11-04T19:33:44.167066] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-01. [2025-11-04T19:33:44.593036] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-01 (0.43 seconds). [2025-11-04T19:33:44.967093] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-07. [2025-11-04T19:33:45.767037] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-02. [2025-11-04T19:33:45.798831] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T19:33:46.199087] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-11-04T19:33:46.567038] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-08. [2025-11-04T19:33:47.068400] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-11-04T19:33:47.367048] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-09. [2025-11-04T19:33:47.914595] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-11-04T19:33:48.167054] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-03. [2025-11-04T19:33:48.668542] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-11-04T19:33:48.967048] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-08. [2025-11-04T19:33:49.660435] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-11-04T19:33:49.767025] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-09. [2025-11-04T19:33:50.200256] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-11-04T19:33:50.567029] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-10. [2025-11-04T19:33:51.014781] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-11-04T19:33:51.367048] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-11. [2025-11-04T19:33:51.828610] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-11 (0.46 seconds). [2025-11-04T19:33:51.829499] [INFO ] [REMOTE ] IGI: Full fetch complete (5685 records). [2025-11-04T19:33:51.829570] [DEBUG] [VALIDATOR ] IGI: Validating primary data. [2025-11-04T19:33:52.172159] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-04. [2025-11-04T19:33:52.203024] [DEBUG] [VALIDATOR ] IGI: All expected trading days are present. [2025-11-04T19:33:52.203086] [DEBUG] [VALIDATOR ] IGI: Primary data validation passed. [2025-11-04T19:33:52.203283] [INFO ] [MAIN ] IGI: Fetched primary data is valid. [2025-11-04T19:33:52.203330] [DEBUG] [LOCAL ] IGI: Saving primary data. [2025-11-04T19:33:52.203754] [DEBUG] [LOCAL ] IGI: Trimmed 5685 records to 4208 (keeping 200 days of data) [2025-11-04T19:33:52.229582] [DEBUG] [LOCAL ] IGI: Primary data saved (4208 records). [2025-11-04T19:33:52.229724] [INFO ] [MAIN ] IGI: Full history saved. [2025-11-04T19:33:52.229810] [INFO ] [MAIN ] Update Progress: 1095/2367 symbols completed. IGI took 29.8s. Estimated remaining: 2h 58m 41s. Rate: 7.1 symbols/min [2025-11-04T19:33:52.393316] [INFO ] [MAIN ] IHD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:33:52.401779] [DEBUG] [LOCAL ] IHD: Loaded 7391 existing records from primary storage. [2025-11-04T19:33:52.401986] [INFO ] [MAIN ] IHD: Loaded 7391 existing records. [2025-11-04T19:33:52.505479] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-11-04T19:33:52.584443] [WARN ] [MAIN ] IHD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:33:52.584701] [INFO ] [MAIN ] IHD: Wiping existing primary data before full refresh. [2025-11-04T19:33:52.584758] [INFO ] [LOCAL ] IHD: Starting deletion of primary data. [2025-11-04T19:33:52.585017] [DEBUG] [LOCAL ] IHD: Deleted primary data file. [2025-11-04T19:33:52.585073] [INFO ] [MAIN ] IHD: Starting full primary refresh. [2025-11-04T19:33:52.967683] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-05. [2025-11-04T19:33:52.989111] [INFO ] [REMOTE ] IHD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:33:53.291179] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-11-04T19:33:53.767048] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-06. [2025-11-04T19:33:54.073747] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-11-04T19:33:54.567052] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-07. [2025-11-04T19:33:55.077126] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-11-04T19:33:55.367047] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-02. [2025-11-04T19:33:56.167062] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-08. [2025-11-04T19:33:56.470378] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-08 (0.30 seconds). [2025-11-04T19:33:56.542486] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T19:33:56.967066] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-01. [2025-11-04T19:33:57.678284] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T19:33:57.767067] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-09. [2025-11-04T19:33:58.094427] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-09 (0.33 seconds). [2025-11-04T19:33:58.567064] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-02. [2025-11-04T19:33:59.148485] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-11-04T19:33:59.367076] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-10. [2025-11-04T19:33:59.950231] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-11-04T19:34:00.167064] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-11. [2025-11-04T19:34:00.738542] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-11 (0.57 seconds). [2025-11-04T19:34:00.739490] [INFO ] [REMOTE ] IGA: Full fetch complete (14471 records). [2025-11-04T19:34:00.739561] [DEBUG] [VALIDATOR ] IGA: Validating primary data. [2025-11-04T19:34:00.973735] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-03. [2025-11-04T19:34:01.113289] [DEBUG] [VALIDATOR ] IGA: All expected trading days are present. [2025-11-04T19:34:01.113374] [DEBUG] [VALIDATOR ] IGA: Primary data validation passed. [2025-11-04T19:34:01.113769] [INFO ] [MAIN ] IGA: Fetched primary data is valid. [2025-11-04T19:34:01.113832] [DEBUG] [LOCAL ] IGA: Saving primary data. [2025-11-04T19:34:01.114394] [DEBUG] [LOCAL ] IGA: Trimmed 14471 records to 9834 (keeping 200 days of data) [2025-11-04T19:34:01.172000] [DEBUG] [LOCAL ] IGA: Primary data saved (9834 records). [2025-11-04T19:34:01.172141] [INFO ] [MAIN ] IGA: Full history saved. [2025-11-04T19:34:01.172215] [INFO ] [MAIN ] Update Progress: 1096/2367 symbols completed. IGA took 43.5s. Estimated remaining: 2h 58m 33s. Rate: 7.1 symbols/min [2025-11-04T19:34:01.349138] [INFO ] [MAIN ] IHG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:34:01.370543] [DEBUG] [LOCAL ] IHG: Loaded 23315 existing records from primary storage. [2025-11-04T19:34:01.370841] [INFO ] [MAIN ] IHG: Loaded 23315 existing records. [2025-11-04T19:34:01.559668] [WARN ] [MAIN ] IHG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:34:01.559885] [INFO ] [MAIN ] IHG: Wiping existing primary data before full refresh. [2025-11-04T19:34:01.559933] [INFO ] [LOCAL ] IHG: Starting deletion of primary data. [2025-11-04T19:34:01.560408] [DEBUG] [LOCAL ] IHG: Deleted primary data file. [2025-11-04T19:34:01.560465] [INFO ] [MAIN ] IHG: Starting full primary refresh. [2025-11-04T19:34:01.774603] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-03. [2025-11-04T19:34:01.985459] [INFO ] [REMOTE ] IHG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:34:02.016475] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T19:34:02.331037] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-11-04T19:34:02.567044] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-10. [2025-11-04T19:34:02.881748] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-10 (0.31 seconds). [2025-11-04T19:34:03.367063] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-04. [2025-11-04T19:34:04.167093] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-04. [2025-11-04T19:34:04.524257] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-11-04T19:34:04.598670] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-11-04T19:34:04.967044] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-11. [2025-11-04T19:34:05.394125] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T19:34:05.394792] [INFO ] [REMOTE ] IH: Full fetch complete (5077 records). [2025-11-04T19:34:05.394859] [DEBUG] [VALIDATOR ] IH: Validating primary data. [2025-11-04T19:34:05.754123] [DEBUG] [VALIDATOR ] IH: All expected trading days are present. [2025-11-04T19:34:05.754202] [DEBUG] [VALIDATOR ] IH: Primary data validation passed. [2025-11-04T19:34:05.754391] [INFO ] [MAIN ] IH: Fetched primary data is valid. [2025-11-04T19:34:05.754431] [DEBUG] [LOCAL ] IH: Saving primary data. [2025-11-04T19:34:05.754843] [DEBUG] [LOCAL ] IH: Trimmed 5077 records to 2649 (keeping 200 days of data) [2025-11-04T19:34:05.767059] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-05. [2025-11-04T19:34:05.770120] [DEBUG] [LOCAL ] IH: Primary data saved (2649 records). [2025-11-04T19:34:05.770209] [INFO ] [MAIN ] IH: Full history saved. [2025-11-04T19:34:05.770315] [INFO ] [MAIN ] Update Progress: 1097/2367 symbols completed. IH took 24.2s. Estimated remaining: 2h 58m 20s. Rate: 7.1 symbols/min [2025-11-04T19:34:05.955849] [INFO ] [MAIN ] IHS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:34:05.988894] [DEBUG] [LOCAL ] IHS: Loaded 36822 existing records from primary storage. [2025-11-04T19:34:05.989206] [INFO ] [MAIN ] IHS: Loaded 36822 existing records. [2025-11-04T19:34:06.167440] [WARN ] [MAIN ] IHS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:34:06.167661] [INFO ] [MAIN ] IHS: Wiping existing primary data before full refresh. [2025-11-04T19:34:06.167722] [INFO ] [LOCAL ] IHS: Starting deletion of primary data. [2025-11-04T19:34:06.168278] [DEBUG] [LOCAL ] IHS: Deleted primary data file. [2025-11-04T19:34:06.168346] [INFO ] [MAIN ] IHS: Starting full primary refresh. [2025-11-04T19:34:06.409855] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T19:34:06.542880] [INFO ] [REMOTE ] IHS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:34:06.567025] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-05. [2025-11-04T19:34:07.367027] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-01. [2025-11-04T19:34:07.544936] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-11-04T19:34:08.167026] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-06. [2025-11-04T19:34:08.305383] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-11-04T19:34:08.967030] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-06. [2025-11-04T19:34:08.982275] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-11-04T19:34:09.447597] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-11-04T19:34:09.767038] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-01. [2025-11-04T19:34:10.567012] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-02. [2025-11-04T19:34:10.718940] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T19:34:11.355472] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T19:34:11.366993] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-03. [2025-11-04T19:34:12.167039] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-07. [2025-11-04T19:34:12.269194] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-11-04T19:34:12.967040] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-07. [2025-11-04T19:34:13.168255] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T19:34:13.363306] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-11-04T19:34:13.767039] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-08. [2025-11-04T19:34:14.173774] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-11-04T19:34:14.567042] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-08. [2025-11-04T19:34:15.367006] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-04. [2025-11-04T19:34:15.556264] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-11-04T19:34:16.167041] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-02. [2025-11-04T19:34:16.674816] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T19:34:16.960632] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T19:34:16.966989] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-09. [2025-11-04T19:34:17.767016] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-05. [2025-11-04T19:34:17.858956] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-11-04T19:34:18.575290] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-03. [2025-11-04T19:34:18.579984] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-11-04T19:34:19.367102] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-10. [2025-11-04T19:34:19.691835] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T19:34:20.172858] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-04. [2025-11-04T19:34:20.176539] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-11-04T19:34:20.967305] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-11. [2025-11-04T19:34:21.189253] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T19:34:21.767074] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-05. [2025-11-04T19:34:21.823609] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-11 (0.86 seconds). [2025-11-04T19:34:21.825344] [INFO ] [REMOTE ] IGR: Full fetch complete (44247 records). [2025-11-04T19:34:21.825413] [DEBUG] [VALIDATOR ] IGR: Validating primary data. [2025-11-04T19:34:22.193215] [DEBUG] [VALIDATOR ] IGR: All expected trading days are present. [2025-11-04T19:34:22.193288] [DEBUG] [VALIDATOR ] IGR: Primary data validation passed. [2025-11-04T19:34:22.193954] [INFO ] [MAIN ] IGR: Fetched primary data is valid. [2025-11-04T19:34:22.194014] [DEBUG] [LOCAL ] IGR: Saving primary data. [2025-11-04T19:34:22.194887] [DEBUG] [LOCAL ] IGR: Trimmed 44247 records to 29947 (keeping 200 days of data) [2025-11-04T19:34:22.363007] [DEBUG] [LOCAL ] IGR: Primary data saved (29947 records). [2025-11-04T19:34:22.363128] [INFO ] [MAIN ] IGR: Full history saved. [2025-11-04T19:34:22.363202] [INFO ] [MAIN ] Update Progress: 1098/2367 symbols completed. IGR took 45.1s. Estimated remaining: 2h 58m 21s. Rate: 7.1 symbols/min [2025-11-04T19:34:22.532234] [INFO ] [MAIN ] IIF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:34:22.537751] [DEBUG] [LOCAL ] IIF: Loaded 4811 existing records from primary storage. [2025-11-04T19:34:22.537894] [INFO ] [MAIN ] IIF: Loaded 4811 existing records. [2025-11-04T19:34:22.572191] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-06. [2025-11-04T19:34:22.714715] [WARN ] [MAIN ] IIF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:34:22.714920] [INFO ] [MAIN ] IIF: Wiping existing primary data before full refresh. [2025-11-04T19:34:22.714965] [INFO ] [LOCAL ] IIF: Starting deletion of primary data. [2025-11-04T19:34:22.715202] [DEBUG] [LOCAL ] IIF: Deleted primary data file. [2025-11-04T19:34:22.715257] [INFO ] [MAIN ] IIF: Starting full primary refresh. [2025-11-04T19:34:23.130554] [INFO ] [REMOTE ] IIF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:34:23.154933] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-11-04T19:34:23.367074] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-09. [2025-11-04T19:34:23.451049] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-11-04T19:34:23.760812] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-11-04T19:34:24.167069] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-10. [2025-11-04T19:34:24.738384] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-11-04T19:34:24.967039] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-01. [2025-11-04T19:34:25.761666] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-11-04T19:34:25.766986] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-02. [2025-11-04T19:34:26.228864] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-11-04T19:34:26.567046] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-03. [2025-11-04T19:34:26.987433] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T19:34:27.367044] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-07. [2025-11-04T19:34:28.167019] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-11. [2025-11-04T19:34:28.215895] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T19:34:28.746814] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-11 (0.58 seconds). [2025-11-04T19:34:28.747731] [INFO ] [REMOTE ] IHD: Full fetch complete (11582 records). [2025-11-04T19:34:28.747800] [DEBUG] [VALIDATOR ] IHD: Validating primary data. [2025-11-04T19:34:28.972203] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-08. [2025-11-04T19:34:29.113120] [DEBUG] [VALIDATOR ] IHD: All expected trading days are present. [2025-11-04T19:34:29.113195] [DEBUG] [VALIDATOR ] IHD: Primary data validation passed. [2025-11-04T19:34:29.113458] [INFO ] [MAIN ] IHD: Fetched primary data is valid. [2025-11-04T19:34:29.113495] [DEBUG] [LOCAL ] IHD: Saving primary data. [2025-11-04T19:34:29.113947] [DEBUG] [LOCAL ] IHD: Trimmed 11582 records to 7473 (keeping 200 days of data) [2025-11-04T19:34:29.160190] [DEBUG] [LOCAL ] IHD: Primary data saved (7473 records). [2025-11-04T19:34:29.160410] [INFO ] [MAIN ] IHD: Full history saved. [2025-11-04T19:34:29.160535] [INFO ] [MAIN ] Update Progress: 1099/2367 symbols completed. IHD took 36.9s. Estimated remaining: 2h 58m 11s. Rate: 7.1 symbols/min [2025-11-04T19:34:29.328632] [INFO ] [MAIN ] IIIN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:34:29.343246] [DEBUG] [LOCAL ] IIIN: Loaded 15785 existing records from primary storage. [2025-11-04T19:34:29.343484] [INFO ] [MAIN ] IIIN: Loaded 15785 existing records. [2025-11-04T19:34:29.517721] [WARN ] [MAIN ] IIIN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:34:29.517919] [INFO ] [MAIN ] IIIN: Wiping existing primary data before full refresh. [2025-11-04T19:34:29.517966] [INFO ] [LOCAL ] IIIN: Starting deletion of primary data. [2025-11-04T19:34:29.518342] [DEBUG] [LOCAL ] IIIN: Deleted primary data file. [2025-11-04T19:34:29.518401] [INFO ] [MAIN ] IIIN: Starting full primary refresh. [2025-11-04T19:34:29.772203] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-06. [2025-11-04T19:34:29.884389] [INFO ] [REMOTE ] IIIN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:34:29.898406] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-11-04T19:34:30.567052] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-09. [2025-11-04T19:34:30.895951] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-11-04T19:34:31.367015] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-01. [2025-11-04T19:34:31.416844] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-11-04T19:34:32.167040] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-04. [2025-11-04T19:34:32.203354] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-11-04T19:34:32.590280] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-11-04T19:34:32.967050] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-07. [2025-11-04T19:34:33.767023] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-05. [2025-11-04T19:34:34.237099] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-11-04T19:34:34.236615] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T19:34:34.567086] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-06. [2025-11-04T19:34:34.968991] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-11-04T19:34:35.367045] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-07. [2025-11-04T19:34:35.893051] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-11-04T19:34:36.167037] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-10. [2025-11-04T19:34:36.967087] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-02. [2025-11-04T19:34:37.193290] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-11-04T19:34:37.676700] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T19:34:37.767054] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-11. [2025-11-04T19:34:38.567056] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-08. [2025-11-04T19:34:38.633867] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T19:34:38.635076] [INFO ] [REMOTE ] IHG: Full fetch complete (35025 records). [2025-11-04T19:34:38.635145] [DEBUG] [VALIDATOR ] IHG: Validating primary data. [2025-11-04T19:34:39.018391] [DEBUG] [VALIDATOR ] IHG: All expected trading days are present. [2025-11-04T19:34:39.023909] [DEBUG] [VALIDATOR ] IHG: Primary data validation passed. [2025-11-04T19:34:39.024817] [INFO ] [MAIN ] IHG: Fetched primary data is valid. [2025-11-04T19:34:39.024127] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-11-04T19:34:39.024984] [DEBUG] [LOCAL ] IHG: Saving primary data. [2025-11-04T19:34:39.027953] [DEBUG] [LOCAL ] IHG: Trimmed 35025 records to 23275 (keeping 200 days of data) [2025-11-04T19:34:39.167990] [DEBUG] [LOCAL ] IHG: Primary data saved (23275 records). [2025-11-04T19:34:39.168122] [INFO ] [MAIN ] IHG: Full history saved. [2025-11-04T19:34:39.168197] [INFO ] [MAIN ] Update Progress: 1100/2367 symbols completed. IHG took 38.0s. Estimated remaining: 2h 58m 4s. Rate: 7.1 symbols/min [2025-11-04T19:34:39.334021] [INFO ] [MAIN ] IIM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:34:39.348748] [DEBUG] [LOCAL ] IIM: Loaded 15357 existing records from primary storage. [2025-11-04T19:34:39.348975] [INFO ] [MAIN ] IIM: Loaded 15357 existing records. [2025-11-04T19:34:39.372158] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-08. [2025-11-04T19:34:39.528217] [WARN ] [MAIN ] IIM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:34:39.528445] [INFO ] [MAIN ] IIM: Wiping existing primary data before full refresh. [2025-11-04T19:34:39.528499] [INFO ] [LOCAL ] IIM: Starting deletion of primary data. [2025-11-04T19:34:39.528927] [DEBUG] [LOCAL ] IIM: Deleted primary data file. [2025-11-04T19:34:39.528987] [INFO ] [MAIN ] IIM: Starting full primary refresh. [2025-11-04T19:34:39.878437] [INFO ] [REMOTE ] IIM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:34:40.167037] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-03. [2025-11-04T19:34:40.542115] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T19:34:40.887889] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-11-04T19:34:40.967044] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-04. [2025-11-04T19:34:41.767046] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-09. [2025-11-04T19:34:41.834011] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-11-04T19:34:42.567072] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-01. [2025-11-04T19:34:42.647031] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T19:34:43.367035] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-05. [2025-11-04T19:34:43.537738] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-11-04T19:34:44.064677] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-11-04T19:34:44.167009] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-06. [2025-11-04T19:34:44.791822] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-11-04T19:34:44.967062] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-02. [2025-11-04T19:34:45.762980] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-11-04T19:34:45.767793] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-07. [2025-11-04T19:34:46.567020] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-03. [2025-11-04T19:34:46.584852] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T19:34:47.328980] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T19:34:47.367039] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-08. [2025-11-04T19:34:47.979451] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-11-04T19:34:48.167044] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-04. [2025-11-04T19:34:48.845377] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-11-04T19:34:48.967042] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-09. [2025-11-04T19:34:49.318211] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-11-04T19:34:49.767068] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-05. [2025-11-04T19:34:50.403256] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-11-04T19:34:50.567032] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-06. [2025-11-04T19:34:51.332535] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T19:34:51.367124] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-09. [2025-11-04T19:34:51.902486] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-11-04T19:34:52.167040] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-10. [2025-11-04T19:34:52.909028] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-11-04T19:34:52.967035] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-11. [2025-11-04T19:34:53.508381] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T19:34:53.509420] [INFO ] [REMOTE ] IIIN: Full fetch complete (24801 records). [2025-11-04T19:34:53.509488] [DEBUG] [VALIDATOR ] IIIN: Validating primary data. [2025-11-04T19:34:53.772222] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-07. [2025-11-04T19:34:53.865536] [DEBUG] [VALIDATOR ] IIIN: All expected trading days are present. [2025-11-04T19:34:53.865603] [DEBUG] [VALIDATOR ] IIIN: Primary data validation passed. [2025-11-04T19:34:53.866120] [INFO ] [MAIN ] IIIN: Fetched primary data is valid. [2025-11-04T19:34:53.866179] [DEBUG] [LOCAL ] IIIN: Saving primary data. [2025-11-04T19:34:53.866807] [DEBUG] [LOCAL ] IIIN: Trimmed 24801 records to 15673 (keeping 200 days of data) [2025-11-04T19:34:53.956741] [DEBUG] [LOCAL ] IIIN: Primary data saved (15673 records). [2025-11-04T19:34:53.956865] [INFO ] [MAIN ] IIIN: Full history saved. [2025-11-04T19:34:53.956937] [INFO ] [MAIN ] Update Progress: 1101/2367 symbols completed. IIIN took 24.8s. Estimated remaining: 2h 58m 3s. Rate: 7.1 symbols/min [2025-11-04T19:34:54.124116] [INFO ] [MAIN ] IIPR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:34:54.151714] [DEBUG] [LOCAL ] IIPR: Loaded 28465 existing records from primary storage. [2025-11-04T19:34:54.151986] [INFO ] [MAIN ] IIPR: Loaded 28465 existing records. [2025-11-04T19:34:54.329717] [WARN ] [MAIN ] IIPR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:34:54.329919] [INFO ] [MAIN ] IIPR: Wiping existing primary data before full refresh. [2025-11-04T19:34:54.329967] [INFO ] [LOCAL ] IIPR: Starting deletion of primary data. [2025-11-04T19:34:54.330484] [DEBUG] [LOCAL ] IIPR: Deleted primary data file. [2025-11-04T19:34:54.330541] [INFO ] [MAIN ] IIPR: Starting full primary refresh. [2025-11-04T19:34:54.519762] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-11-04T19:34:54.572270] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-10. [2025-11-04T19:34:54.709795] [INFO ] [REMOTE ] IIPR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:34:55.028606] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-11-04T19:34:55.367071] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-11. [2025-11-04T19:34:55.783123] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T19:34:55.783863] [INFO ] [REMOTE ] IIF: Full fetch complete (8016 records). [2025-11-04T19:34:55.783933] [DEBUG] [VALIDATOR ] IIF: Validating primary data. [2025-11-04T19:34:56.161108] [DEBUG] [VALIDATOR ] IIF: All expected trading days are present. [2025-11-04T19:34:56.161184] [DEBUG] [VALIDATOR ] IIF: Primary data validation passed. [2025-11-04T19:34:56.161398] [INFO ] [MAIN ] IIF: Fetched primary data is valid. [2025-11-04T19:34:56.161437] [DEBUG] [LOCAL ] IIF: Saving primary data. [2025-11-04T19:34:56.161885] [DEBUG] [LOCAL ] IIF: Trimmed 8016 records to 4762 (keeping 200 days of data) [2025-11-04T19:34:56.172984] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-10. [2025-11-04T19:34:56.190610] [DEBUG] [LOCAL ] IIF: Primary data saved (4762 records). [2025-11-04T19:34:56.190727] [INFO ] [MAIN ] IIF: Full history saved. [2025-11-04T19:34:56.190814] [INFO ] [MAIN ] Update Progress: 1102/2367 symbols completed. IIF took 33.8s. Estimated remaining: 2h 57m 47s. Rate: 7.1 symbols/min [2025-11-04T19:34:56.363721] [INFO ] [MAIN ] IMAX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:34:56.399598] [DEBUG] [LOCAL ] IMAX: Loaded 43313 existing records from primary storage. [2025-11-04T19:34:56.399909] [INFO ] [MAIN ] IMAX: Loaded 43313 existing records. [2025-11-04T19:34:56.577181] [WARN ] [MAIN ] IMAX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:34:56.577382] [INFO ] [MAIN ] IMAX: Wiping existing primary data before full refresh. [2025-11-04T19:34:56.577428] [INFO ] [LOCAL ] IMAX: Starting deletion of primary data. [2025-11-04T19:34:56.578142] [DEBUG] [LOCAL ] IMAX: Deleted primary data file. [2025-11-04T19:34:56.578199] [INFO ] [MAIN ] IMAX: Starting full primary refresh. [2025-11-04T19:34:56.937881] [INFO ] [REMOTE ] IMAX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:34:56.967057] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-01. [2025-11-04T19:34:57.269698] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T19:34:57.767107] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-08. [2025-11-04T19:34:57.948974] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T19:34:58.410990] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-11-04T19:34:58.567062] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-02. [2025-11-04T19:34:59.367052] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-01. [2025-11-04T19:34:59.594290] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T19:35:00.167008] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-03. [2025-11-04T19:35:00.334188] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-11-04T19:35:00.967026] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-09. [2025-11-04T19:35:01.283957] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T19:35:01.655520] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-11-04T19:35:01.767101] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-10. [2025-11-04T19:35:02.484592] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-11-04T19:35:02.567059] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-02. [2025-11-04T19:35:03.367016] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-11. [2025-11-04T19:35:03.554137] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T19:35:03.929138] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-11 (0.56 seconds). [2025-11-04T19:35:03.930275] [INFO ] [REMOTE ] IIM: Full fetch complete (24466 records). [2025-11-04T19:35:03.930350] [DEBUG] [VALIDATOR ] IIM: Validating primary data. [2025-11-04T19:35:04.175238] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-03. [2025-11-04T19:35:04.314581] [DEBUG] [VALIDATOR ] IIM: All expected trading days are present. [2025-11-04T19:35:04.314681] [DEBUG] [VALIDATOR ] IIM: Primary data validation passed. [2025-11-04T19:35:04.315198] [INFO ] [MAIN ] IIM: Fetched primary data is valid. [2025-11-04T19:35:04.315238] [DEBUG] [LOCAL ] IIM: Saving primary data. [2025-11-04T19:35:04.315932] [DEBUG] [LOCAL ] IIM: Trimmed 24466 records to 15430 (keeping 200 days of data) [2025-11-04T19:35:04.409540] [DEBUG] [LOCAL ] IIM: Primary data saved (15430 records). [2025-11-04T19:35:04.409706] [INFO ] [MAIN ] IIM: Full history saved. [2025-11-04T19:35:04.409798] [INFO ] [MAIN ] Update Progress: 1103/2367 symbols completed. IIM took 25.2s. Estimated remaining: 2h 57m 39s. Rate: 7.1 symbols/min [2025-11-04T19:35:04.580841] [INFO ] [MAIN ] INFA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:35:04.621405] [DEBUG] [LOCAL ] INFA: Loaded 48017 existing records from primary storage. [2025-11-04T19:35:04.621741] [INFO ] [MAIN ] INFA: Loaded 48017 existing records. [2025-11-04T19:35:04.798946] [WARN ] [MAIN ] INFA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:35:04.799178] [INFO ] [MAIN ] INFA: Wiping existing primary data before full refresh. [2025-11-04T19:35:04.799227] [INFO ] [LOCAL ] INFA: Starting deletion of primary data. [2025-11-04T19:35:04.800088] [DEBUG] [LOCAL ] INFA: Deleted primary data file. [2025-11-04T19:35:04.800147] [INFO ] [MAIN ] INFA: Starting full primary refresh. [2025-11-04T19:35:04.972165] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-11. [2025-11-04T19:35:05.169388] [INFO ] [REMOTE ] INFA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:35:05.356100] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-11-04T19:35:05.767051] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-01. [2025-11-04T19:35:05.888346] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-11 (0.92 seconds). [2025-11-04T19:35:05.889833] [INFO ] [REMOTE ] IHS: Full fetch complete (51745 records). [2025-11-04T19:35:05.889940] [DEBUG] [VALIDATOR ] IHS: Validating primary data. [2025-11-04T19:35:06.304224] [DEBUG] [VALIDATOR ] IHS: All expected trading days are present. [2025-11-04T19:35:06.304367] [DEBUG] [VALIDATOR ] IHS: Primary data validation passed. [2025-11-04T19:35:06.305321] [INFO ] [MAIN ] IHS: Fetched primary data is valid. [2025-11-04T19:35:06.305419] [DEBUG] [LOCAL ] IHS: Saving primary data. [2025-11-04T19:35:06.306794] [DEBUG] [LOCAL ] IHS: Trimmed 51745 records to 36708 (keeping 200 days of data) [2025-11-04T19:35:06.511498] [DEBUG] [LOCAL ] IHS: Primary data saved (36708 records). [2025-11-04T19:35:06.511667] [INFO ] [MAIN ] IHS: Full history saved. [2025-11-04T19:35:06.511767] [INFO ] [MAIN ] Update Progress: 1104/2367 symbols completed. IHS took 1m 1s. Estimated remaining: 2h 57m 23s. Rate: 7.1 symbols/min [2025-11-04T19:35:06.576969] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-04. [2025-11-04T19:35:06.694772] [INFO ] [MAIN ] INFY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:35:06.742122] [DEBUG] [LOCAL ] INFY: Loaded 53594 existing records from primary storage. [2025-11-04T19:35:06.742438] [INFO ] [MAIN ] INFY: Loaded 53594 existing records. [2025-11-04T19:35:06.917549] [WARN ] [MAIN ] INFY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:35:06.917812] [INFO ] [MAIN ] INFY: Wiping existing primary data before full refresh. [2025-11-04T19:35:06.917862] [INFO ] [LOCAL ] INFY: Starting deletion of primary data. [2025-11-04T19:35:06.918850] [DEBUG] [LOCAL ] INFY: Deleted primary data file. [2025-11-04T19:35:06.918906] [INFO ] [MAIN ] INFY: Starting full primary refresh. [2025-11-04T19:35:07.274248] [INFO ] [REMOTE ] INFY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:35:07.299226] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-11-04T19:35:07.367033] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-01. [2025-11-04T19:35:07.792291] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T19:35:08.167044] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-05. [2025-11-04T19:35:08.967070] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-02. [2025-11-04T19:35:09.060951] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-11-04T19:35:09.191673] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T19:35:09.767073] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-06. [2025-11-04T19:35:10.308584] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-11-04T19:35:10.567037] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-03. [2025-11-04T19:35:10.732088] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-11-04T19:35:11.367043] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-04. [2025-11-04T19:35:11.945791] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T19:35:12.167048] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-04. [2025-11-04T19:35:12.575360] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T19:35:12.967016] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-07. [2025-11-04T19:35:13.386467] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T19:35:13.767028] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-05. [2025-11-04T19:35:13.994777] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T19:35:14.567078] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-08. [2025-11-04T19:35:14.802345] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-11-04T19:35:15.367076] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-02. [2025-11-04T19:35:15.721064] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-11-04T19:35:16.167022] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-05. [2025-11-04T19:35:16.587851] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T19:35:16.967063] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-06. [2025-11-04T19:35:17.490161] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T19:35:17.767035] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-03. [2025-11-04T19:35:17.999507] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T19:35:18.567077] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-06. [2025-11-04T19:35:19.298368] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T19:35:19.366998] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-04. [2025-11-04T19:35:19.805704] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T19:35:20.167110] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-09. [2025-11-04T19:35:20.705634] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T19:35:20.967081] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-05. [2025-11-04T19:35:21.086608] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T19:35:21.767051] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-07. [2025-11-04T19:35:22.459781] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-11-04T19:35:22.567012] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-06. [2025-11-04T19:35:22.894373] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T19:35:23.367044] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-08. [2025-11-04T19:35:24.032408] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T19:35:24.167082] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-07. [2025-11-04T19:35:24.666137] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T19:35:24.967157] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-09. [2025-11-04T19:35:25.462030] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-11-04T19:35:25.767034] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-08. [2025-11-04T19:35:26.102631] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T19:35:26.567061] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-10. [2025-11-04T19:35:26.764527] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T19:35:27.367008] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-07. [2025-11-04T19:35:27.667595] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T19:35:28.167070] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-09. [2025-11-04T19:35:28.770863] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T19:35:28.967063] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-10. [2025-11-04T19:35:29.357727] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T19:35:29.767084] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-11. [2025-11-04T19:35:30.135436] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-11-04T19:35:30.567051] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-08. [2025-11-04T19:35:30.711206] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T19:35:30.713054] [INFO ] [REMOTE ] INFA: Full fetch complete (70511 records). [2025-11-04T19:35:30.713150] [DEBUG] [VALIDATOR ] INFA: Validating primary data. [2025-11-04T19:35:31.091716] [DEBUG] [VALIDATOR ] INFA: All expected trading days are present. [2025-11-04T19:35:31.091795] [DEBUG] [VALIDATOR ] INFA: Primary data validation passed. [2025-11-04T19:35:31.092677] [INFO ] [MAIN ] INFA: Fetched primary data is valid. [2025-11-04T19:35:31.092738] [DEBUG] [LOCAL ] INFA: Saving primary data. [2025-11-04T19:35:31.093999] [DEBUG] [LOCAL ] INFA: Trimmed 70511 records to 48004 (keeping 200 days of data) [2025-11-04T19:35:31.361253] [DEBUG] [LOCAL ] INFA: Primary data saved (48004 records). [2025-11-04T19:35:31.361381] [INFO ] [MAIN ] INFA: Full history saved. [2025-11-04T19:35:31.361457] [INFO ] [MAIN ] Update Progress: 1105/2367 symbols completed. INFA took 27.0s. Estimated remaining: 2h 57m 33s. Rate: 7.1 symbols/min [2025-11-04T19:35:31.372693] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-10. [2025-11-04T19:35:31.530776] [INFO ] [MAIN ] ING: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:35:31.574399] [DEBUG] [LOCAL ] ING: Loaded 50598 existing records from primary storage. [2025-11-04T19:35:31.574714] [INFO ] [MAIN ] ING: Loaded 50598 existing records. [2025-11-04T19:35:31.752078] [WARN ] [MAIN ] ING: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:35:31.752415] [INFO ] [MAIN ] ING: Wiping existing primary data before full refresh. [2025-11-04T19:35:31.752464] [INFO ] [LOCAL ] ING: Starting deletion of primary data. [2025-11-04T19:35:31.753256] [DEBUG] [LOCAL ] ING: Deleted primary data file. [2025-11-04T19:35:31.753312] [INFO ] [MAIN ] ING: Starting full primary refresh. [2025-11-04T19:35:32.114207] [INFO ] [REMOTE ] ING: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:35:32.147534] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-11-04T19:35:32.167034] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-09. [2025-11-04T19:35:32.710672] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-11-04T19:35:32.967075] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-01. [2025-11-04T19:35:33.559571] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T19:35:33.767046] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-11. [2025-11-04T19:35:34.567075] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-10. [2025-11-04T19:35:34.613765] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-11-04T19:35:34.666038] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T19:35:34.667680] [INFO ] [REMOTE ] IIPR: Full fetch complete (43365 records). [2025-11-04T19:35:34.667823] [DEBUG] [VALIDATOR ] IIPR: Validating primary data. [2025-11-04T19:35:35.065555] [DEBUG] [VALIDATOR ] IIPR: All expected trading days are present. [2025-11-04T19:35:35.065680] [DEBUG] [VALIDATOR ] IIPR: Primary data validation passed. [2025-11-04T19:35:35.066404] [INFO ] [MAIN ] IIPR: Fetched primary data is valid. [2025-11-04T19:35:35.066449] [DEBUG] [LOCAL ] IIPR: Saving primary data. [2025-11-04T19:35:35.067530] [DEBUG] [LOCAL ] IIPR: Trimmed 43365 records to 28532 (keeping 200 days of data) [2025-11-04T19:35:35.246246] [DEBUG] [LOCAL ] IIPR: Primary data saved (28532 records). [2025-11-04T19:35:35.246423] [INFO ] [MAIN ] IIPR: Full history saved. [2025-11-04T19:35:35.246535] [INFO ] [MAIN ] Update Progress: 1106/2367 symbols completed. IIPR took 41.3s. Estimated remaining: 2h 57m 20s. Rate: 7.1 symbols/min [2025-11-04T19:35:35.372160] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-11. [2025-11-04T19:35:35.422448] [INFO ] [MAIN ] INGM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:35:35.449148] [DEBUG] [LOCAL ] INGM: Loaded 27565 existing records from primary storage. [2025-11-04T19:35:35.449437] [INFO ] [MAIN ] INGM: Loaded 27565 existing records. [2025-11-04T19:35:35.642791] [WARN ] [MAIN ] INGM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:35:35.642992] [INFO ] [MAIN ] INGM: Wiping existing primary data before full refresh. [2025-11-04T19:35:35.643039] [INFO ] [LOCAL ] INGM: Starting deletion of primary data. [2025-11-04T19:35:35.643570] [DEBUG] [LOCAL ] INGM: Deleted primary data file. [2025-11-04T19:35:35.643628] [INFO ] [MAIN ] INGM: Starting full primary refresh. [2025-11-04T19:35:36.008000] [INFO ] [REMOTE ] INGM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:35:36.075451] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T19:35:36.167013] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-11. [2025-11-04T19:35:36.477676] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T19:35:36.479488] [INFO ] [REMOTE ] IMAX: Full fetch complete (61739 records). [2025-11-04T19:35:36.479595] [DEBUG] [VALIDATOR ] IMAX: Validating primary data. [2025-11-04T19:35:36.878147] [DEBUG] [VALIDATOR ] IMAX: All expected trading days are present. [2025-11-04T19:35:36.878227] [DEBUG] [VALIDATOR ] IMAX: Primary data validation passed. [2025-11-04T19:35:36.879044] [INFO ] [MAIN ] IMAX: Fetched primary data is valid. [2025-11-04T19:35:36.879105] [DEBUG] [LOCAL ] IMAX: Saving primary data. [2025-11-04T19:35:36.880275] [DEBUG] [LOCAL ] IMAX: Trimmed 61739 records to 43309 (keeping 200 days of data) [2025-11-04T19:35:36.967080] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-02. [2025-11-04T19:35:37.124531] [DEBUG] [LOCAL ] IMAX: Primary data saved (43309 records). [2025-11-04T19:35:37.124669] [INFO ] [MAIN ] IMAX: Full history saved. [2025-11-04T19:35:37.124763] [INFO ] [MAIN ] Update Progress: 1107/2367 symbols completed. IMAX took 40.9s. Estimated remaining: 2h 57m 4s. Rate: 7.1 symbols/min [2025-11-04T19:35:37.289631] [INFO ] [MAIN ] INGR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:35:37.320309] [DEBUG] [LOCAL ] INGR: Loaded 33874 existing records from primary storage. [2025-11-04T19:35:37.320571] [INFO ] [MAIN ] INGR: Loaded 33874 existing records. [2025-11-04T19:35:37.497471] [WARN ] [MAIN ] INGR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:35:37.503155] [INFO ] [MAIN ] INGR: Wiping existing primary data before full refresh. [2025-11-04T19:35:37.502998] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-11 (1.34 seconds). [2025-11-04T19:35:37.503320] [INFO ] [LOCAL ] INGR: Starting deletion of primary data. [2025-11-04T19:35:37.505486] [INFO ] [REMOTE ] INFY: Full fetch complete (77750 records). [2025-11-04T19:35:37.505604] [DEBUG] [VALIDATOR ] INFY: Validating primary data. [2025-11-04T19:35:37.505783] [DEBUG] [LOCAL ] INGR: Deleted primary data file. [2025-11-04T19:35:37.505903] [INFO ] [MAIN ] INGR: Starting full primary refresh. [2025-11-04T19:35:37.774392] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-01. [2025-11-04T19:35:38.327712] [INFO ] [REMOTE ] INGR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:35:38.351368] [DEBUG] [VALIDATOR ] INFY: All expected trading days are present. [2025-11-04T19:35:38.351558] [DEBUG] [VALIDATOR ] INFY: Primary data validation passed. [2025-11-04T19:35:38.352879] [INFO ] [MAIN ] INFY: Fetched primary data is valid. [2025-11-04T19:35:38.352986] [DEBUG] [LOCAL ] INFY: Saving primary data. [2025-11-04T19:35:38.355403] [DEBUG] [LOCAL ] INFY: Trimmed 77750 records to 53593 (keeping 200 days of data) [2025-11-04T19:35:38.523597] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-11-04T19:35:38.572090] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-01. [2025-11-04T19:35:38.927030] [DEBUG] [LOCAL ] INFY: Primary data saved (53593 records). [2025-11-04T19:35:38.927319] [INFO ] [MAIN ] INFY: Full history saved. [2025-11-04T19:35:38.927446] [INFO ] [MAIN ] Update Progress: 1108/2367 symbols completed. INFY took 32.4s. Estimated remaining: 2h 56m 48s. Rate: 7.1 symbols/min [2025-11-04T19:35:39.161549] [INFO ] [MAIN ] INN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:35:39.158135] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T19:35:39.206986] [DEBUG] [LOCAL ] INN: Loaded 40812 existing records from primary storage. [2025-11-04T19:35:39.207317] [INFO ] [MAIN ] INN: Loaded 40812 existing records. [2025-11-04T19:35:39.368675] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-03. [2025-11-04T19:35:39.397341] [WARN ] [MAIN ] INN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:35:39.398301] [INFO ] [MAIN ] INN: Wiping existing primary data before full refresh. [2025-11-04T19:35:39.398353] [INFO ] [LOCAL ] INN: Starting deletion of primary data. [2025-11-04T19:35:39.398996] [DEBUG] [LOCAL ] INN: Deleted primary data file. [2025-11-04T19:35:39.399088] [INFO ] [MAIN ] INN: Starting full primary refresh. [2025-11-04T19:35:39.768256] [INFO ] [REMOTE ] INN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:35:39.957507] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-11-04T19:35:40.167037] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-01. [2025-11-04T19:35:40.886103] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T19:35:40.967032] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-02. [2025-11-04T19:35:41.067285] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T19:35:41.728026] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T19:35:41.767039] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-03. [2025-11-04T19:35:42.567076] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-04. [2025-11-04T19:35:42.901931] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-11-04T19:35:43.367082] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-04. [2025-11-04T19:35:43.908038] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T19:35:44.167087] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-02. [2025-11-04T19:35:44.282143] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T19:35:44.967062] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-05. [2025-11-04T19:35:45.077356] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-11-04T19:35:45.772343] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-02. [2025-11-04T19:35:45.785570] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-11-04T19:35:46.567027] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-06. [2025-11-04T19:35:46.879130] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T19:35:47.367013] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-03. [2025-11-04T19:35:47.434309] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T19:35:48.167029] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-07. [2025-11-04T19:35:48.518195] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T19:35:48.967016] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-04. [2025-11-04T19:35:49.114509] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-11-04T19:35:49.767073] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-03. [2025-11-04T19:35:49.869970] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-11-04T19:35:50.567029] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-05. [2025-11-04T19:35:50.721558] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-11-04T19:35:51.367052] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-05. [2025-11-04T19:35:51.637628] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T19:35:52.167041] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-06. [2025-11-04T19:35:52.633940] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-11-04T19:35:52.967070] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-08. [2025-11-04T19:35:53.024449] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T19:35:53.767015] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-06. [2025-11-04T19:35:53.805780] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-11-04T19:35:54.567010] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-09. [2025-11-04T19:35:54.907161] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-11-04T19:35:55.367022] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-07. [2025-11-04T19:35:55.444809] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T19:35:56.167059] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-07. [2025-11-04T19:35:56.750396] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-11-04T19:35:56.967040] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-08. [2025-11-04T19:35:57.237753] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T19:35:57.767080] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-08. [2025-11-04T19:35:58.230669] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-11-04T19:35:58.567090] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-10. [2025-11-04T19:35:58.798618] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-11-04T19:35:59.367030] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-09. [2025-11-04T19:35:59.505051] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T19:36:00.167145] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-11. [2025-11-04T19:36:00.411326] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-11-04T19:36:00.967051] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-10. [2025-11-04T19:36:01.119405] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-11 (0.95 seconds). [2025-11-04T19:36:01.120919] [INFO ] [REMOTE ] INGM: Full fetch complete (41998 records). [2025-11-04T19:36:01.120989] [DEBUG] [VALIDATOR ] INGM: Validating primary data. [2025-11-04T19:36:01.534078] [DEBUG] [VALIDATOR ] INGM: All expected trading days are present. [2025-11-04T19:36:01.534173] [DEBUG] [VALIDATOR ] INGM: Primary data validation passed. [2025-11-04T19:36:01.534978] [INFO ] [MAIN ] INGM: Fetched primary data is valid. [2025-11-04T19:36:01.535065] [DEBUG] [LOCAL ] INGM: Saving primary data. [2025-11-04T19:36:01.536228] [DEBUG] [LOCAL ] INGM: Trimmed 41998 records to 27556 (keeping 200 days of data) [2025-11-04T19:36:01.710914] [DEBUG] [LOCAL ] INGM: Primary data saved (27556 records). [2025-11-04T19:36:01.711053] [INFO ] [MAIN ] INGM: Full history saved. [2025-11-04T19:36:01.711159] [INFO ] [MAIN ] Update Progress: 1109/2367 symbols completed. INGM took 26.5s. Estimated remaining: 2h 56m 56s. Rate: 7.1 symbols/min [2025-11-04T19:36:01.777530] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-09. [2025-11-04T19:36:01.930282] [INFO ] [MAIN ] INR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:36:01.930617] [DEBUG] [LOCAL ] INR: No existing data file found in primary storage. [2025-11-04T19:36:01.930710] [WARN ] [MAIN ] INR: No existing data found, performing full refresh instead. [2025-11-04T19:36:01.930870] [INFO ] [MAIN ] INR: Wiping existing primary data before full refresh. [2025-11-04T19:36:01.930927] [INFO ] [LOCAL ] INR: Starting deletion of primary data. [2025-11-04T19:36:01.930979] [DEBUG] [LOCAL ] INR: No primary data file found to delete. [2025-11-04T19:36:01.931043] [INFO ] [MAIN ] INR: Starting full primary refresh. [2025-11-04T19:36:02.371017] [INFO ] [REMOTE ] INR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:36:02.372581] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T19:36:02.567023] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-11. [2025-11-04T19:36:03.159469] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T19:36:03.367038] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-04. [2025-11-04T19:36:03.654809] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T19:36:03.657227] [INFO ] [REMOTE ] INGR: Full fetch complete (50806 records). [2025-11-04T19:36:03.657329] [DEBUG] [VALIDATOR ] INGR: Validating primary data. [2025-11-04T19:36:04.040485] [DEBUG] [VALIDATOR ] INGR: All expected trading days are present. [2025-11-04T19:36:04.040559] [DEBUG] [VALIDATOR ] INGR: Primary data validation passed. [2025-11-04T19:36:04.041279] [INFO ] [MAIN ] INGR: Fetched primary data is valid. [2025-11-04T19:36:04.041339] [DEBUG] [LOCAL ] INGR: Saving primary data. [2025-11-04T19:36:04.042286] [DEBUG] [LOCAL ] INGR: Trimmed 50806 records to 33953 (keeping 200 days of data) [2025-11-04T19:36:04.196840] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-10. [2025-11-04T19:36:04.246744] [DEBUG] [LOCAL ] INGR: Primary data saved (33953 records). [2025-11-04T19:36:04.246871] [INFO ] [MAIN ] INGR: Full history saved. [2025-11-04T19:36:04.246952] [INFO ] [MAIN ] Update Progress: 1110/2367 symbols completed. INGR took 27.1s. Estimated remaining: 2h 56m 41s. Rate: 7.1 symbols/min [2025-11-04T19:36:04.413204] [INFO ] [MAIN ] INSP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:36:04.446734] [DEBUG] [LOCAL ] INSP: Loaded 39136 existing records from primary storage. [2025-11-04T19:36:04.446998] [INFO ] [MAIN ] INSP: Loaded 39136 existing records. [2025-11-04T19:36:04.624083] [WARN ] [MAIN ] INSP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:36:04.624364] [INFO ] [MAIN ] INSP: Wiping existing primary data before full refresh. [2025-11-04T19:36:04.624469] [INFO ] [LOCAL ] INSP: Starting deletion of primary data. [2025-11-04T19:36:04.633315] [DEBUG] [LOCAL ] INSP: Deleted primary data file. [2025-11-04T19:36:04.647410] [INFO ] [MAIN ] INSP: Starting full primary refresh. [2025-11-04T19:36:04.723104] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:36:04.974597] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-05. [2025-11-04T19:36:05.018017] [INFO ] [REMOTE ] INSP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:36:05.767043] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-01. [2025-11-04T19:36:06.034636] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-11-04T19:36:06.120168] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T19:36:06.567087] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-06. [2025-11-04T19:36:06.999881] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T19:36:07.367030] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-02. [2025-11-04T19:36:07.712779] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T19:36:08.167047] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-01. [2025-11-04T19:36:08.508183] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-11-04T19:36:08.528076] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T19:36:08.967059] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-11. [2025-11-04T19:36:09.767032] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-02. [2025-11-04T19:36:10.318290] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-11 (1.35 seconds). [2025-11-04T19:36:10.320228] [INFO ] [REMOTE ] ING: Full fetch complete (73785 records). [2025-11-04T19:36:10.320317] [DEBUG] [VALIDATOR ] ING: Validating primary data. [2025-11-04T19:36:10.576747] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-03. [2025-11-04T19:36:10.729991] [DEBUG] [VALIDATOR ] ING: All expected trading days are present. [2025-11-04T19:36:10.730146] [DEBUG] [VALIDATOR ] ING: Primary data validation passed. [2025-11-04T19:36:10.731142] [INFO ] [MAIN ] ING: Fetched primary data is valid. [2025-11-04T19:36:10.731262] [DEBUG] [LOCAL ] ING: Saving primary data. [2025-11-04T19:36:10.732901] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-11-04T19:36:10.740749] [DEBUG] [LOCAL ] ING: Trimmed 73785 records to 50581 (keeping 200 days of data) [2025-11-04T19:36:11.056565] [DEBUG] [LOCAL ] ING: Primary data saved (50581 records). [2025-11-04T19:36:11.056742] [INFO ] [MAIN ] ING: Full history saved. [2025-11-04T19:36:11.056829] [INFO ] [MAIN ] Update Progress: 1111/2367 symbols completed. ING took 39.7s. Estimated remaining: 2h 56m 30s. Rate: 7.1 symbols/min [2025-11-04T19:36:11.226675] [INFO ] [MAIN ] INSW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:36:11.260384] [DEBUG] [LOCAL ] INSW: Loaded 38405 existing records from primary storage. [2025-11-04T19:36:11.260732] [INFO ] [MAIN ] INSW: Loaded 38405 existing records. [2025-11-04T19:36:11.372121] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-03. [2025-11-04T19:36:11.448022] [WARN ] [MAIN ] INSW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:36:11.448256] [INFO ] [MAIN ] INSW: Wiping existing primary data before full refresh. [2025-11-04T19:36:11.448311] [INFO ] [LOCAL ] INSW: Starting deletion of primary data. [2025-11-04T19:36:11.448998] [DEBUG] [LOCAL ] INSW: Deleted primary data file. [2025-11-04T19:36:11.449057] [INFO ] [MAIN ] INSW: Starting full primary refresh. [2025-11-04T19:36:11.823695] [INFO ] [REMOTE ] INSW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:36:11.845912] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-11-04T19:36:12.167037] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-04. [2025-11-04T19:36:12.479831] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T19:36:12.967123] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-07. [2025-11-04T19:36:13.322767] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-11-04T19:36:13.767094] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-01. [2025-11-04T19:36:14.118421] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-11-04T19:36:14.567099] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-05. [2025-11-04T19:36:15.046677] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T19:36:15.367054] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-04. [2025-11-04T19:36:15.616547] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T19:36:16.152697] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-11-04T19:36:16.166993] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-05. [2025-11-04T19:36:16.967048] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-06. [2025-11-04T19:36:17.020321] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-11-04T19:36:17.759363] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T19:36:17.766996] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-02. [2025-11-04T19:36:18.567031] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-07. [2025-11-04T19:36:18.745850] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-11-04T19:36:19.367066] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-08. [2025-11-04T19:36:19.538294] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-11-04T19:36:20.167043] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-03. [2025-11-04T19:36:20.501115] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-11-04T19:36:20.967084] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-06. [2025-11-04T19:36:21.538347] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T19:36:21.767025] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-04. [2025-11-04T19:36:21.815983] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T19:36:22.567129] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-07. [2025-11-04T19:36:22.955256] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T19:36:23.367056] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-08. [2025-11-04T19:36:23.393619] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T19:36:24.167108] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-09. [2025-11-04T19:36:24.633357] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T19:36:24.967047] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-05. [2025-11-04T19:36:25.129429] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T19:36:25.767082] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-09. [2025-11-04T19:36:26.091558] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-11-04T19:36:26.567068] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-10. [2025-11-04T19:36:26.854735] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-11-04T19:36:27.367052] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-08. [2025-11-04T19:36:27.673186] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-11-04T19:36:28.167011] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-06. [2025-11-04T19:36:28.302491] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-11-04T19:36:28.967090] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-09. [2025-11-04T19:36:29.203176] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T19:36:29.767045] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-11. [2025-11-04T19:36:29.908147] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-11-04T19:36:30.567046] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-07. [2025-11-04T19:36:30.576275] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-11 (0.81 seconds). [2025-11-04T19:36:30.578405] [INFO ] [REMOTE ] INN: Full fetch complete (57912 records). [2025-11-04T19:36:30.578543] [DEBUG] [VALIDATOR ] INN: Validating primary data. [2025-11-04T19:36:30.989624] [DEBUG] [VALIDATOR ] INN: All expected trading days are present. [2025-11-04T19:36:30.989733] [DEBUG] [VALIDATOR ] INN: Primary data validation passed. [2025-11-04T19:36:30.990589] [INFO ] [MAIN ] INN: Fetched primary data is valid. [2025-11-04T19:36:30.990688] [DEBUG] [LOCAL ] INN: Saving primary data. [2025-11-04T19:36:30.991848] [DEBUG] [LOCAL ] INN: Trimmed 57912 records to 40842 (keeping 200 days of data) [2025-11-04T19:36:31.220665] [DEBUG] [LOCAL ] INN: Primary data saved (40842 records). [2025-11-04T19:36:31.220795] [INFO ] [MAIN ] INN: Full history saved. [2025-11-04T19:36:31.220870] [INFO ] [MAIN ] Update Progress: 1112/2367 symbols completed. INN took 52.3s. Estimated remaining: 2h 56m 35s. Rate: 7.1 symbols/min [2025-11-04T19:36:31.371890] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-10. [2025-11-04T19:36:31.396495] [INFO ] [MAIN ] INVH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:36:31.447522] [DEBUG] [LOCAL ] INVH: Loaded 53189 existing records from primary storage. [2025-11-04T19:36:31.447888] [INFO ] [MAIN ] INVH: Loaded 53189 existing records. [2025-11-04T19:36:31.628566] [WARN ] [MAIN ] INVH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:36:31.628843] [INFO ] [MAIN ] INVH: Wiping existing primary data before full refresh. [2025-11-04T19:36:31.628968] [INFO ] [LOCAL ] INVH: Starting deletion of primary data. [2025-11-04T19:36:31.629830] [DEBUG] [LOCAL ] INVH: Deleted primary data file. [2025-11-04T19:36:31.629886] [INFO ] [MAIN ] INVH: Starting full primary refresh. [2025-11-04T19:36:32.013474] [INFO ] [REMOTE ] INVH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:36:32.014730] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T19:36:32.167011] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-01. [2025-11-04T19:36:32.721224] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T19:36:32.967073] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-11. [2025-11-04T19:36:33.478757] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T19:36:33.767051] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-02. [2025-11-04T19:36:34.098707] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:36:34.100754] [INFO ] [REMOTE ] INSP: Full fetch complete (54302 records). [2025-11-04T19:36:34.100906] [DEBUG] [VALIDATOR ] INSP: Validating primary data. [2025-11-04T19:36:34.487111] [DEBUG] [VALIDATOR ] INSP: All expected trading days are present. [2025-11-04T19:36:34.487186] [DEBUG] [VALIDATOR ] INSP: Primary data validation passed. [2025-11-04T19:36:34.487942] [INFO ] [MAIN ] INSP: Fetched primary data is valid. [2025-11-04T19:36:34.488002] [DEBUG] [LOCAL ] INSP: Saving primary data. [2025-11-04T19:36:34.489064] [DEBUG] [LOCAL ] INSP: Trimmed 54302 records to 39204 (keeping 200 days of data) [2025-11-04T19:36:34.572479] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-08. [2025-11-04T19:36:34.718902] [DEBUG] [LOCAL ] INSP: Primary data saved (39204 records). [2025-11-04T19:36:34.719042] [INFO ] [MAIN ] INSP: Full history saved. [2025-11-04T19:36:34.719122] [INFO ] [MAIN ] Update Progress: 1113/2367 symbols completed. INSP took 30.5s. Estimated remaining: 2h 56m 21s. Rate: 7.1 symbols/min [2025-11-04T19:36:34.895172] [INFO ] [MAIN ] INVX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:36:34.921244] [DEBUG] [LOCAL ] INVX: Loaded 28302 existing records from primary storage. [2025-11-04T19:36:34.921502] [INFO ] [MAIN ] INVX: Loaded 28302 existing records. [2025-11-04T19:36:35.112566] [WARN ] [MAIN ] INVX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:36:35.113095] [INFO ] [MAIN ] INVX: Wiping existing primary data before full refresh. [2025-11-04T19:36:35.113226] [INFO ] [LOCAL ] INVX: Starting deletion of primary data. [2025-11-04T19:36:35.113896] [DEBUG] [LOCAL ] INVX: Deleted primary data file. [2025-11-04T19:36:35.113967] [INFO ] [MAIN ] INVX: Starting full primary refresh. [2025-11-04T19:36:35.289925] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-11-04T19:36:35.372236] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-03. [2025-11-04T19:36:35.520950] [INFO ] [REMOTE ] INVX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:36:35.738878] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T19:36:36.167045] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-09. [2025-11-04T19:36:36.627594] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-11-04T19:36:36.967034] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-10. [2025-11-04T19:36:37.347174] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-11-04T19:36:37.767105] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-10. [2025-11-04T19:36:37.850050] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-11-04T19:36:38.567068] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-11. [2025-11-04T19:36:38.938525] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-11-04T19:36:39.367006] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-04. [2025-11-04T19:36:39.393916] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T19:36:39.395826] [INFO ] [REMOTE ] INR: Full fetch complete (39172 records). [2025-11-04T19:36:39.395939] [DEBUG] [VALIDATOR ] INR: Validating primary data. [2025-11-04T19:36:39.587856] [WARN ] [VALIDATOR ] INR: Validation failed - insufficient historical data. Have 192 unique days, need 200 days [2025-11-04T19:36:39.588397] [FATAL] [MAIN ] INR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:36:39.588474] [WARN ] [MAIN ] INR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:36:39.588524] [INFO ] [LOCAL ] INR: Starting deletion of primary data. [2025-11-04T19:36:39.588587] [DEBUG] [LOCAL ] INR: No primary data file found to delete. [2025-11-04T19:36:39.588623] [INFO ] [LOCAL ] INR: Starting deletion of derivative data. [2025-11-04T19:36:39.588700] [DEBUG] [LOCAL ] INR: No derivative data file found to delete. [2025-11-04T19:36:39.588781] [INFO ] [MAIN ] Update Progress: 1114/2367 symbols completed. INR took 37.9s. Estimated remaining: 2h 56m 9s. Rate: 7.1 symbols/min [2025-11-04T19:36:39.753164] [INFO ] [MAIN ] IONQ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:36:39.799932] [DEBUG] [LOCAL ] IONQ: Loaded 53788 existing records from primary storage. [2025-11-04T19:36:39.800232] [INFO ] [MAIN ] IONQ: Loaded 53788 existing records. [2025-11-04T19:36:39.997813] [WARN ] [MAIN ] IONQ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:36:39.998257] [INFO ] [MAIN ] IONQ: Wiping existing primary data before full refresh. [2025-11-04T19:36:39.998310] [INFO ] [LOCAL ] IONQ: Starting deletion of primary data. [2025-11-04T19:36:39.998978] [DEBUG] [LOCAL ] IONQ: Deleted primary data file. [2025-11-04T19:36:39.999035] [INFO ] [MAIN ] IONQ: Starting full primary refresh. [2025-11-04T19:36:40.171281] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-11. [2025-11-04T19:36:40.356282] [INFO ] [REMOTE ] IONQ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:36:40.890981] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-11-04T19:36:40.967025] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-05. [2025-11-04T19:36:41.206253] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T19:36:41.207953] [INFO ] [REMOTE ] INSW: Full fetch complete (58557 records). [2025-11-04T19:36:41.208051] [DEBUG] [VALIDATOR ] INSW: Validating primary data. [2025-11-04T19:36:41.603180] [DEBUG] [VALIDATOR ] INSW: All expected trading days are present. [2025-11-04T19:36:41.603269] [DEBUG] [VALIDATOR ] INSW: Primary data validation passed. [2025-11-04T19:36:41.604077] [INFO ] [MAIN ] INSW: Fetched primary data is valid. [2025-11-04T19:36:41.604141] [DEBUG] [LOCAL ] INSW: Saving primary data. [2025-11-04T19:36:41.605370] [DEBUG] [LOCAL ] INSW: Trimmed 58557 records to 38376 (keeping 200 days of data) [2025-11-04T19:36:41.775060] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-01. [2025-11-04T19:36:41.843780] [DEBUG] [LOCAL ] INSW: Primary data saved (38376 records). [2025-11-04T19:36:41.843942] [INFO ] [MAIN ] INSW: Full history saved. [2025-11-04T19:36:41.844026] [INFO ] [MAIN ] Update Progress: 1115/2367 symbols completed. INSW took 30.8s. Estimated remaining: 2h 55m 53s. Rate: 7.1 symbols/min [2025-11-04T19:36:42.007346] [INFO ] [MAIN ] IOT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:36:42.055294] [DEBUG] [LOCAL ] IOT: Loaded 53444 existing records from primary storage. [2025-11-04T19:36:42.055583] [INFO ] [MAIN ] IOT: Loaded 53444 existing records. [2025-11-04T19:36:42.227733] [WARN ] [MAIN ] IOT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:36:42.227927] [INFO ] [MAIN ] IOT: Wiping existing primary data before full refresh. [2025-11-04T19:36:42.227983] [INFO ] [LOCAL ] IOT: Starting deletion of primary data. [2025-11-04T19:36:42.228400] [DEBUG] [LOCAL ] IOT: Deleted primary data file. [2025-11-04T19:36:42.228457] [INFO ] [MAIN ] IOT: Starting full primary refresh. [2025-11-04T19:36:42.575452] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-01. [2025-11-04T19:36:42.580005] [INFO ] [REMOTE ] IOT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:36:42.611911] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-11-04T19:36:43.367085] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-06. [2025-11-04T19:36:43.785347] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T19:36:43.927334] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-01 (2.15 seconds). [2025-11-04T19:36:44.167034] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-01. [2025-11-04T19:36:44.607352] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T19:36:44.967120] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-02. [2025-11-04T19:36:45.564480] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T19:36:45.767060] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-02. [2025-11-04T19:36:45.954283] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T19:36:46.567035] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-07. [2025-11-04T19:36:47.367055] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-03. [2025-11-04T19:36:47.529954] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-11-04T19:36:47.841242] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T19:36:48.068624] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T19:36:48.167122] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-02. [2025-11-04T19:36:48.967021] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-03. [2025-11-04T19:36:49.375208] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T19:36:49.767043] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-03. [2025-11-04T19:36:50.567100] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-08. [2025-11-04T19:36:51.244476] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-03 (2.28 seconds). [2025-11-04T19:36:51.285268] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-11-04T19:36:51.367059] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-04. [2025-11-04T19:36:51.968574] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-11-04T19:36:52.167039] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-04. [2025-11-04T19:36:52.967039] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-09. [2025-11-04T19:36:53.384179] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-11-04T19:36:53.424842] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T19:36:53.767050] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-05. [2025-11-04T19:36:54.242337] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T19:36:54.567126] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-05. [2025-11-04T19:36:55.367102] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-10. [2025-11-04T19:36:55.811046] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-11-04T19:36:56.057667] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-05 (2.29 seconds). [2025-11-04T19:36:56.167014] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-06. [2025-11-04T19:36:56.794712] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T19:36:56.967027] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-11. [2025-11-04T19:36:57.767040] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-06. [2025-11-04T19:36:58.156822] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-11-04T19:36:58.294877] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-11 (1.33 seconds). [2025-11-04T19:36:58.296903] [INFO ] [REMOTE ] INVH: Full fetch complete (77005 records). [2025-11-04T19:36:58.297026] [DEBUG] [VALIDATOR ] INVH: Validating primary data. [2025-11-04T19:36:58.572139] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-07. [2025-11-04T19:36:58.699953] [DEBUG] [VALIDATOR ] INVH: All expected trading days are present. [2025-11-04T19:36:58.700029] [DEBUG] [VALIDATOR ] INVH: Primary data validation passed. [2025-11-04T19:36:58.700951] [INFO ] [MAIN ] INVH: Fetched primary data is valid. [2025-11-04T19:36:58.701012] [DEBUG] [LOCAL ] INVH: Saving primary data. [2025-11-04T19:36:58.702377] [DEBUG] [LOCAL ] INVH: Trimmed 77005 records to 53190 (keeping 200 days of data) [2025-11-04T19:36:59.025520] [DEBUG] [LOCAL ] INVH: Primary data saved (53190 records). [2025-11-04T19:36:59.025677] [INFO ] [MAIN ] INVH: Full history saved. [2025-11-04T19:36:59.025771] [INFO ] [MAIN ] Update Progress: 1116/2367 symbols completed. INVH took 27.8s. Estimated remaining: 2h 55m 55s. Rate: 7.1 symbols/min [2025-11-04T19:36:59.196119] [INFO ] [MAIN ] IP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:36:59.228404] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-11-04T19:36:59.280172] [DEBUG] [LOCAL ] IP: Loaded 53304 existing records from primary storage. [2025-11-04T19:36:59.280494] [INFO ] [MAIN ] IP: Loaded 53304 existing records. [2025-11-04T19:36:59.372716] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-04. [2025-11-04T19:36:59.461835] [WARN ] [MAIN ] IP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:36:59.462051] [INFO ] [MAIN ] IP: Wiping existing primary data before full refresh. [2025-11-04T19:36:59.462106] [INFO ] [LOCAL ] IP: Starting deletion of primary data. [2025-11-04T19:36:59.462378] [DEBUG] [LOCAL ] IP: Deleted primary data file. [2025-11-04T19:36:59.462436] [INFO ] [MAIN ] IP: Starting full primary refresh. [2025-11-04T19:36:59.817308] [INFO ] [REMOTE ] IP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:37:00.167081] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-01. [2025-11-04T19:37:00.301557] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-11-04T19:37:00.967054] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-07. [2025-11-04T19:37:01.072615] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-07 (2.50 seconds). [2025-11-04T19:37:01.582831] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-11-04T19:37:01.767058] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-05. [2025-11-04T19:37:02.377524] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T19:37:02.567043] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-08. [2025-11-04T19:37:02.686174] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-11-04T19:37:03.367043] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-02. [2025-11-04T19:37:04.167040] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-06. [2025-11-04T19:37:04.568440] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-11-04T19:37:04.839059] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-11-04T19:37:04.967003] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-03. [2025-11-04T19:37:05.141210] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-11-04T19:37:05.767048] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-08. [2025-11-04T19:37:06.501742] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T19:37:06.567004] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-07. [2025-11-04T19:37:07.127742] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-11-04T19:37:07.367015] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-04. [2025-11-04T19:37:07.501972] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-11-04T19:37:08.167056] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-08. [2025-11-04T19:37:08.949407] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-11-04T19:37:08.966992] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-05. [2025-11-04T19:37:09.044099] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-11-04T19:37:09.767035] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-09. [2025-11-04T19:37:10.249815] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T19:37:10.567023] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-09. [2025-11-04T19:37:10.657523] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-11-04T19:37:11.367042] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-09. [2025-11-04T19:37:12.167033] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-06. [2025-11-04T19:37:12.321112] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-11-04T19:37:12.967144] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-10. [2025-11-04T19:37:13.639595] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T19:37:13.718421] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-09 (2.35 seconds). [2025-11-04T19:37:13.767036] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-10. [2025-11-04T19:37:14.567032] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-10. [2025-11-04T19:37:14.625021] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-11-04T19:37:14.663101] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-11-04T19:37:15.367048] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-11. [2025-11-04T19:37:16.167020] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-11. [2025-11-04T19:37:16.663941] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T19:37:16.665979] [INFO ] [REMOTE ] IOT: Full fetch complete (77513 records). [2025-11-04T19:37:16.666068] [DEBUG] [VALIDATOR ] IOT: Validating primary data. [2025-11-04T19:37:16.977745] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-07. [2025-11-04T19:37:16.998466] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T19:37:17.008095] [INFO ] [REMOTE ] INVX: Full fetch complete (42385 records). [2025-11-04T19:37:17.019012] [DEBUG] [VALIDATOR ] INVX: Validating primary data. [2025-11-04T19:37:17.169731] [DEBUG] [VALIDATOR ] IOT: All expected trading days are present. [2025-11-04T19:37:17.187065] [DEBUG] [VALIDATOR ] IOT: Primary data validation passed. [2025-11-04T19:37:17.194929] [INFO ] [MAIN ] IOT: Fetched primary data is valid. [2025-11-04T19:37:17.202802] [DEBUG] [LOCAL ] IOT: Saving primary data. [2025-11-04T19:37:17.322188] [DEBUG] [LOCAL ] IOT: Trimmed 77513 records to 53453 (keeping 200 days of data) [2025-11-04T19:37:17.378299] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-10 (2.81 seconds). [2025-11-04T19:37:17.738966] [DEBUG] [VALIDATOR ] INVX: All expected trading days are present. [2025-11-04T19:37:17.740974] [DEBUG] [VALIDATOR ] INVX: Primary data validation passed. [2025-11-04T19:37:17.751578] [INFO ] [MAIN ] INVX: Fetched primary data is valid. [2025-11-04T19:37:17.752190] [DEBUG] [LOCAL ] INVX: Saving primary data. [2025-11-04T19:37:17.767010] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-11. [2025-11-04T19:37:17.803997] [DEBUG] [LOCAL ] INVX: Trimmed 42385 records to 28290 (keeping 200 days of data) [2025-11-04T19:37:17.906300] [DEBUG] [LOCAL ] IOT: Primary data saved (53453 records). [2025-11-04T19:37:17.906852] [INFO ] [MAIN ] IOT: Full history saved. [2025-11-04T19:37:17.933587] [INFO ] [MAIN ] Update Progress: 1117/2367 symbols completed. IOT took 36.1s. Estimated remaining: 2h 55m 58s. Rate: 7.1 symbols/min [2025-11-04T19:37:18.011707] [DEBUG] [LOCAL ] INVX: Primary data saved (28290 records). [2025-11-04T19:37:18.011838] [INFO ] [MAIN ] INVX: Full history saved. [2025-11-04T19:37:18.011919] [INFO ] [MAIN ] Update Progress: 1118/2367 symbols completed. INVX took 43.3s. Estimated remaining: 2h 55m 40s. Rate: 7.1 symbols/min [2025-11-04T19:37:18.527161] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-11-04T19:37:18.567021] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-08. [2025-11-04T19:37:19.776178] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-11 (2.01 seconds). [2025-11-04T19:37:19.778229] [INFO ] [REMOTE ] IONQ: Full fetch complete (77968 records). [2025-11-04T19:37:19.778353] [DEBUG] [VALIDATOR ] IONQ: Validating primary data. [2025-11-04T19:37:20.176755] [DEBUG] [VALIDATOR ] IONQ: All expected trading days are present. [2025-11-04T19:37:20.176941] [DEBUG] [VALIDATOR ] IONQ: Primary data validation passed. [2025-11-04T19:37:20.177960] [INFO ] [MAIN ] IONQ: Fetched primary data is valid. [2025-11-04T19:37:20.178074] [DEBUG] [LOCAL ] IONQ: Saving primary data. [2025-11-04T19:37:20.206557] [DEBUG] [LOCAL ] IONQ: Trimmed 77968 records to 53788 (keeping 200 days of data) [2025-11-04T19:37:20.208365] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-11-04T19:37:20.220155] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-09. [2025-11-04T19:37:20.535034] [DEBUG] [LOCAL ] IONQ: Primary data saved (53788 records). [2025-11-04T19:37:20.535178] [INFO ] [MAIN ] IONQ: Full history saved. [2025-11-04T19:37:20.535273] [INFO ] [MAIN ] Update Progress: 1119/2367 symbols completed. IONQ took 40.9s. Estimated remaining: 2h 55m 25s. Rate: 7.1 symbols/min [2025-11-04T19:37:21.597562] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T19:37:21.608843] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-10. [2025-11-04T19:37:23.156881] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-11-04T19:37:23.169125] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-11. [2025-11-04T19:37:24.351901] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T19:37:24.354325] [INFO ] [REMOTE ] IP: Full fetch complete (77444 records). [2025-11-04T19:37:24.354489] [DEBUG] [VALIDATOR ] IP: Validating primary data. [2025-11-04T19:37:24.744535] [DEBUG] [VALIDATOR ] IP: All expected trading days are present. [2025-11-04T19:37:24.744636] [DEBUG] [VALIDATOR ] IP: Primary data validation passed. [2025-11-04T19:37:24.745684] [INFO ] [MAIN ] IP: Fetched primary data is valid. [2025-11-04T19:37:24.745746] [DEBUG] [LOCAL ] IP: Saving primary data. [2025-11-04T19:37:24.747243] [DEBUG] [LOCAL ] IP: Trimmed 77444 records to 53307 (keeping 200 days of data) [2025-11-04T19:37:25.056404] [DEBUG] [LOCAL ] IP: Primary data saved (53307 records). [2025-11-04T19:37:25.056530] [INFO ] [MAIN ] IP: Full history saved. [2025-11-04T19:37:25.056614] [INFO ] [MAIN ] Update Progress: 1120/2367 symbols completed. IP took 26.0s. Estimated remaining: 2h 55m 12s. Rate: 7.1 symbols/min [2025-11-04T19:37:25.642936] [INFO ] [MAIN ] IPG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:37:25.708706] [INFO ] [MAIN ] IQV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:37:25.749590] [INFO ] [MAIN ] IQI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:37:25.782582] [INFO ] [MAIN ] IPI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:37:25.817207] [DEBUG] [LOCAL ] IQI: Loaded 17431 existing records from primary storage. [2025-11-04T19:37:25.817515] [INFO ] [MAIN ] IQI: Loaded 17431 existing records. [2025-11-04T19:37:25.831177] [DEBUG] [LOCAL ] IPI: Loaded 21043 existing records from primary storage. [2025-11-04T19:37:25.906931] [INFO ] [MAIN ] IPI: Loaded 21043 existing records. [2025-11-04T19:37:26.077315] [DEBUG] [LOCAL ] IQV: Loaded 49826 existing records from primary storage. [2025-11-04T19:37:26.108060] [INFO ] [MAIN ] IQV: Loaded 49826 existing records. [2025-11-04T19:37:26.132978] [DEBUG] [LOCAL ] IPG: Loaded 53325 existing records from primary storage. [2025-11-04T19:37:26.162036] [INFO ] [MAIN ] IPG: Loaded 53325 existing records. [2025-11-04T19:37:26.244084] [WARN ] [MAIN ] IQI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:37:26.292601] [INFO ] [MAIN ] IQI: Wiping existing primary data before full refresh. [2025-11-04T19:37:26.313275] [INFO ] [LOCAL ] IQI: Starting deletion of primary data. [2025-11-04T19:37:26.364515] [DEBUG] [LOCAL ] IQI: Deleted primary data file. [2025-11-04T19:37:26.419712] [INFO ] [MAIN ] IQI: Starting full primary refresh. [2025-11-04T19:37:26.498290] [WARN ] [MAIN ] IPI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:37:26.568724] [INFO ] [MAIN ] IPI: Wiping existing primary data before full refresh. [2025-11-04T19:37:26.592232] [INFO ] [LOCAL ] IPI: Starting deletion of primary data. [2025-11-04T19:37:26.669753] [WARN ] [MAIN ] IQV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:37:26.715568] [DEBUG] [LOCAL ] IPI: Deleted primary data file. [2025-11-04T19:37:26.753224] [INFO ] [MAIN ] IPI: Starting full primary refresh. [2025-11-04T19:37:26.753474] [INFO ] [MAIN ] IQV: Wiping existing primary data before full refresh. [2025-11-04T19:37:26.756604] [INFO ] [LOCAL ] IQV: Starting deletion of primary data. [2025-11-04T19:37:26.844572] [WARN ] [MAIN ] IPG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:37:26.893736] [DEBUG] [LOCAL ] IQV: Deleted primary data file. [2025-11-04T19:37:26.924062] [INFO ] [MAIN ] IPG: Wiping existing primary data before full refresh. [2025-11-04T19:37:26.934999] [INFO ] [MAIN ] IQV: Starting full primary refresh. [2025-11-04T19:37:27.026496] [INFO ] [LOCAL ] IPG: Starting deletion of primary data. [2025-11-04T19:37:27.156961] [DEBUG] [LOCAL ] IPG: Deleted primary data file. [2025-11-04T19:37:27.193215] [INFO ] [MAIN ] IPG: Starting full primary refresh. [2025-11-04T19:37:27.880034] [INFO ] [REMOTE ] IQI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:37:28.051055] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-01. [2025-11-04T19:37:28.088882] [INFO ] [REMOTE ] IPI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:37:28.120803] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-01. [2025-11-04T19:37:28.201529] [INFO ] [REMOTE ] IQV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:37:28.217467] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-01. [2025-11-04T19:37:28.262598] [INFO ] [REMOTE ] IPG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:37:28.263033] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-01. [2025-11-04T19:37:28.964374] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-11-04T19:37:28.968027] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-02. [2025-11-04T19:37:29.234950] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T19:37:29.239488] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-02. [2025-11-04T19:37:29.449116] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T19:37:29.459332] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-02. [2025-11-04T19:37:29.470475] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-11-04T19:37:29.474407] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-03. [2025-11-04T19:37:29.583293] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-11-04T19:37:29.593534] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-02. [2025-11-04T19:37:30.075278] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-11-04T19:37:30.080254] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-03. [2025-11-04T19:37:30.251106] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-11-04T19:37:30.256359] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-04. [2025-11-04T19:37:30.617579] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T19:37:30.627374] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-03. [2025-11-04T19:37:30.825173] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-11-04T19:37:30.836590] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-11-04T19:37:30.926162] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-11-04T19:37:31.367119] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-04. [2025-11-04T19:37:31.816085] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T19:37:32.167022] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-05. [2025-11-04T19:37:32.207997] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-11-04T19:37:32.967043] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-03. [2025-11-04T19:37:33.092116] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-11-04T19:37:33.767032] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-06. [2025-11-04T19:37:34.293460] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T19:37:34.415923] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-11-04T19:37:34.567122] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-07. [2025-11-04T19:37:35.238149] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T19:37:35.367138] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-05. [2025-11-04T19:37:36.015733] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-11-04T19:37:36.167050] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-06. [2025-11-04T19:37:36.775264] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T19:37:36.967039] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-04. [2025-11-04T19:37:37.767037] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-07. [2025-11-04T19:37:38.301315] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T19:37:38.439961] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T19:37:38.567038] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-08. [2025-11-04T19:37:38.623615] [DEBUG] [REMOTE ] IPI: Error fetching primary data for 2025-08: 'API call returned an empty response.' [2025-11-04T19:37:38.623949] [WARN ] [REMOTE ] IPI: No data returned for month 2025-08 [2025-11-04T19:37:38.624054] [FATAL] [REMOTE ] IPI: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:37:38.624155] [FATAL] [MAIN ] IPI: Failed to fetch primary data. [2025-11-04T19:37:38.624226] [WARN ] [MAIN ] IPI: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:37:38.624317] [INFO ] [LOCAL ] IPI: Starting deletion of primary data. [2025-11-04T19:37:38.624400] [DEBUG] [LOCAL ] IPI: No primary data file found to delete. [2025-11-04T19:37:38.624440] [INFO ] [LOCAL ] IPI: Starting deletion of derivative data. [2025-11-04T19:37:38.624575] [DEBUG] [LOCAL ] IPI: Deleted derivative data file. [2025-11-04T19:37:38.624730] [INFO ] [MAIN ] Update Progress: 1121/2367 symbols completed. IPI took 13.6s. Estimated remaining: 2h 55m 10s. Rate: 7.1 symbols/min [2025-11-04T19:37:38.789826] [INFO ] [MAIN ] IR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:37:38.836207] [DEBUG] [LOCAL ] IR: Loaded 52740 existing records from primary storage. [2025-11-04T19:37:38.836521] [INFO ] [MAIN ] IR: Loaded 52740 existing records. [2025-11-04T19:37:39.008932] [WARN ] [MAIN ] IR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:37:39.009132] [INFO ] [MAIN ] IR: Wiping existing primary data before full refresh. [2025-11-04T19:37:39.009177] [INFO ] [LOCAL ] IR: Starting deletion of primary data. [2025-11-04T19:37:39.009346] [DEBUG] [LOCAL ] IR: Deleted primary data file. [2025-11-04T19:37:39.009382] [INFO ] [MAIN ] IR: Starting full primary refresh. [2025-11-04T19:37:39.366131] [INFO ] [REMOTE ] IR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:37:39.367002] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-01. [2025-11-04T19:37:40.167063] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-05. [2025-11-04T19:37:40.767881] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T19:37:40.967040] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-02. [2025-11-04T19:37:41.587570] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-11-04T19:37:41.767048] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-06. [2025-11-04T19:37:42.311906] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-11-04T19:37:42.567069] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-04. [2025-11-04T19:37:42.989456] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-11-04T19:37:43.367031] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-07. [2025-11-04T19:37:44.001822] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-11-04T19:37:44.167078] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-08. [2025-11-04T19:37:44.770727] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T19:37:44.932957] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-11-04T19:37:44.967009] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-09. [2025-11-04T19:37:45.642159] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T19:37:45.767100] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-05. [2025-11-04T19:37:46.567055] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-08. [2025-11-04T19:37:47.037009] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-11-04T19:37:47.367060] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-03. [2025-11-04T19:37:47.800720] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T19:37:48.167084] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-10. [2025-11-04T19:37:48.752936] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T19:37:48.798767] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-11-04T19:37:48.967115] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-09. [2025-11-04T19:37:49.767080] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-04. [2025-11-04T19:37:50.190932] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T19:37:50.567096] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-06. [2025-11-04T19:37:51.057687] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-11-04T19:37:51.367061] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-11. [2025-11-04T19:37:51.776557] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-11-04T19:37:51.953041] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-11 (0.59 seconds). [2025-11-04T19:37:51.954197] [INFO ] [REMOTE ] IQI: Full fetch complete (26225 records). [2025-11-04T19:37:51.954266] [DEBUG] [VALIDATOR ] IQI: Validating primary data. [2025-11-04T19:37:52.172170] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-07. [2025-11-04T19:37:52.321399] [DEBUG] [VALIDATOR ] IQI: All expected trading days are present. [2025-11-04T19:37:52.321467] [DEBUG] [VALIDATOR ] IQI: Primary data validation passed. [2025-11-04T19:37:52.321999] [INFO ] [MAIN ] IQI: Fetched primary data is valid. [2025-11-04T19:37:52.322058] [DEBUG] [LOCAL ] IQI: Saving primary data. [2025-11-04T19:37:52.322634] [DEBUG] [LOCAL ] IQI: Trimmed 26225 records to 17481 (keeping 200 days of data) [2025-11-04T19:37:52.425275] [DEBUG] [LOCAL ] IQI: Primary data saved (17481 records). [2025-11-04T19:37:52.425394] [INFO ] [MAIN ] IQI: Full history saved. [2025-11-04T19:37:52.425484] [INFO ] [MAIN ] Update Progress: 1122/2367 symbols completed. IQI took 27.4s. Estimated remaining: 2h 55m 7s. Rate: 7.1 symbols/min [2025-11-04T19:37:52.591044] [INFO ] [MAIN ] IRM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:37:52.632864] [DEBUG] [LOCAL ] IRM: Loaded 48512 existing records from primary storage. [2025-11-04T19:37:52.633170] [INFO ] [MAIN ] IRM: Loaded 48512 existing records. [2025-11-04T19:37:52.805195] [WARN ] [MAIN ] IRM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:37:52.805366] [INFO ] [MAIN ] IRM: Wiping existing primary data before full refresh. [2025-11-04T19:37:52.805411] [INFO ] [LOCAL ] IRM: Starting deletion of primary data. [2025-11-04T19:37:52.805800] [DEBUG] [LOCAL ] IRM: Deleted primary data file. [2025-11-04T19:37:52.805862] [INFO ] [MAIN ] IRM: Starting full primary refresh. [2025-11-04T19:37:52.975577] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-10. [2025-11-04T19:37:53.158493] [INFO ] [REMOTE ] IRM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:37:53.487590] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T19:37:53.767047] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-08. [2025-11-04T19:37:54.390024] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-11-04T19:37:54.567036] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-01. [2025-11-04T19:37:54.973976] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T19:37:55.367064] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-05. [2025-11-04T19:37:55.902482] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-11-04T19:37:56.167042] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-02. [2025-11-04T19:37:56.514712] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T19:37:56.967046] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-09. [2025-11-04T19:37:57.452310] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T19:37:57.767046] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-11. [2025-11-04T19:37:58.344284] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-11-04T19:37:58.567028] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-10. [2025-11-04T19:37:58.836794] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T19:37:58.838761] [INFO ] [REMOTE ] IQV: Full fetch complete (72628 records). [2025-11-04T19:37:58.838858] [DEBUG] [VALIDATOR ] IQV: Validating primary data. [2025-11-04T19:37:59.223337] [DEBUG] [VALIDATOR ] IQV: All expected trading days are present. [2025-11-04T19:37:59.223425] [DEBUG] [VALIDATOR ] IQV: Primary data validation passed. [2025-11-04T19:37:59.224332] [INFO ] [MAIN ] IQV: Fetched primary data is valid. [2025-11-04T19:37:59.224393] [DEBUG] [LOCAL ] IQV: Saving primary data. [2025-11-04T19:37:59.225698] [DEBUG] [LOCAL ] IQV: Trimmed 72628 records to 49815 (keeping 200 days of data) [2025-11-04T19:37:59.391177] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-03. [2025-11-04T19:37:59.520949] [DEBUG] [LOCAL ] IQV: Primary data saved (49815 records). [2025-11-04T19:37:59.521100] [INFO ] [MAIN ] IQV: Full history saved. [2025-11-04T19:37:59.521183] [INFO ] [MAIN ] Update Progress: 1123/2367 symbols completed. IQV took 34.5s. Estimated remaining: 2h 54m 57s. Rate: 7.1 symbols/min [2025-11-04T19:37:59.686161] [INFO ] [MAIN ] IRS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:37:59.709227] [DEBUG] [LOCAL ] IRS: Loaded 24001 existing records from primary storage. [2025-11-04T19:37:59.709461] [INFO ] [MAIN ] IRS: Loaded 24001 existing records. [2025-11-04T19:37:59.887934] [WARN ] [MAIN ] IRS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:37:59.888247] [INFO ] [MAIN ] IRS: Wiping existing primary data before full refresh. [2025-11-04T19:37:59.888302] [INFO ] [LOCAL ] IRS: Starting deletion of primary data. [2025-11-04T19:37:59.888596] [DEBUG] [LOCAL ] IRS: Deleted primary data file. [2025-11-04T19:37:59.888709] [INFO ] [MAIN ] IRS: Starting full primary refresh. [2025-11-04T19:38:00.178306] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-06. [2025-11-04T19:38:00.258409] [INFO ] [REMOTE ] IRS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:38:00.294345] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-11-04T19:38:00.695197] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T19:38:00.967052] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-11. [2025-11-04T19:38:01.470271] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-11-04T19:38:01.767056] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-07. [2025-11-04T19:38:02.020729] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T19:38:02.022782] [INFO ] [REMOTE ] IPG: Full fetch complete (77440 records). [2025-11-04T19:38:02.022888] [DEBUG] [VALIDATOR ] IPG: Validating primary data. [2025-11-04T19:38:02.422850] [DEBUG] [VALIDATOR ] IPG: All expected trading days are present. [2025-11-04T19:38:02.422931] [DEBUG] [VALIDATOR ] IPG: Primary data validation passed. [2025-11-04T19:38:02.423939] [INFO ] [MAIN ] IPG: Fetched primary data is valid. [2025-11-04T19:38:02.424006] [DEBUG] [LOCAL ] IPG: Saving primary data. [2025-11-04T19:38:02.425468] [DEBUG] [LOCAL ] IPG: Trimmed 77440 records to 53324 (keeping 200 days of data) [2025-11-04T19:38:02.600446] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-04. [2025-11-04T19:38:02.751334] [DEBUG] [LOCAL ] IPG: Primary data saved (53324 records). [2025-11-04T19:38:02.751471] [INFO ] [MAIN ] IPG: Full history saved. [2025-11-04T19:38:02.751567] [INFO ] [MAIN ] Update Progress: 1124/2367 symbols completed. IPG took 37.7s. Estimated remaining: 2h 54m 43s. Rate: 7.1 symbols/min [2025-11-04T19:38:02.923755] [INFO ] [MAIN ] IRT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:38:02.968905] [DEBUG] [LOCAL ] IRT: Loaded 50368 existing records from primary storage. [2025-11-04T19:38:02.969209] [INFO ] [MAIN ] IRT: Loaded 50368 existing records. [2025-11-04T19:38:03.154068] [WARN ] [MAIN ] IRT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:38:03.154404] [INFO ] [MAIN ] IRT: Wiping existing primary data before full refresh. [2025-11-04T19:38:03.154512] [INFO ] [LOCAL ] IRT: Starting deletion of primary data. [2025-11-04T19:38:03.165915] [DEBUG] [LOCAL ] IRT: Deleted primary data file. [2025-11-04T19:38:03.184942] [INFO ] [MAIN ] IRT: Starting full primary refresh. [2025-11-04T19:38:03.260285] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-11-04T19:38:03.380557] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-08. [2025-11-04T19:38:03.566446] [INFO ] [REMOTE ] IRT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:38:03.948938] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:38:04.167040] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-05. [2025-11-04T19:38:04.888556] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-11-04T19:38:04.967034] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-01. [2025-11-04T19:38:05.502590] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T19:38:05.767047] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-01. [2025-11-04T19:38:06.322287] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T19:38:06.567041] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-06. [2025-11-04T19:38:06.830778] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T19:38:07.367044] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-09. [2025-11-04T19:38:07.929147] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T19:38:08.167084] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-02. [2025-11-04T19:38:08.621852] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T19:38:08.967043] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-07. [2025-11-04T19:38:09.338741] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T19:38:09.767043] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-03. [2025-11-04T19:38:10.232564] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T19:38:10.567023] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-10. [2025-11-04T19:38:10.911741] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T19:38:11.367050] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-04. [2025-11-04T19:38:11.938483] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T19:38:12.167040] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-11. [2025-11-04T19:38:12.684763] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T19:38:12.967102] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-08. [2025-11-04T19:38:13.464839] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T19:38:13.466845] [INFO ] [REMOTE ] IR: Full fetch complete (76515 records). [2025-11-04T19:38:13.466956] [DEBUG] [VALIDATOR ] IR: Validating primary data. [2025-11-04T19:38:13.772233] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-02. [2025-11-04T19:38:13.854172] [DEBUG] [VALIDATOR ] IR: All expected trading days are present. [2025-11-04T19:38:13.854247] [DEBUG] [VALIDATOR ] IR: Primary data validation passed. [2025-11-04T19:38:13.855163] [INFO ] [MAIN ] IR: Fetched primary data is valid. [2025-11-04T19:38:13.855223] [DEBUG] [LOCAL ] IR: Saving primary data. [2025-11-04T19:38:13.856565] [DEBUG] [LOCAL ] IR: Trimmed 76515 records to 52752 (keeping 200 days of data) [2025-11-04T19:38:14.170111] [DEBUG] [LOCAL ] IR: Primary data saved (52752 records). [2025-11-04T19:38:14.170232] [INFO ] [MAIN ] IR: Full history saved. [2025-11-04T19:38:14.170304] [INFO ] [MAIN ] Update Progress: 1125/2367 symbols completed. IR took 35.5s. Estimated remaining: 2h 54m 38s. Rate: 7.1 symbols/min [2025-11-04T19:38:14.342779] [INFO ] [MAIN ] ISD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:38:14.374310] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-11-04T19:38:14.394502] [DEBUG] [LOCAL ] ISD: Loaded 15098 existing records from primary storage. [2025-11-04T19:38:14.394749] [INFO ] [MAIN ] ISD: Loaded 15098 existing records. [2025-11-04T19:38:14.569106] [WARN ] [MAIN ] ISD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:38:14.569408] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-05. [2025-11-04T19:38:14.569517] [INFO ] [MAIN ] ISD: Wiping existing primary data before full refresh. [2025-11-04T19:38:14.570335] [INFO ] [LOCAL ] ISD: Starting deletion of primary data. [2025-11-04T19:38:14.570503] [DEBUG] [LOCAL ] ISD: Deleted primary data file. [2025-11-04T19:38:14.570542] [INFO ] [MAIN ] ISD: Starting full primary refresh. [2025-11-04T19:38:14.828403] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-11-04T19:38:14.938878] [INFO ] [REMOTE ] ISD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:38:15.367068] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-01. [2025-11-04T19:38:15.968983] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-11-04T19:38:16.167061] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-03. [2025-11-04T19:38:16.363431] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T19:38:16.825239] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-11-04T19:38:16.967073] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-09. [2025-11-04T19:38:17.767027] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-04. [2025-11-04T19:38:18.354855] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T19:38:18.567027] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-10. [2025-11-04T19:38:18.612275] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-11-04T19:38:19.367121] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-06. [2025-11-04T19:38:19.833544] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T19:38:20.167071] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-02. [2025-11-04T19:38:20.596395] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-11-04T19:38:20.763953] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-11-04T19:38:20.967051] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-07. [2025-11-04T19:38:21.767035] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-05. [2025-11-04T19:38:22.160556] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-11-04T19:38:22.481332] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-11-04T19:38:22.567041] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-08. [2025-11-04T19:38:23.367036] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-03. [2025-11-04T19:38:23.773504] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T19:38:24.059520] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-11-04T19:38:24.167058] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-04. [2025-11-04T19:38:24.721878] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-11-04T19:38:24.967044] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-11. [2025-11-04T19:38:25.767040] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-06. [2025-11-04T19:38:26.120005] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T19:38:26.121959] [INFO ] [REMOTE ] IRM: Full fetch complete (71722 records). [2025-11-04T19:38:26.122092] [DEBUG] [VALIDATOR ] IRM: Validating primary data. [2025-11-04T19:38:26.572427] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-09. [2025-11-04T19:38:26.583041] [DEBUG] [VALIDATOR ] IRM: All expected trading days are present. [2025-11-04T19:38:26.583276] [DEBUG] [VALIDATOR ] IRM: Primary data validation passed. [2025-11-04T19:38:26.585963] [INFO ] [MAIN ] IRM: Fetched primary data is valid. [2025-11-04T19:38:26.586488] [DEBUG] [LOCAL ] IRM: Saving primary data. [2025-11-04T19:38:26.589631] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-11-04T19:38:26.598701] [DEBUG] [LOCAL ] IRM: Trimmed 71722 records to 48526 (keeping 200 days of data) [2025-11-04T19:38:26.910440] [DEBUG] [LOCAL ] IRM: Primary data saved (48526 records). [2025-11-04T19:38:26.910583] [INFO ] [MAIN ] IRM: Full history saved. [2025-11-04T19:38:26.910687] [INFO ] [MAIN ] Update Progress: 1126/2367 symbols completed. IRM took 34.5s. Estimated remaining: 2h 54m 34s. Rate: 7.1 symbols/min [2025-11-04T19:38:27.080983] [INFO ] [MAIN ] IT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:38:27.119583] [DEBUG] [LOCAL ] IT: Loaded 42374 existing records from primary storage. [2025-11-04T19:38:27.119928] [INFO ] [MAIN ] IT: Loaded 42374 existing records. [2025-11-04T19:38:27.347397] [WARN ] [MAIN ] IT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:38:27.347731] [INFO ] [MAIN ] IT: Wiping existing primary data before full refresh. [2025-11-04T19:38:27.347798] [INFO ] [LOCAL ] IT: Starting deletion of primary data. [2025-11-04T19:38:27.348700] [DEBUG] [LOCAL ] IT: Deleted primary data file. [2025-11-04T19:38:27.348763] [INFO ] [MAIN ] IT: Starting full primary refresh. [2025-11-04T19:38:27.372244] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-07. [2025-11-04T19:38:27.908324] [INFO ] [REMOTE ] IT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:38:27.958098] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T19:38:28.058738] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-11-04T19:38:28.167101] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-01. [2025-11-04T19:38:28.967124] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-05. [2025-11-04T19:38:29.301923] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-11-04T19:38:29.578072] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-11-04T19:38:29.767052] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-10. [2025-11-04T19:38:30.567042] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-06. [2025-11-04T19:38:31.249877] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-11-04T19:38:31.277030] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-11-04T19:38:31.367089] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-11. [2025-11-04T19:38:32.167031] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-08. [2025-11-04T19:38:32.518595] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T19:38:32.520620] [INFO ] [REMOTE ] IRT: Full fetch complete (72992 records). [2025-11-04T19:38:32.520765] [DEBUG] [VALIDATOR ] IRT: Validating primary data. [2025-11-04T19:38:32.910025] [DEBUG] [VALIDATOR ] IRT: All expected trading days are present. [2025-11-04T19:38:32.910123] [DEBUG] [VALIDATOR ] IRT: Primary data validation passed. [2025-11-04T19:38:32.911086] [INFO ] [MAIN ] IRT: Fetched primary data is valid. [2025-11-04T19:38:32.911166] [DEBUG] [LOCAL ] IRT: Saving primary data. [2025-11-04T19:38:32.912559] [DEBUG] [LOCAL ] IRT: Trimmed 72992 records to 50387 (keeping 200 days of data) [2025-11-04T19:38:32.971974] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-07. [2025-11-04T19:38:33.020334] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-11-04T19:38:33.226870] [DEBUG] [LOCAL ] IRT: Primary data saved (50387 records). [2025-11-04T19:38:33.227016] [INFO ] [MAIN ] IRT: Full history saved. [2025-11-04T19:38:33.227110] [INFO ] [MAIN ] Update Progress: 1127/2367 symbols completed. IRT took 30.5s. Estimated remaining: 2h 54m 23s. Rate: 7.1 symbols/min [2025-11-04T19:38:33.392073] [INFO ] [MAIN ] ITGR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:38:33.421956] [DEBUG] [LOCAL ] ITGR: Loaded 31278 existing records from primary storage. [2025-11-04T19:38:33.422210] [INFO ] [MAIN ] ITGR: Loaded 31278 existing records. [2025-11-04T19:38:33.602479] [WARN ] [MAIN ] ITGR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:38:33.602799] [INFO ] [MAIN ] ITGR: Wiping existing primary data before full refresh. [2025-11-04T19:38:33.602882] [INFO ] [LOCAL ] ITGR: Starting deletion of primary data. [2025-11-04T19:38:33.603226] [DEBUG] [LOCAL ] ITGR: Deleted primary data file. [2025-11-04T19:38:33.603281] [INFO ] [MAIN ] ITGR: Starting full primary refresh. [2025-11-04T19:38:33.787482] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T19:38:33.786980] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-09. [2025-11-04T19:38:33.983217] [INFO ] [REMOTE ] ITGR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:38:34.567081] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-08. [2025-11-04T19:38:34.766923] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T19:38:35.254218] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-11-04T19:38:35.367054] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-09. [2025-11-04T19:38:36.000601] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:38:36.167058] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-01. [2025-11-04T19:38:36.922313] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-11-04T19:38:36.967054] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-10. [2025-11-04T19:38:37.767047] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-10. [2025-11-04T19:38:37.874555] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-11-04T19:38:38.508450] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-11-04T19:38:38.567033] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-02. [2025-11-04T19:38:39.367051] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-11. [2025-11-04T19:38:39.394227] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T19:38:40.154585] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-11 (0.79 seconds). [2025-11-04T19:38:40.155876] [INFO ] [REMOTE ] IRS: Full fetch complete (32979 records). [2025-11-04T19:38:40.155946] [DEBUG] [VALIDATOR ] IRS: Validating primary data. [2025-11-04T19:38:40.176441] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-02. [2025-11-04T19:38:40.539182] [DEBUG] [VALIDATOR ] IRS: All expected trading days are present. [2025-11-04T19:38:40.539259] [DEBUG] [VALIDATOR ] IRS: Primary data validation passed. [2025-11-04T19:38:40.539926] [INFO ] [MAIN ] IRS: Fetched primary data is valid. [2025-11-04T19:38:40.539988] [DEBUG] [LOCAL ] IRS: Saving primary data. [2025-11-04T19:38:40.540802] [DEBUG] [LOCAL ] IRS: Trimmed 32979 records to 24017 (keeping 200 days of data) [2025-11-04T19:38:40.675614] [DEBUG] [LOCAL ] IRS: Primary data saved (24017 records). [2025-11-04T19:38:40.675765] [INFO ] [MAIN ] IRS: Full history saved. [2025-11-04T19:38:40.675844] [INFO ] [MAIN ] Update Progress: 1128/2367 symbols completed. IRS took 41.2s. Estimated remaining: 2h 54m 14s. Rate: 7.1 symbols/min [2025-11-04T19:38:40.851535] [INFO ] [MAIN ] ITT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:38:40.881190] [DEBUG] [LOCAL ] ITT: Loaded 31756 existing records from primary storage. [2025-11-04T19:38:40.881474] [INFO ] [MAIN ] ITT: Loaded 31756 existing records. [2025-11-04T19:38:40.972230] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-03. [2025-11-04T19:38:41.073106] [WARN ] [MAIN ] ITT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:38:41.073375] [INFO ] [MAIN ] ITT: Wiping existing primary data before full refresh. [2025-11-04T19:38:41.073492] [INFO ] [LOCAL ] ITT: Starting deletion of primary data. [2025-11-04T19:38:41.074159] [DEBUG] [LOCAL ] ITT: Deleted primary data file. [2025-11-04T19:38:41.074217] [INFO ] [MAIN ] ITT: Starting full primary refresh. [2025-11-04T19:38:41.474062] [INFO ] [REMOTE ] ITT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:38:41.476147] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-11-04T19:38:41.767070] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-03. [2025-11-04T19:38:42.029376] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-11-04T19:38:42.567027] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-04. [2025-11-04T19:38:42.802194] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T19:38:43.367040] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-04. [2025-11-04T19:38:43.714746] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-11-04T19:38:44.167037] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-11. [2025-11-04T19:38:44.527472] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-11-04T19:38:44.792789] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T19:38:44.793701] [INFO ] [REMOTE ] ISD: Full fetch complete (23180 records). [2025-11-04T19:38:44.793770] [DEBUG] [VALIDATOR ] ISD: Validating primary data. [2025-11-04T19:38:44.972160] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-05. [2025-11-04T19:38:45.152994] [DEBUG] [VALIDATOR ] ISD: All expected trading days are present. [2025-11-04T19:38:45.153074] [DEBUG] [VALIDATOR ] ISD: Primary data validation passed. [2025-11-04T19:38:45.153529] [INFO ] [MAIN ] ISD: Fetched primary data is valid. [2025-11-04T19:38:45.153567] [DEBUG] [LOCAL ] ISD: Saving primary data. [2025-11-04T19:38:45.154161] [DEBUG] [LOCAL ] ISD: Trimmed 23180 records to 15009 (keeping 200 days of data) [2025-11-04T19:38:45.244472] [DEBUG] [LOCAL ] ISD: Primary data saved (15009 records). [2025-11-04T19:38:45.244659] [INFO ] [MAIN ] ISD: Full history saved. [2025-11-04T19:38:45.244755] [INFO ] [MAIN ] Update Progress: 1129/2367 symbols completed. ISD took 31.1s. Estimated remaining: 2h 54m 1s. Rate: 7.1 symbols/min [2025-11-04T19:38:45.409762] [INFO ] [MAIN ] ITUB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:38:45.454181] [DEBUG] [LOCAL ] ITUB: Loaded 53298 existing records from primary storage. [2025-11-04T19:38:45.454510] [INFO ] [MAIN ] ITUB: Loaded 53298 existing records. [2025-11-04T19:38:45.627321] [WARN ] [MAIN ] ITUB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:38:45.627535] [INFO ] [MAIN ] ITUB: Wiping existing primary data before full refresh. [2025-11-04T19:38:45.627583] [INFO ] [LOCAL ] ITUB: Starting deletion of primary data. [2025-11-04T19:38:45.628497] [DEBUG] [LOCAL ] ITUB: Deleted primary data file. [2025-11-04T19:38:45.628555] [INFO ] [MAIN ] ITUB: Starting full primary refresh. [2025-11-04T19:38:45.772119] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-01. [2025-11-04T19:38:46.004931] [INFO ] [REMOTE ] ITUB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:38:46.157629] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T19:38:46.567068] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-06. [2025-11-04T19:38:47.078638] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T19:38:47.367070] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-02. [2025-11-04T19:38:47.755031] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-11-04T19:38:48.167004] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-05. [2025-11-04T19:38:48.375271] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-11-04T19:38:48.967087] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-07. [2025-11-04T19:38:49.183616] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T19:38:49.767111] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-01. [2025-11-04T19:38:50.238898] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T19:38:50.567079] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-08. [2025-11-04T19:38:51.266111] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T19:38:51.367027] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-02. [2025-11-04T19:38:52.060436] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-11-04T19:38:52.167018] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-06. [2025-11-04T19:38:52.624831] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T19:38:52.862005] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-11-04T19:38:52.967014] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-09. [2025-11-04T19:38:53.767053] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-03. [2025-11-04T19:38:54.235284] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-11-04T19:38:54.567047] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-03. [2025-11-04T19:38:54.839808] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T19:38:55.367044] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-04. [2025-11-04T19:38:56.005591] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-11-04T19:38:56.167044] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-04. [2025-11-04T19:38:56.729411] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:38:56.967035] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-05. [2025-11-04T19:38:57.611767] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T19:38:57.767008] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-10. [2025-11-04T19:38:58.015178] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T19:38:58.567073] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-06. [2025-11-04T19:38:59.112597] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T19:38:59.367009] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-05. [2025-11-04T19:38:59.503526] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-11-04T19:39:00.167039] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-07. [2025-11-04T19:39:00.652706] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T19:39:00.967057] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-11. [2025-11-04T19:39:01.164370] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T19:39:01.767073] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-08. [2025-11-04T19:39:02.208869] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T19:39:02.210804] [INFO ] [REMOTE ] IT: Full fetch complete (58813 records). [2025-11-04T19:39:02.210896] [DEBUG] [VALIDATOR ] IT: Validating primary data. [2025-11-04T19:39:02.589258] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-06. [2025-11-04T19:39:02.614522] [DEBUG] [VALIDATOR ] IT: All expected trading days are present. [2025-11-04T19:39:02.614662] [DEBUG] [VALIDATOR ] IT: Primary data validation passed. [2025-11-04T19:39:02.615628] [INFO ] [MAIN ] IT: Fetched primary data is valid. [2025-11-04T19:39:02.615821] [DEBUG] [LOCAL ] IT: Saving primary data. [2025-11-04T19:39:02.622692] [DEBUG] [LOCAL ] IT: Trimmed 58813 records to 42541 (keeping 200 days of data) [2025-11-04T19:39:02.633338] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-11-04T19:39:02.888353] [DEBUG] [LOCAL ] IT: Primary data saved (42541 records). [2025-11-04T19:39:02.888479] [INFO ] [MAIN ] IT: Full history saved. [2025-11-04T19:39:02.888556] [INFO ] [MAIN ] Update Progress: 1130/2367 symbols completed. IT took 36.0s. Estimated remaining: 2h 54m 3s. Rate: 7.1 symbols/min [2025-11-04T19:39:03.068891] [INFO ] [MAIN ] ITW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:39:03.109509] [DEBUG] [LOCAL ] ITW: Loaded 45555 existing records from primary storage. [2025-11-04T19:39:03.109813] [INFO ] [MAIN ] ITW: Loaded 45555 existing records. [2025-11-04T19:39:03.298080] [WARN ] [MAIN ] ITW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:39:03.298296] [INFO ] [MAIN ] ITW: Wiping existing primary data before full refresh. [2025-11-04T19:39:03.298344] [INFO ] [LOCAL ] ITW: Starting deletion of primary data. [2025-11-04T19:39:03.298686] [DEBUG] [LOCAL ] ITW: Deleted primary data file. [2025-11-04T19:39:03.298743] [INFO ] [MAIN ] ITW: Starting full primary refresh. [2025-11-04T19:39:03.372227] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-09. [2025-11-04T19:39:03.670445] [INFO ] [REMOTE ] ITW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:39:04.172185] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-07. [2025-11-04T19:39:04.191708] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-11-04T19:39:04.350539] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-11-04T19:39:04.967034] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-01. [2025-11-04T19:39:05.329538] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-11-04T19:39:05.767046] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-08. [2025-11-04T19:39:06.218829] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-11-04T19:39:06.567014] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-02. [2025-11-04T19:39:06.705544] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-11-04T19:39:07.367039] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-07. [2025-11-04T19:39:07.914472] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T19:39:08.167046] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-09. [2025-11-04T19:39:08.748305] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-11-04T19:39:08.967055] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-10. [2025-11-04T19:39:09.129282] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-11-04T19:39:09.767057] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-10. [2025-11-04T19:39:10.015367] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-11-04T19:39:10.567028] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-11. [2025-11-04T19:39:10.703756] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T19:39:11.367082] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-08. [2025-11-04T19:39:11.817292] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T19:39:11.819086] [INFO ] [REMOTE ] ITGR: Full fetch complete (45395 records). [2025-11-04T19:39:11.819194] [DEBUG] [VALIDATOR ] ITGR: Validating primary data. [2025-11-04T19:39:12.172464] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-03. [2025-11-04T19:39:12.188914] [DEBUG] [VALIDATOR ] ITGR: All expected trading days are present. [2025-11-04T19:39:12.188970] [DEBUG] [VALIDATOR ] ITGR: Primary data validation passed. [2025-11-04T19:39:12.189791] [INFO ] [MAIN ] ITGR: Fetched primary data is valid. [2025-11-04T19:39:12.189940] [DEBUG] [LOCAL ] ITGR: Saving primary data. [2025-11-04T19:39:12.190929] [DEBUG] [LOCAL ] ITGR: Trimmed 45395 records to 31277 (keeping 200 days of data) [2025-11-04T19:39:12.371883] [DEBUG] [LOCAL ] ITGR: Primary data saved (31277 records). [2025-11-04T19:39:12.372008] [INFO ] [MAIN ] ITGR: Full history saved. [2025-11-04T19:39:12.372086] [INFO ] [MAIN ] Update Progress: 1131/2367 symbols completed. ITGR took 39.1s. Estimated remaining: 2h 53m 56s. Rate: 7.1 symbols/min [2025-11-04T19:39:12.540699] [INFO ] [MAIN ] IVR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:39:12.582125] [DEBUG] [LOCAL ] IVR: Loaded 47524 existing records from primary storage. [2025-11-04T19:39:12.582426] [INFO ] [MAIN ] IVR: Loaded 47524 existing records. [2025-11-04T19:39:12.761977] [WARN ] [MAIN ] IVR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:39:12.762242] [INFO ] [MAIN ] IVR: Wiping existing primary data before full refresh. [2025-11-04T19:39:12.762398] [INFO ] [LOCAL ] IVR: Starting deletion of primary data. [2025-11-04T19:39:12.762715] [DEBUG] [LOCAL ] IVR: Deleted primary data file. [2025-11-04T19:39:12.762856] [INFO ] [MAIN ] IVR: Starting full primary refresh. [2025-11-04T19:39:12.982336] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-11. [2025-11-04T19:39:13.101165] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-11-04T19:39:13.172463] [INFO ] [REMOTE ] IVR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:39:13.634821] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T19:39:13.767000] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-04. [2025-11-04T19:39:14.073538] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T19:39:14.075078] [INFO ] [REMOTE ] ITT: Full fetch complete (48015 records). [2025-11-04T19:39:14.075152] [DEBUG] [VALIDATOR ] ITT: Validating primary data. [2025-11-04T19:39:14.461513] [DEBUG] [VALIDATOR ] ITT: All expected trading days are present. [2025-11-04T19:39:14.461672] [DEBUG] [VALIDATOR ] ITT: Primary data validation passed. [2025-11-04T19:39:14.462931] [INFO ] [MAIN ] ITT: Fetched primary data is valid. [2025-11-04T19:39:14.463042] [DEBUG] [LOCAL ] ITT: Saving primary data. [2025-11-04T19:39:14.464581] [DEBUG] [LOCAL ] ITT: Trimmed 48015 records to 31638 (keeping 200 days of data) [2025-11-04T19:39:14.577121] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-01. [2025-11-04T19:39:14.731107] [DEBUG] [LOCAL ] ITT: Primary data saved (31638 records). [2025-11-04T19:39:14.731247] [INFO ] [MAIN ] ITT: Full history saved. [2025-11-04T19:39:14.731349] [INFO ] [MAIN ] Update Progress: 1132/2367 symbols completed. ITT took 34.1s. Estimated remaining: 2h 53m 41s. Rate: 7.1 symbols/min [2025-11-04T19:39:15.007602] [INFO ] [MAIN ] IVT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:39:15.052478] [DEBUG] [LOCAL ] IVT: Loaded 31214 existing records from primary storage. [2025-11-04T19:39:15.052951] [INFO ] [MAIN ] IVT: Loaded 31214 existing records. [2025-11-04T19:39:15.322733] [WARN ] [MAIN ] IVT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:39:15.323037] [INFO ] [MAIN ] IVT: Wiping existing primary data before full refresh. [2025-11-04T19:39:15.323185] [INFO ] [LOCAL ] IVT: Starting deletion of primary data. [2025-11-04T19:39:15.323530] [DEBUG] [LOCAL ] IVT: Deleted primary data file. [2025-11-04T19:39:15.323735] [INFO ] [MAIN ] IVT: Starting full primary refresh. [2025-11-04T19:39:15.369545] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-09. [2025-11-04T19:39:15.477240] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-11-04T19:39:15.741946] [INFO ] [REMOTE ] IVT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:39:15.952637] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T19:39:16.167043] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-01. [2025-11-04T19:39:16.760135] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T19:39:16.967801] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-05. [2025-11-04T19:39:16.969998] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-11-04T19:39:17.767023] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-02. [2025-11-04T19:39:18.253854] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T19:39:18.567014] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-06. [2025-11-04T19:39:18.659095] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T19:39:19.367099] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-03. [2025-11-04T19:39:19.626555] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-11-04T19:39:20.167015] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-07. [2025-11-04T19:39:20.372540] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T19:39:20.967037] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-02. [2025-11-04T19:39:21.553212] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-11-04T19:39:21.767048] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-08. [2025-11-04T19:39:22.141707] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-11-04T19:39:22.567038] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-04. [2025-11-04T19:39:23.075985] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-11-04T19:39:23.367034] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-09. [2025-11-04T19:39:23.597104] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-11-04T19:39:24.167064] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-05. [2025-11-04T19:39:24.466518] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-11-04T19:39:24.903988] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-11-04T19:39:24.967053] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-10. [2025-11-04T19:39:25.767048] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-06. [2025-11-04T19:39:26.534855] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-11-04T19:39:26.567050] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-11. [2025-11-04T19:39:26.678313] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T19:39:27.367041] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-10. [2025-11-04T19:39:27.889237] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-11 (1.32 seconds). [2025-11-04T19:39:27.891166] [INFO ] [REMOTE ] ITW: Full fetch complete (67198 records). [2025-11-04T19:39:27.891273] [DEBUG] [VALIDATOR ] ITW: Validating primary data. [2025-11-04T19:39:28.172290] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-03. [2025-11-04T19:39:28.294523] [DEBUG] [VALIDATOR ] ITW: All expected trading days are present. [2025-11-04T19:39:28.294617] [DEBUG] [VALIDATOR ] ITW: Primary data validation passed. [2025-11-04T19:39:28.295477] [INFO ] [MAIN ] ITW: Fetched primary data is valid. [2025-11-04T19:39:28.295531] [DEBUG] [LOCAL ] ITW: Saving primary data. [2025-11-04T19:39:28.296792] [DEBUG] [LOCAL ] ITW: Trimmed 67198 records to 45585 (keeping 200 days of data) [2025-11-04T19:39:28.576190] [DEBUG] [LOCAL ] ITW: Primary data saved (45585 records). [2025-11-04T19:39:28.576334] [INFO ] [MAIN ] ITW: Full history saved. [2025-11-04T19:39:28.576423] [INFO ] [MAIN ] Update Progress: 1133/2367 symbols completed. ITW took 25.7s. Estimated remaining: 2h 53m 38s. Rate: 7.1 symbols/min [2025-11-04T19:39:28.742508] [INFO ] [MAIN ] IVZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:39:28.791751] [DEBUG] [LOCAL ] IVZ: Loaded 53126 existing records from primary storage. [2025-11-04T19:39:28.792054] [INFO ] [MAIN ] IVZ: Loaded 53126 existing records. [2025-11-04T19:39:28.976096] [WARN ] [MAIN ] IVZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:39:28.970776] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-07. [2025-11-04T19:39:28.976491] [INFO ] [MAIN ] IVZ: Wiping existing primary data before full refresh. [2025-11-04T19:39:28.977900] [INFO ] [LOCAL ] IVZ: Starting deletion of primary data. [2025-11-04T19:39:28.978163] [DEBUG] [LOCAL ] IVZ: Deleted primary data file. [2025-11-04T19:39:28.978218] [INFO ] [MAIN ] IVZ: Starting full primary refresh. [2025-11-04T19:39:29.293269] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-11-04T19:39:29.404933] [INFO ] [REMOTE ] IVZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:39:29.683032] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-11-04T19:39:29.766995] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-11. [2025-11-04T19:39:29.847568] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-11-04T19:39:30.567074] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-01. [2025-11-04T19:39:31.114017] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-11 (1.35 seconds). [2025-11-04T19:39:31.115985] [INFO ] [REMOTE ] ITUB: Full fetch complete (77298 records). [2025-11-04T19:39:31.116087] [DEBUG] [VALIDATOR ] ITUB: Validating primary data. [2025-11-04T19:39:31.379209] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-08. [2025-11-04T19:39:31.520551] [DEBUG] [VALIDATOR ] ITUB: All expected trading days are present. [2025-11-04T19:39:31.520623] [DEBUG] [VALIDATOR ] ITUB: Primary data validation passed. [2025-11-04T19:39:31.521569] [INFO ] [MAIN ] ITUB: Fetched primary data is valid. [2025-11-04T19:39:31.521620] [DEBUG] [LOCAL ] ITUB: Saving primary data. [2025-11-04T19:39:31.523001] [DEBUG] [LOCAL ] ITUB: Trimmed 77298 records to 53300 (keeping 200 days of data) [2025-11-04T19:39:31.835904] [DEBUG] [LOCAL ] ITUB: Primary data saved (53300 records). [2025-11-04T19:39:31.836065] [INFO ] [MAIN ] ITUB: Full history saved. [2025-11-04T19:39:31.836150] [INFO ] [MAIN ] Update Progress: 1134/2367 symbols completed. ITUB took 46.6s. Estimated remaining: 2h 53m 24s. Rate: 7.1 symbols/min [2025-11-04T19:39:32.008551] [INFO ] [MAIN ] IX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:39:32.039582] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T19:39:32.063349] [DEBUG] [LOCAL ] IX: Loaded 21306 existing records from primary storage. [2025-11-04T19:39:32.063563] [INFO ] [MAIN ] IX: Loaded 21306 existing records. [2025-11-04T19:39:32.169742] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-02. [2025-11-04T19:39:32.268751] [WARN ] [MAIN ] IX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:39:32.269047] [INFO ] [MAIN ] IX: Wiping existing primary data before full refresh. [2025-11-04T19:39:32.269122] [INFO ] [LOCAL ] IX: Starting deletion of primary data. [2025-11-04T19:39:32.269447] [DEBUG] [LOCAL ] IX: Deleted primary data file. [2025-11-04T19:39:32.269574] [INFO ] [MAIN ] IX: Starting full primary refresh. [2025-11-04T19:39:32.555681] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-11-04T19:39:32.674119] [INFO ] [REMOTE ] IX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:39:32.967043] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-09. [2025-11-04T19:39:33.517578] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T19:39:33.767019] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-01. [2025-11-04T19:39:33.875624] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-11-04T19:39:34.269565] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-11-04T19:39:34.567035] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-02. [2025-11-04T19:39:35.226566] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-11-04T19:39:35.367059] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-04. [2025-11-04T19:39:36.167082] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-03. [2025-11-04T19:39:36.809336] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T19:39:36.954430] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T19:39:36.966992] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-04. [2025-11-04T19:39:37.767094] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-10. [2025-11-04T19:39:37.991474] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T19:39:38.567006] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-05. [2025-11-04T19:39:38.605281] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-11-04T19:39:39.367031] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-05. [2025-11-04T19:39:39.698459] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T19:39:40.167025] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-06. [2025-11-04T19:39:40.269189] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T19:39:40.967033] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-03. [2025-11-04T19:39:41.529794] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T19:39:41.767055] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-11. [2025-11-04T19:39:42.278696] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-11-04T19:39:42.567082] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-04. [2025-11-04T19:39:42.700160] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T19:39:42.702179] [INFO ] [REMOTE ] IVT: Full fetch complete (45878 records). [2025-11-04T19:39:42.702269] [DEBUG] [VALIDATOR ] IVT: Validating primary data. [2025-11-04T19:39:43.083123] [DEBUG] [VALIDATOR ] IVT: All expected trading days are present. [2025-11-04T19:39:43.083196] [DEBUG] [VALIDATOR ] IVT: Primary data validation passed. [2025-11-04T19:39:43.083875] [INFO ] [MAIN ] IVT: Fetched primary data is valid. [2025-11-04T19:39:43.083937] [DEBUG] [LOCAL ] IVT: Saving primary data. [2025-11-04T19:39:43.084801] [DEBUG] [LOCAL ] IVT: Trimmed 45878 records to 31191 (keeping 200 days of data) [2025-11-04T19:39:43.264709] [DEBUG] [LOCAL ] IVT: Primary data saved (31191 records). [2025-11-04T19:39:43.264839] [INFO ] [MAIN ] IVT: Full history saved. [2025-11-04T19:39:43.264914] [INFO ] [MAIN ] Update Progress: 1135/2367 symbols completed. IVT took 28.5s. Estimated remaining: 2h 53m 19s. Rate: 7.1 symbols/min [2025-11-04T19:39:43.372184] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-06. [2025-11-04T19:39:43.434090] [INFO ] [MAIN ] J: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:39:43.471756] [DEBUG] [LOCAL ] J: Loaded 40644 existing records from primary storage. [2025-11-04T19:39:43.472050] [INFO ] [MAIN ] J: Loaded 40644 existing records. [2025-11-04T19:39:43.645661] [WARN ] [MAIN ] J: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:39:43.645862] [INFO ] [MAIN ] J: Wiping existing primary data before full refresh. [2025-11-04T19:39:43.645908] [INFO ] [LOCAL ] J: Starting deletion of primary data. [2025-11-04T19:39:43.646297] [DEBUG] [LOCAL ] J: Deleted primary data file. [2025-11-04T19:39:43.646353] [INFO ] [MAIN ] J: Starting full primary refresh. [2025-11-04T19:39:43.998684] [INFO ] [REMOTE ] J: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:39:44.153827] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T19:39:44.172118] [INFO ] [REMOTE ] J: Fetching primary data for 2025-01. [2025-11-04T19:39:44.194880] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-11-04T19:39:44.967035] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-05. [2025-11-04T19:39:45.319108] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-11-04T19:39:45.767051] [INFO ] [REMOTE ] J: Fetching primary data for 2025-02. [2025-11-04T19:39:46.283463] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-11-04T19:39:46.567052] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-07. [2025-11-04T19:39:46.929756] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T19:39:47.367021] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-06. [2025-11-04T19:39:47.400670] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-11-04T19:39:48.167074] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-07. [2025-11-04T19:39:48.805237] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-11-04T19:39:48.967062] [INFO ] [REMOTE ] J: Fetching primary data for 2025-03. [2025-11-04T19:39:49.528487] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T19:39:49.767049] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-08. [2025-11-04T19:39:50.172992] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-11-04T19:39:50.519782] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T19:39:50.567011] [INFO ] [REMOTE ] J: Fetching primary data for 2025-04. [2025-11-04T19:39:51.367042] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-07. [2025-11-04T19:39:51.979320] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-11-04T19:39:52.167081] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-09. [2025-11-04T19:39:52.737055] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-11-04T19:39:52.863960] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-11-04T19:39:52.967047] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-10. [2025-11-04T19:39:53.767063] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-08. [2025-11-04T19:39:53.862107] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-11-04T19:39:54.567034] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-08. [2025-11-04T19:39:55.154515] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T19:39:55.367060] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-09. [2025-11-04T19:39:55.860579] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T19:39:56.167096] [INFO ] [REMOTE ] J: Fetching primary data for 2025-05. [2025-11-04T19:39:56.863035] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T19:39:56.967002] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-10. [2025-11-04T19:39:57.417691] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-11-04T19:39:57.767069] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-09. [2025-11-04T19:39:58.566441] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-11-04T19:39:58.570726] [INFO ] [REMOTE ] J: Fetching primary data for 2025-06. [2025-11-04T19:39:58.929575] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T19:39:59.367030] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-10. [2025-11-04T19:39:59.652709] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T19:40:00.167039] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-11. [2025-11-04T19:40:00.967018] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-11. [2025-11-04T19:40:01.059730] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-11-04T19:40:01.418984] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T19:40:01.421281] [INFO ] [REMOTE ] IVZ: Full fetch complete (77090 records). [2025-11-04T19:40:01.421428] [DEBUG] [VALIDATOR ] IVZ: Validating primary data. [2025-11-04T19:40:01.767073] [INFO ] [REMOTE ] J: Fetching primary data for 2025-07. [2025-11-04T19:40:01.819284] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-11 (0.85 seconds). [2025-11-04T19:40:01.845439] [DEBUG] [VALIDATOR ] IVZ: All expected trading days are present. [2025-11-04T19:40:01.845590] [INFO ] [REMOTE ] IX: Full fetch complete (29416 records). [2025-11-04T19:40:01.845695] [DEBUG] [VALIDATOR ] IVZ: Primary data validation passed. [2025-11-04T19:40:01.846962] [INFO ] [MAIN ] IVZ: Fetched primary data is valid. [2025-11-04T19:40:01.847173] [DEBUG] [LOCAL ] IVZ: Saving primary data. [2025-11-04T19:40:01.847070] [DEBUG] [VALIDATOR ] IX: Validating primary data. [2025-11-04T19:40:01.941127] [DEBUG] [LOCAL ] IVZ: Trimmed 77090 records to 53123 (keeping 200 days of data) [2025-11-04T19:40:02.375194] [DEBUG] [VALIDATOR ] IX: All expected trading days are present. [2025-11-04T19:40:02.375379] [DEBUG] [VALIDATOR ] IX: Primary data validation passed. [2025-11-04T19:40:02.376436] [INFO ] [MAIN ] IX: Fetched primary data is valid. [2025-11-04T19:40:02.376528] [DEBUG] [LOCAL ] IX: Saving primary data. [2025-11-04T19:40:02.477968] [DEBUG] [LOCAL ] IX: Trimmed 29416 records to 21192 (keeping 200 days of data) [2025-11-04T19:40:02.572057] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-11. [2025-11-04T19:40:02.724807] [DEBUG] [LOCAL ] IX: Primary data saved (21192 records). [2025-11-04T19:40:02.734525] [DEBUG] [LOCAL ] IVZ: Primary data saved (53123 records). [2025-11-04T19:40:02.734672] [INFO ] [MAIN ] IVZ: Full history saved. [2025-11-04T19:40:02.734794] [INFO ] [MAIN ] Update Progress: 1136/2367 symbols completed. IVZ took 34.2s. Estimated remaining: 2h 53m 22s. Rate: 7.1 symbols/min [2025-11-04T19:40:02.734435] [INFO ] [MAIN ] IX: Full history saved. [2025-11-04T19:40:02.784917] [INFO ] [MAIN ] Update Progress: 1137/2367 symbols completed. IX took 30.9s. Estimated remaining: 2h 53m 5s. Rate: 7.1 symbols/min [2025-11-04T19:40:03.051452] [INFO ] [MAIN ] JBGS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:40:03.152121] [INFO ] [MAIN ] JBI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:40:03.177428] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T19:40:03.222903] [DEBUG] [LOCAL ] JBGS: Loaded 41125 existing records from primary storage. [2025-11-04T19:40:03.223217] [INFO ] [MAIN ] JBGS: Loaded 41125 existing records. [2025-11-04T19:40:03.347911] [DEBUG] [LOCAL ] JBI: Loaded 41776 existing records from primary storage. [2025-11-04T19:40:03.363736] [INFO ] [MAIN ] JBI: Loaded 41776 existing records. [2025-11-04T19:40:03.368975] [INFO ] [REMOTE ] J: Fetching primary data for 2025-08. [2025-11-04T19:40:03.428401] [WARN ] [MAIN ] JBGS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:40:03.429626] [INFO ] [MAIN ] JBGS: Wiping existing primary data before full refresh. [2025-11-04T19:40:03.429713] [INFO ] [LOCAL ] JBGS: Starting deletion of primary data. [2025-11-04T19:40:03.457015] [DEBUG] [LOCAL ] JBGS: Deleted primary data file. [2025-11-04T19:40:03.490504] [INFO ] [MAIN ] JBGS: Starting full primary refresh. [2025-11-04T19:40:03.777952] [WARN ] [MAIN ] JBI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:40:03.803388] [INFO ] [MAIN ] JBI: Wiping existing primary data before full refresh. [2025-11-04T19:40:03.842598] [INFO ] [LOCAL ] JBI: Starting deletion of primary data. [2025-11-04T19:40:03.891254] [DEBUG] [LOCAL ] JBI: Deleted primary data file. [2025-11-04T19:40:03.899570] [INFO ] [MAIN ] JBI: Starting full primary refresh. [2025-11-04T19:40:04.060545] [INFO ] [REMOTE ] JBGS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:40:04.099134] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-11 (1.53 seconds). [2025-11-04T19:40:04.136166] [INFO ] [REMOTE ] IVR: Full fetch complete (69094 records). [2025-11-04T19:40:04.147325] [DEBUG] [VALIDATOR ] IVR: Validating primary data. [2025-11-04T19:40:04.174004] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-01. [2025-11-04T19:40:04.538201] [INFO ] [REMOTE ] JBI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:40:04.785375] [DEBUG] [VALIDATOR ] IVR: All expected trading days are present. [2025-11-04T19:40:04.785587] [DEBUG] [VALIDATOR ] IVR: Primary data validation passed. [2025-11-04T19:40:04.786481] [INFO ] [MAIN ] IVR: Fetched primary data is valid. [2025-11-04T19:40:04.786562] [DEBUG] [LOCAL ] IVR: Saving primary data. [2025-11-04T19:40:04.810435] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-11-04T19:40:04.809329] [DEBUG] [LOCAL ] IVR: Trimmed 69094 records to 47520 (keeping 200 days of data) [2025-11-04T19:40:04.980490] [INFO ] [REMOTE ] J: Fetching primary data for 2025-09. [2025-11-04T19:40:05.112789] [DEBUG] [LOCAL ] IVR: Primary data saved (47520 records). [2025-11-04T19:40:05.113092] [INFO ] [MAIN ] IVR: Full history saved. [2025-11-04T19:40:05.113208] [INFO ] [MAIN ] Update Progress: 1138/2367 symbols completed. IVR took 52.7s. Estimated remaining: 2h 52m 50s. Rate: 7.1 symbols/min [2025-11-04T19:40:05.249186] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-11-04T19:40:05.312615] [INFO ] [MAIN ] JBL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:40:05.355824] [DEBUG] [LOCAL ] JBL: Loaded 48067 existing records from primary storage. [2025-11-04T19:40:05.356087] [INFO ] [MAIN ] JBL: Loaded 48067 existing records. [2025-11-04T19:40:05.542306] [WARN ] [MAIN ] JBL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:40:05.542574] [INFO ] [MAIN ] JBL: Wiping existing primary data before full refresh. [2025-11-04T19:40:05.542673] [INFO ] [LOCAL ] JBL: Starting deletion of primary data. [2025-11-04T19:40:05.543570] [DEBUG] [LOCAL ] JBL: Deleted primary data file. [2025-11-04T19:40:05.543637] [INFO ] [MAIN ] JBL: Starting full primary refresh. [2025-11-04T19:40:05.772194] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-01. [2025-11-04T19:40:05.921595] [INFO ] [REMOTE ] JBL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:40:06.271896] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T19:40:06.567065] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-02. [2025-11-04T19:40:07.151433] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T19:40:07.367010] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-01. [2025-11-04T19:40:07.560569] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-11-04T19:40:08.167076] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-02. [2025-11-04T19:40:08.730549] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T19:40:08.967146] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-02. [2025-11-04T19:40:09.128545] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T19:40:09.767050] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-03. [2025-11-04T19:40:10.260867] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-11-04T19:40:10.567026] [INFO ] [REMOTE ] J: Fetching primary data for 2025-10. [2025-11-04T19:40:11.003184] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-11-04T19:40:11.367010] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-03. [2025-11-04T19:40:11.933678] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T19:40:12.167043] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-03. [2025-11-04T19:40:12.910520] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-11-04T19:40:12.967009] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-04. [2025-11-04T19:40:13.243410] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-11-04T19:40:13.767044] [INFO ] [REMOTE ] J: Fetching primary data for 2025-11. [2025-11-04T19:40:14.377079] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-11-04T19:40:14.567035] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-04. [2025-11-04T19:40:14.791238] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T19:40:14.793498] [INFO ] [REMOTE ] J: Full fetch complete (60147 records). [2025-11-04T19:40:14.793694] [DEBUG] [VALIDATOR ] J: Validating primary data. [2025-11-04T19:40:15.189461] [DEBUG] [VALIDATOR ] J: All expected trading days are present. [2025-11-04T19:40:15.189542] [DEBUG] [VALIDATOR ] J: Primary data validation passed. [2025-11-04T19:40:15.190357] [INFO ] [MAIN ] J: Fetched primary data is valid. [2025-11-04T19:40:15.190420] [DEBUG] [LOCAL ] J: Saving primary data. [2025-11-04T19:40:15.191585] [DEBUG] [LOCAL ] J: Trimmed 60147 records to 40647 (keeping 200 days of data) [2025-11-04T19:40:15.367078] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-04. [2025-11-04T19:40:15.447113] [DEBUG] [LOCAL ] J: Primary data saved (40647 records). [2025-11-04T19:40:15.447241] [INFO ] [MAIN ] J: Full history saved. [2025-11-04T19:40:15.447315] [INFO ] [MAIN ] Update Progress: 1139/2367 symbols completed. J took 32.2s. Estimated remaining: 2h 52m 43s. Rate: 7.1 symbols/min [2025-11-04T19:40:15.616330] [INFO ] [MAIN ] JBND: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:40:15.641688] [DEBUG] [LOCAL ] JBND: Loaded 27965 existing records from primary storage. [2025-11-04T19:40:15.641947] [INFO ] [MAIN ] JBND: Loaded 27965 existing records. [2025-11-04T19:40:15.822314] [WARN ] [MAIN ] JBND: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:40:15.822696] [INFO ] [MAIN ] JBND: Wiping existing primary data before full refresh. [2025-11-04T19:40:15.822815] [INFO ] [LOCAL ] JBND: Starting deletion of primary data. [2025-11-04T19:40:15.823498] [DEBUG] [LOCAL ] JBND: Deleted primary data file. [2025-11-04T19:40:15.823563] [INFO ] [MAIN ] JBND: Starting full primary refresh. [2025-11-04T19:40:15.973854] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-11-04T19:40:16.177485] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-05. [2025-11-04T19:40:16.217472] [INFO ] [REMOTE ] JBND: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:40:16.629363] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T19:40:16.967060] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-01. [2025-11-04T19:40:17.592860] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-11-04T19:40:17.732617] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-11-04T19:40:17.767069] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-02. [2025-11-04T19:40:18.419953] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-11-04T19:40:18.567028] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-06. [2025-11-04T19:40:19.367040] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-03. [2025-11-04T19:40:20.020269] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-11-04T19:40:20.103185] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T19:40:20.167023] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-04. [2025-11-04T19:40:20.866701] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T19:40:20.967056] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-05. [2025-11-04T19:40:21.767074] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-05. [2025-11-04T19:40:22.128871] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T19:40:22.557320] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-11-04T19:40:22.567010] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-05. [2025-11-04T19:40:23.367090] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-06. [2025-11-04T19:40:23.758260] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-11-04T19:40:24.172497] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-07. [2025-11-04T19:40:24.179692] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-11-04T19:40:24.967045] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-06. [2025-11-04T19:40:25.691085] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-11-04T19:40:25.767023] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-08. [2025-11-04T19:40:26.128158] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-11-04T19:40:26.567034] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-07. [2025-11-04T19:40:27.054050] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-11-04T19:40:27.367010] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-06. [2025-11-04T19:40:27.385666] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-11-04T19:40:28.167037] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-08. [2025-11-04T19:40:28.472869] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T19:40:28.967021] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-09. [2025-11-04T19:40:29.057130] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-11-04T19:40:29.767033] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-07. [2025-11-04T19:40:30.567052] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-09. [2025-11-04T19:40:30.648427] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-11-04T19:40:30.763852] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T19:40:31.367010] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-08. [2025-11-04T19:40:31.477533] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-11-04T19:40:32.167084] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-10. [2025-11-04T19:40:32.436861] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-11-04T19:40:32.967083] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-09. [2025-11-04T19:40:33.614521] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-11-04T19:40:33.767037] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-07. [2025-11-04T19:40:33.967133] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-11-04T19:40:34.567024] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-10. [2025-11-04T19:40:34.809919] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T19:40:35.367057] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-11. [2025-11-04T19:40:35.648923] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-11-04T19:40:35.745039] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T19:40:35.747046] [INFO ] [REMOTE ] JBL: Full fetch complete (70426 records). [2025-11-04T19:40:35.747157] [DEBUG] [VALIDATOR ] JBL: Validating primary data. [2025-11-04T19:40:36.145703] [DEBUG] [VALIDATOR ] JBL: All expected trading days are present. [2025-11-04T19:40:36.145790] [DEBUG] [VALIDATOR ] JBL: Primary data validation passed. [2025-11-04T19:40:36.146760] [INFO ] [MAIN ] JBL: Fetched primary data is valid. [2025-11-04T19:40:36.146822] [DEBUG] [LOCAL ] JBL: Saving primary data. [2025-11-04T19:40:36.148142] [DEBUG] [LOCAL ] JBL: Trimmed 70426 records to 48051 (keeping 200 days of data) [2025-11-04T19:40:36.202918] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-08. [2025-11-04T19:40:36.447777] [DEBUG] [LOCAL ] JBL: Primary data saved (48051 records). [2025-11-04T19:40:36.447911] [INFO ] [MAIN ] JBL: Full history saved. [2025-11-04T19:40:36.447989] [INFO ] [MAIN ] Update Progress: 1140/2367 symbols completed. JBL took 31.3s. Estimated remaining: 2h 52m 48s. Rate: 7.1 symbols/min [2025-11-04T19:40:36.622895] [INFO ] [MAIN ] JBS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:40:36.623040] [DEBUG] [LOCAL ] JBS: No existing data file found in primary storage. [2025-11-04T19:40:36.623081] [WARN ] [MAIN ] JBS: No existing data found, performing full refresh instead. [2025-11-04T19:40:36.623193] [INFO ] [MAIN ] JBS: Wiping existing primary data before full refresh. [2025-11-04T19:40:36.623234] [INFO ] [LOCAL ] JBS: Starting deletion of primary data. [2025-11-04T19:40:36.623276] [DEBUG] [LOCAL ] JBS: No primary data file found to delete. [2025-11-04T19:40:36.623309] [INFO ] [MAIN ] JBS: Starting full primary refresh. [2025-11-04T19:40:36.978687] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-11. [2025-11-04T19:40:36.983947] [INFO ] [REMOTE ] JBS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:40:37.431323] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T19:40:37.767013] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-09. [2025-11-04T19:40:37.846576] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-11 (0.87 seconds). [2025-11-04T19:40:37.848275] [INFO ] [REMOTE ] JBGS: Full fetch complete (59167 records). [2025-11-04T19:40:37.848356] [DEBUG] [VALIDATOR ] JBGS: Validating primary data. [2025-11-04T19:40:38.242956] [DEBUG] [VALIDATOR ] JBGS: All expected trading days are present. [2025-11-04T19:40:38.243033] [DEBUG] [VALIDATOR ] JBGS: Primary data validation passed. [2025-11-04T19:40:38.243827] [INFO ] [MAIN ] JBGS: Fetched primary data is valid. [2025-11-04T19:40:38.243888] [DEBUG] [LOCAL ] JBGS: Saving primary data. [2025-11-04T19:40:38.245074] [DEBUG] [LOCAL ] JBGS: Trimmed 59167 records to 41043 (keeping 200 days of data) [2025-11-04T19:40:38.489584] [DEBUG] [LOCAL ] JBGS: Primary data saved (41043 records). [2025-11-04T19:40:38.489738] [INFO ] [MAIN ] JBGS: Full history saved. [2025-11-04T19:40:38.489822] [INFO ] [MAIN ] Update Progress: 1141/2367 symbols completed. JBGS took 35.8s. Estimated remaining: 2h 52m 33s. Rate: 7.1 symbols/min [2025-11-04T19:40:38.572199] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-10. [2025-11-04T19:40:38.669409] [INFO ] [MAIN ] JBTM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:40:38.700874] [DEBUG] [LOCAL ] JBTM: Loaded 32877 existing records from primary storage. [2025-11-04T19:40:38.701135] [INFO ] [MAIN ] JBTM: Loaded 32877 existing records. [2025-11-04T19:40:38.890943] [WARN ] [MAIN ] JBTM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:40:38.891225] [INFO ] [MAIN ] JBTM: Wiping existing primary data before full refresh. [2025-11-04T19:40:38.891353] [INFO ] [LOCAL ] JBTM: Starting deletion of primary data. [2025-11-04T19:40:38.899133] [DEBUG] [LOCAL ] JBTM: Deleted primary data file. [2025-11-04T19:40:38.911708] [INFO ] [MAIN ] JBTM: Starting full primary refresh. [2025-11-04T19:40:38.977341] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T19:40:39.279063] [INFO ] [REMOTE ] JBTM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:40:39.367009] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-10. [2025-11-04T19:40:39.767333] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T19:40:40.167057] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-01. [2025-11-04T19:40:40.307317] [DEBUG] [REMOTE ] JBS: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T19:40:40.307674] [WARN ] [REMOTE ] JBS: No data returned for month 2025-01 [2025-11-04T19:40:40.307757] [FATAL] [REMOTE ] JBS: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:40:40.307834] [FATAL] [MAIN ] JBS: Failed to fetch primary data. [2025-11-04T19:40:40.307898] [WARN ] [MAIN ] JBS: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:40:40.307976] [INFO ] [LOCAL ] JBS: Starting deletion of primary data. [2025-11-04T19:40:40.308038] [DEBUG] [LOCAL ] JBS: No primary data file found to delete. [2025-11-04T19:40:40.308073] [INFO ] [LOCAL ] JBS: Starting deletion of derivative data. [2025-11-04T19:40:40.308111] [DEBUG] [LOCAL ] JBS: No derivative data file found to delete. [2025-11-04T19:40:40.308177] [INFO ] [MAIN ] Update Progress: 1142/2367 symbols completed. JBS took 3.9s. Estimated remaining: 2h 52m 17s. Rate: 7.1 symbols/min [2025-11-04T19:40:40.476415] [INFO ] [MAIN ] JCE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:40:40.485855] [DEBUG] [LOCAL ] JCE: Loaded 7143 existing records from primary storage. [2025-11-04T19:40:40.486087] [INFO ] [MAIN ] JCE: Loaded 7143 existing records. [2025-11-04T19:40:40.669972] [WARN ] [MAIN ] JCE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:40:40.670361] [INFO ] [MAIN ] JCE: Wiping existing primary data before full refresh. [2025-11-04T19:40:40.670441] [INFO ] [LOCAL ] JCE: Starting deletion of primary data. [2025-11-04T19:40:40.670683] [DEBUG] [LOCAL ] JCE: Deleted primary data file. [2025-11-04T19:40:40.670744] [INFO ] [MAIN ] JCE: Starting full primary refresh. [2025-11-04T19:40:40.798706] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T19:40:40.972312] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-01. [2025-11-04T19:40:41.064325] [INFO ] [REMOTE ] JCE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:40:41.767021] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-01. [2025-11-04T19:40:41.875024] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-11-04T19:40:42.322107] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T19:40:42.567035] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-11. [2025-11-04T19:40:43.367042] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-02. [2025-11-04T19:40:43.520019] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-11 (0.95 seconds). [2025-11-04T19:40:43.521574] [INFO ] [REMOTE ] JBND: Full fetch complete (37354 records). [2025-11-04T19:40:43.521708] [DEBUG] [VALIDATOR ] JBND: Validating primary data. [2025-11-04T19:40:43.814330] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-11-04T19:40:43.901233] [DEBUG] [VALIDATOR ] JBND: All expected trading days are present. [2025-11-04T19:40:43.901314] [DEBUG] [VALIDATOR ] JBND: Primary data validation passed. [2025-11-04T19:40:43.902047] [INFO ] [MAIN ] JBND: Fetched primary data is valid. [2025-11-04T19:40:43.902112] [DEBUG] [LOCAL ] JBND: Saving primary data. [2025-11-04T19:40:43.903196] [DEBUG] [LOCAL ] JBND: Trimmed 37354 records to 28045 (keeping 200 days of data) [2025-11-04T19:40:44.077027] [DEBUG] [LOCAL ] JBND: Primary data saved (28045 records). [2025-11-04T19:40:44.077254] [INFO ] [MAIN ] JBND: Full history saved. [2025-11-04T19:40:44.077357] [INFO ] [MAIN ] Update Progress: 1143/2367 symbols completed. JBND took 28.6s. Estimated remaining: 2h 52m 4s. Rate: 7.1 symbols/min [2025-11-04T19:40:44.174939] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-11. [2025-11-04T19:40:44.248430] [INFO ] [MAIN ] JCI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:40:44.299186] [DEBUG] [LOCAL ] JCI: Loaded 53505 existing records from primary storage. [2025-11-04T19:40:44.299531] [INFO ] [MAIN ] JCI: Loaded 53505 existing records. [2025-11-04T19:40:44.473817] [WARN ] [MAIN ] JCI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:40:44.474076] [INFO ] [MAIN ] JCI: Wiping existing primary data before full refresh. [2025-11-04T19:40:44.474123] [INFO ] [LOCAL ] JCI: Starting deletion of primary data. [2025-11-04T19:40:44.474345] [DEBUG] [LOCAL ] JCI: Deleted primary data file. [2025-11-04T19:40:44.474400] [INFO ] [MAIN ] JCI: Starting full primary refresh. [2025-11-04T19:40:44.821158] [INFO ] [REMOTE ] JCI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:40:44.967072] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-03. [2025-11-04T19:40:45.233815] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T19:40:45.235528] [INFO ] [REMOTE ] JBI: Full fetch complete (62824 records). [2025-11-04T19:40:45.235617] [DEBUG] [VALIDATOR ] JBI: Validating primary data. [2025-11-04T19:40:45.432633] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-11-04T19:40:45.625422] [DEBUG] [VALIDATOR ] JBI: All expected trading days are present. [2025-11-04T19:40:45.625501] [DEBUG] [VALIDATOR ] JBI: Primary data validation passed. [2025-11-04T19:40:45.626318] [INFO ] [MAIN ] JBI: Fetched primary data is valid. [2025-11-04T19:40:45.626382] [DEBUG] [LOCAL ] JBI: Saving primary data. [2025-11-04T19:40:45.627603] [DEBUG] [LOCAL ] JBI: Trimmed 62824 records to 41706 (keeping 200 days of data) [2025-11-04T19:40:45.788381] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-02. [2025-11-04T19:40:45.868806] [DEBUG] [LOCAL ] JBI: Primary data saved (41706 records). [2025-11-04T19:40:45.868922] [INFO ] [MAIN ] JBI: Full history saved. [2025-11-04T19:40:45.869784] [INFO ] [MAIN ] Update Progress: 1144/2367 symbols completed. JBI took 43.1s. Estimated remaining: 2h 51m 48s. Rate: 7.1 symbols/min [2025-11-04T19:40:46.053747] [INFO ] [MAIN ] JEF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:40:46.100950] [DEBUG] [LOCAL ] JEF: Loaded 50149 existing records from primary storage. [2025-11-04T19:40:46.101284] [INFO ] [MAIN ] JEF: Loaded 50149 existing records. [2025-11-04T19:40:46.296120] [WARN ] [MAIN ] JEF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:40:46.296377] [INFO ] [MAIN ] JEF: Wiping existing primary data before full refresh. [2025-11-04T19:40:46.296424] [INFO ] [LOCAL ] JEF: Starting deletion of primary data. [2025-11-04T19:40:46.297356] [DEBUG] [LOCAL ] JEF: Deleted primary data file. [2025-11-04T19:40:46.297412] [INFO ] [MAIN ] JEF: Starting full primary refresh. [2025-11-04T19:40:46.568788] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-04. [2025-11-04T19:40:46.646526] [INFO ] [REMOTE ] JEF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:40:46.885725] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-11-04T19:40:47.100502] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T19:40:47.367054] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-01. [2025-11-04T19:40:48.167106] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-05. [2025-11-04T19:40:48.631258] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-11-04T19:40:48.631410] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-11-04T19:40:48.967049] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-03. [2025-11-04T19:40:49.767079] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-01. [2025-11-04T19:40:49.991209] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-11-04T19:40:50.567032] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-06. [2025-11-04T19:40:50.970791] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T19:40:51.076690] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-11-04T19:40:51.367041] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-02. [2025-11-04T19:40:52.167140] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-02. [2025-11-04T19:40:52.772627] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-11-04T19:40:52.967044] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-04. [2025-11-04T19:40:53.566673] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-11-04T19:40:53.767053] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-03. [2025-11-04T19:40:54.103885] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-11-04T19:40:54.567074] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-05. [2025-11-04T19:40:55.224856] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T19:40:55.367041] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-07. [2025-11-04T19:40:55.601007] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-11-04T19:40:55.894931] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-11-04T19:40:56.167086] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-06. [2025-11-04T19:40:56.958312] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T19:40:56.967004] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-03. [2025-11-04T19:40:57.767080] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-07. [2025-11-04T19:40:58.228659] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-11-04T19:40:58.567026] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-04. [2025-11-04T19:40:58.747775] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-11-04T19:40:59.367048] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-04. [2025-11-04T19:41:00.033564] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-11-04T19:41:00.167051] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-08. [2025-11-04T19:41:00.975828] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-08. [2025-11-04T19:41:01.001459] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-11-04T19:41:01.267833] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-11-04T19:41:01.487423] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-11-04T19:41:01.767049] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-09. [2025-11-04T19:41:02.567048] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-05. [2025-11-04T19:41:02.699873] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-11-04T19:41:03.367053] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-09. [2025-11-04T19:41:03.902600] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T19:41:03.920414] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-11-04T19:41:04.167044] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-10. [2025-11-04T19:41:04.967032] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-05. [2025-11-04T19:41:05.200258] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-11-04T19:41:05.767036] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-11. [2025-11-04T19:41:06.418024] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T19:41:06.567014] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-06. [2025-11-04T19:41:06.884786] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-11 (1.12 seconds). [2025-11-04T19:41:06.886354] [INFO ] [REMOTE ] JBTM: Full fetch complete (47167 records). [2025-11-04T19:41:06.886431] [DEBUG] [VALIDATOR ] JBTM: Validating primary data. [2025-11-04T19:41:07.282419] [DEBUG] [VALIDATOR ] JBTM: All expected trading days are present. [2025-11-04T19:41:07.282512] [DEBUG] [VALIDATOR ] JBTM: Primary data validation passed. [2025-11-04T19:41:07.283225] [INFO ] [MAIN ] JBTM: Fetched primary data is valid. [2025-11-04T19:41:07.283287] [DEBUG] [LOCAL ] JBTM: Saving primary data. [2025-11-04T19:41:07.284296] [DEBUG] [LOCAL ] JBTM: Trimmed 47167 records to 32966 (keeping 200 days of data) [2025-11-04T19:41:07.375240] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-10. [2025-11-04T19:41:07.493189] [DEBUG] [LOCAL ] JBTM: Primary data saved (32966 records). [2025-11-04T19:41:07.493335] [INFO ] [MAIN ] JBTM: Full history saved. [2025-11-04T19:41:07.493425] [INFO ] [MAIN ] Update Progress: 1145/2367 symbols completed. JBTM took 29.0s. Estimated remaining: 2h 51m 54s. Rate: 7.1 symbols/min [2025-11-04T19:41:07.662422] [INFO ] [MAIN ] JELD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:41:07.701810] [DEBUG] [LOCAL ] JELD: Loaded 45310 existing records from primary storage. [2025-11-04T19:41:07.702091] [INFO ] [MAIN ] JELD: Loaded 45310 existing records. [2025-11-04T19:41:07.884025] [WARN ] [MAIN ] JELD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:41:07.884406] [INFO ] [MAIN ] JELD: Wiping existing primary data before full refresh. [2025-11-04T19:41:07.884621] [INFO ] [LOCAL ] JELD: Starting deletion of primary data. [2025-11-04T19:41:07.885002] [DEBUG] [LOCAL ] JELD: Deleted primary data file. [2025-11-04T19:41:07.885061] [INFO ] [MAIN ] JELD: Starting full primary refresh. [2025-11-04T19:41:07.990789] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-11-04T19:41:08.172031] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-06. [2025-11-04T19:41:08.256525] [INFO ] [REMOTE ] JELD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:41:08.288367] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-11-04T19:41:08.967032] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-01. [2025-11-04T19:41:09.578911] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-11-04T19:41:09.767022] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-07. [2025-11-04T19:41:10.162424] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T19:41:10.567062] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-11. [2025-11-04T19:41:11.115853] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T19:41:11.117293] [INFO ] [REMOTE ] JCE: Full fetch complete (9570 records). [2025-11-04T19:41:11.117433] [DEBUG] [VALIDATOR ] JCE: Validating primary data. [2025-11-04T19:41:11.275364] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-11-04T19:41:11.372187] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-07. [2025-11-04T19:41:11.642888] [DEBUG] [VALIDATOR ] JCE: All expected trading days are present. [2025-11-04T19:41:11.642967] [DEBUG] [VALIDATOR ] JCE: Primary data validation passed. [2025-11-04T19:41:11.643218] [INFO ] [MAIN ] JCE: Fetched primary data is valid. [2025-11-04T19:41:11.643257] [DEBUG] [LOCAL ] JCE: Saving primary data. [2025-11-04T19:41:11.643795] [DEBUG] [LOCAL ] JCE: Trimmed 9570 records to 7143 (keeping 200 days of data) [2025-11-04T19:41:11.687495] [DEBUG] [LOCAL ] JCE: Primary data saved (7143 records). [2025-11-04T19:41:11.687639] [INFO ] [MAIN ] JCE: Full history saved. [2025-11-04T19:41:11.687762] [INFO ] [MAIN ] Update Progress: 1146/2367 symbols completed. JCE took 31.4s. Estimated remaining: 2h 51m 41s. Rate: 7.1 symbols/min [2025-11-04T19:41:11.877035] [INFO ] [MAIN ] JFR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:41:11.906278] [DEBUG] [LOCAL ] JFR: Loaded 30839 existing records from primary storage. [2025-11-04T19:41:11.906583] [INFO ] [MAIN ] JFR: Loaded 30839 existing records. [2025-11-04T19:41:12.088447] [WARN ] [MAIN ] JFR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:41:12.088706] [INFO ] [MAIN ] JFR: Wiping existing primary data before full refresh. [2025-11-04T19:41:12.088762] [INFO ] [LOCAL ] JFR: Starting deletion of primary data. [2025-11-04T19:41:12.088958] [DEBUG] [LOCAL ] JFR: Deleted primary data file. [2025-11-04T19:41:12.089014] [INFO ] [MAIN ] JFR: Starting full primary refresh. [2025-11-04T19:41:12.172260] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-08. [2025-11-04T19:41:12.444718] [INFO ] [REMOTE ] JFR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:41:12.645813] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T19:41:12.967093] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-01. [2025-11-04T19:41:13.577545] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-11-04T19:41:13.767088] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-09. [2025-11-04T19:41:14.251003] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-11-04T19:41:14.567035] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-08. [2025-11-04T19:41:15.184720] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T19:41:15.367050] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-10. [2025-11-04T19:41:15.726071] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T19:41:16.167060] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-02. [2025-11-04T19:41:16.967020] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-02. [2025-11-04T19:41:17.066211] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-11-04T19:41:17.186472] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T19:41:17.767047] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-11. [2025-11-04T19:41:17.979216] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-11-04T19:41:18.111937] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T19:41:18.114017] [INFO ] [REMOTE ] JCI: Full fetch complete (77654 records). [2025-11-04T19:41:18.114138] [DEBUG] [VALIDATOR ] JCI: Validating primary data. [2025-11-04T19:41:18.541583] [DEBUG] [VALIDATOR ] JCI: All expected trading days are present. [2025-11-04T19:41:18.541701] [DEBUG] [VALIDATOR ] JCI: Primary data validation passed. [2025-11-04T19:41:18.542663] [INFO ] [MAIN ] JCI: Fetched primary data is valid. [2025-11-04T19:41:18.542730] [DEBUG] [LOCAL ] JCI: Saving primary data. [2025-11-04T19:41:18.544207] [DEBUG] [LOCAL ] JCI: Trimmed 77654 records to 53505 (keeping 200 days of data) [2025-11-04T19:41:18.600930] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-09. [2025-11-04T19:41:18.887450] [DEBUG] [LOCAL ] JCI: Primary data saved (53505 records). [2025-11-04T19:41:18.887585] [INFO ] [MAIN ] JCI: Full history saved. [2025-11-04T19:41:18.887682] [INFO ] [MAIN ] Update Progress: 1147/2367 symbols completed. JCI took 34.8s. Estimated remaining: 2h 51m 31s. Rate: 7.1 symbols/min [2025-11-04T19:41:19.055637] [INFO ] [MAIN ] JGH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:41:19.069352] [DEBUG] [LOCAL ] JGH: Loaded 12320 existing records from primary storage. [2025-11-04T19:41:19.069566] [INFO ] [MAIN ] JGH: Loaded 12320 existing records. [2025-11-04T19:41:19.244170] [WARN ] [MAIN ] JGH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:41:19.244388] [INFO ] [MAIN ] JGH: Wiping existing primary data before full refresh. [2025-11-04T19:41:19.244437] [INFO ] [LOCAL ] JGH: Starting deletion of primary data. [2025-11-04T19:41:19.244589] [DEBUG] [LOCAL ] JGH: Deleted primary data file. [2025-11-04T19:41:19.244627] [INFO ] [MAIN ] JGH: Starting full primary refresh. [2025-11-04T19:41:19.372135] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-03. [2025-11-04T19:41:19.625580] [INFO ] [REMOTE ] JGH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:41:20.005740] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-11-04T19:41:20.167097] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-01. [2025-11-04T19:41:20.384037] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-11-04T19:41:20.886554] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-11-04T19:41:20.967045] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-02. [2025-11-04T19:41:21.365843] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-11-04T19:41:21.767083] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-04. [2025-11-04T19:41:22.567041] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-03. [2025-11-04T19:41:22.836072] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-11-04T19:41:23.367069] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-05. [2025-11-04T19:41:23.743892] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-11-04T19:41:24.167094] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-04. [2025-11-04T19:41:24.253800] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-11-04T19:41:24.967070] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-06. [2025-11-04T19:41:25.502280] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T19:41:25.767054] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-03. [2025-11-04T19:41:25.890748] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-11-04T19:41:26.339212] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-11-04T19:41:26.567045] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-10. [2025-11-04T19:41:27.367067] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-07. [2025-11-04T19:41:28.082666] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-11-04T19:41:28.167038] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-11. [2025-11-04T19:41:28.508520] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T19:41:28.967037] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-08. [2025-11-04T19:41:29.517527] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-11 (1.35 seconds). [2025-11-04T19:41:29.519487] [INFO ] [REMOTE ] JEF: Full fetch complete (73278 records). [2025-11-04T19:41:29.519582] [DEBUG] [VALIDATOR ] JEF: Validating primary data. [2025-11-04T19:41:29.775360] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-04. [2025-11-04T19:41:29.930725] [DEBUG] [VALIDATOR ] JEF: All expected trading days are present. [2025-11-04T19:41:29.931235] [DEBUG] [VALIDATOR ] JEF: Primary data validation passed. [2025-11-04T19:41:29.931869] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-11-04T19:41:29.932811] [INFO ] [MAIN ] JEF: Fetched primary data is valid. [2025-11-04T19:41:29.938981] [DEBUG] [LOCAL ] JEF: Saving primary data. [2025-11-04T19:41:29.940438] [DEBUG] [LOCAL ] JEF: Trimmed 73278 records to 50169 (keeping 200 days of data) [2025-11-04T19:41:30.244503] [DEBUG] [LOCAL ] JEF: Primary data saved (50169 records). [2025-11-04T19:41:30.244632] [INFO ] [MAIN ] JEF: Full history saved. [2025-11-04T19:41:30.244755] [INFO ] [MAIN ] Update Progress: 1148/2367 symbols completed. JEF took 44.4s. Estimated remaining: 2h 51m 26s. Rate: 7.1 symbols/min [2025-11-04T19:41:30.426495] [INFO ] [MAIN ] JHG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:41:30.468625] [DEBUG] [LOCAL ] JHG: Loaded 47455 existing records from primary storage. [2025-11-04T19:41:30.468965] [INFO ] [MAIN ] JHG: Loaded 47455 existing records. [2025-11-04T19:41:30.582380] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-11-04T19:41:30.569723] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-09. [2025-11-04T19:41:30.664886] [WARN ] [MAIN ] JHG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:41:30.665105] [INFO ] [MAIN ] JHG: Wiping existing primary data before full refresh. [2025-11-04T19:41:30.665153] [INFO ] [LOCAL ] JHG: Starting deletion of primary data. [2025-11-04T19:41:30.665351] [DEBUG] [LOCAL ] JHG: Deleted primary data file. [2025-11-04T19:41:30.665407] [INFO ] [MAIN ] JHG: Starting full primary refresh. [2025-11-04T19:41:31.022094] [INFO ] [REMOTE ] JHG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:41:31.367105] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-05. [2025-11-04T19:41:31.728026] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-11-04T19:41:32.167076] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-05. [2025-11-04T19:41:32.511506] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T19:41:32.562629] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-11-04T19:41:32.967028] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-06. [2025-11-04T19:41:33.767088] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-01. [2025-11-04T19:41:34.007426] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T19:41:34.567040] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-06. [2025-11-04T19:41:34.989911] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-11-04T19:41:35.138782] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-11-04T19:41:35.367057] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-10. [2025-11-04T19:41:36.167083] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-07. [2025-11-04T19:41:36.548230] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T19:41:36.612521] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-11-04T19:41:36.967080] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-11. [2025-11-04T19:41:37.767037] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-08. [2025-11-04T19:41:37.956051] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-11 (0.99 seconds). [2025-11-04T19:41:37.957595] [INFO ] [REMOTE ] JFR: Full fetch complete (48270 records). [2025-11-04T19:41:37.957699] [DEBUG] [VALIDATOR ] JFR: Validating primary data. [2025-11-04T19:41:38.355427] [DEBUG] [VALIDATOR ] JFR: All expected trading days are present. [2025-11-04T19:41:38.356001] [DEBUG] [VALIDATOR ] JFR: Primary data validation passed. [2025-11-04T19:41:38.356254] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-11-04T19:41:38.357047] [INFO ] [MAIN ] JFR: Fetched primary data is valid. [2025-11-04T19:41:38.360158] [DEBUG] [LOCAL ] JFR: Saving primary data. [2025-11-04T19:41:38.361048] [DEBUG] [LOCAL ] JFR: Trimmed 48270 records to 30948 (keeping 200 days of data) [2025-11-04T19:41:38.537760] [DEBUG] [LOCAL ] JFR: Primary data saved (30948 records). [2025-11-04T19:41:38.537900] [INFO ] [MAIN ] JFR: Full history saved. [2025-11-04T19:41:38.537980] [INFO ] [MAIN ] Update Progress: 1149/2367 symbols completed. JFR took 26.9s. Estimated remaining: 2h 51m 17s. Rate: 7.1 symbols/min [2025-11-04T19:41:38.572130] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-07. [2025-11-04T19:41:38.706399] [INFO ] [MAIN ] JHI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:41:38.712558] [DEBUG] [LOCAL ] JHI: Loaded 4031 existing records from primary storage. [2025-11-04T19:41:38.712754] [INFO ] [MAIN ] JHI: Loaded 4031 existing records. [2025-11-04T19:41:38.888835] [WARN ] [MAIN ] JHI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:41:38.889045] [INFO ] [MAIN ] JHI: Wiping existing primary data before full refresh. [2025-11-04T19:41:38.889092] [INFO ] [LOCAL ] JHI: Starting deletion of primary data. [2025-11-04T19:41:38.889218] [DEBUG] [LOCAL ] JHI: Deleted primary data file. [2025-11-04T19:41:38.889255] [INFO ] [MAIN ] JHI: Starting full primary refresh. [2025-11-04T19:41:39.251355] [INFO ] [REMOTE ] JHI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:41:39.367037] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-09. [2025-11-04T19:41:39.818304] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T19:41:39.925588] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-11-04T19:41:40.167047] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-08. [2025-11-04T19:41:40.967044] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-10. [2025-11-04T19:41:41.309498] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T19:41:41.685100] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-11-04T19:41:41.767051] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-11. [2025-11-04T19:41:42.561194] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-11 (0.79 seconds). [2025-11-04T19:41:42.562213] [INFO ] [REMOTE ] JGH: Full fetch complete (16660 records). [2025-11-04T19:41:42.562281] [DEBUG] [VALIDATOR ] JGH: Validating primary data. [2025-11-04T19:41:42.573434] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-02. [2025-11-04T19:41:42.937796] [DEBUG] [VALIDATOR ] JGH: All expected trading days are present. [2025-11-04T19:41:42.937872] [DEBUG] [VALIDATOR ] JGH: Primary data validation passed. [2025-11-04T19:41:42.938298] [INFO ] [MAIN ] JGH: Fetched primary data is valid. [2025-11-04T19:41:42.938359] [DEBUG] [LOCAL ] JGH: Saving primary data. [2025-11-04T19:41:42.938906] [DEBUG] [LOCAL ] JGH: Trimmed 16660 records to 12388 (keeping 200 days of data) [2025-11-04T19:41:43.012450] [DEBUG] [LOCAL ] JGH: Primary data saved (12388 records). [2025-11-04T19:41:43.012573] [INFO ] [MAIN ] JGH: Full history saved. [2025-11-04T19:41:43.012668] [INFO ] [MAIN ] Update Progress: 1150/2367 symbols completed. JGH took 24.1s. Estimated remaining: 2h 51m 5s. Rate: 7.1 symbols/min [2025-11-04T19:41:43.189638] [INFO ] [MAIN ] JHS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:41:43.193825] [DEBUG] [LOCAL ] JHS: Loaded 2956 existing records from primary storage. [2025-11-04T19:41:43.193974] [INFO ] [MAIN ] JHS: Loaded 2956 existing records. [2025-11-04T19:41:43.366070] [WARN ] [MAIN ] JHS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:41:43.366302] [INFO ] [MAIN ] JHS: Wiping existing primary data before full refresh. [2025-11-04T19:41:43.366350] [INFO ] [LOCAL ] JHS: Starting deletion of primary data. [2025-11-04T19:41:43.366517] [DEBUG] [LOCAL ] JHS: Deleted primary data file. [2025-11-04T19:41:43.366555] [INFO ] [MAIN ] JHS: Starting full primary refresh. [2025-11-04T19:41:43.377570] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-09. [2025-11-04T19:41:43.726185] [INFO ] [REMOTE ] JHS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:41:43.889548] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-11-04T19:41:44.167066] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-01. [2025-11-04T19:41:44.671762] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T19:41:44.815033] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-11-04T19:41:44.967038] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-02. [2025-11-04T19:41:45.361586] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-11-04T19:41:45.767041] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-03. [2025-11-04T19:41:46.110637] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-11-04T19:41:46.567039] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-04. [2025-11-04T19:41:46.890949] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-04 (0.32 seconds). [2025-11-04T19:41:47.367049] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-10. [2025-11-04T19:41:48.167068] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-01. [2025-11-04T19:41:48.413706] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-11-04T19:41:48.874634] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T19:41:48.967067] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-02. [2025-11-04T19:41:49.314632] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-11-04T19:41:49.767051] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-05. [2025-11-04T19:41:50.070346] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-11-04T19:41:50.567086] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-03. [2025-11-04T19:41:50.959727] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-11-04T19:41:51.367068] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-11. [2025-11-04T19:41:52.167040] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-03. [2025-11-04T19:41:52.405835] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T19:41:52.407808] [INFO ] [REMOTE ] JELD: Full fetch complete (64725 records). [2025-11-04T19:41:52.407896] [DEBUG] [VALIDATOR ] JELD: Validating primary data. [2025-11-04T19:41:52.816721] [DEBUG] [VALIDATOR ] JELD: All expected trading days are present. [2025-11-04T19:41:52.816833] [DEBUG] [VALIDATOR ] JELD: Primary data validation passed. [2025-11-04T19:41:52.817811] [INFO ] [MAIN ] JELD: Fetched primary data is valid. [2025-11-04T19:41:52.817885] [DEBUG] [LOCAL ] JELD: Saving primary data. [2025-11-04T19:41:52.819129] [DEBUG] [LOCAL ] JELD: Trimmed 64725 records to 45355 (keeping 200 days of data) [2025-11-04T19:41:52.970676] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-06. [2025-11-04T19:41:53.071271] [DEBUG] [LOCAL ] JELD: Primary data saved (45355 records). [2025-11-04T19:41:53.071409] [INFO ] [MAIN ] JELD: Full history saved. [2025-11-04T19:41:53.071489] [INFO ] [MAIN ] Update Progress: 1151/2367 symbols completed. JELD took 45.6s. Estimated remaining: 2h 50m 58s. Rate: 7.1 symbols/min [2025-11-04T19:41:53.250229] [INFO ] [MAIN ] JHX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:41:53.296528] [DEBUG] [LOCAL ] JHX: Loaded 53217 existing records from primary storage. [2025-11-04T19:41:53.296920] [INFO ] [MAIN ] JHX: Loaded 53217 existing records. [2025-11-04T19:41:53.402699] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-11-04T19:41:53.484097] [WARN ] [MAIN ] JHX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:41:53.484399] [INFO ] [MAIN ] JHX: Wiping existing primary data before full refresh. [2025-11-04T19:41:53.484480] [INFO ] [LOCAL ] JHX: Starting deletion of primary data. [2025-11-04T19:41:53.485381] [DEBUG] [LOCAL ] JHX: Deleted primary data file. [2025-11-04T19:41:53.485441] [INFO ] [MAIN ] JHX: Starting full primary refresh. [2025-11-04T19:41:53.616405] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-11-04T19:41:53.772277] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-04. [2025-11-04T19:41:53.877884] [INFO ] [REMOTE ] JHX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:41:54.567075] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-04. [2025-11-04T19:41:55.035059] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-11-04T19:41:55.093942] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T19:41:55.367044] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-05. [2025-11-04T19:41:55.644003] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-11-04T19:41:56.167080] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-05. [2025-11-04T19:41:56.967045] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-07. [2025-11-04T19:41:57.324077] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-11-04T19:41:57.501320] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-11-04T19:41:57.767042] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-06. [2025-11-04T19:41:58.106035] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-11-04T19:41:58.567045] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-08. [2025-11-04T19:41:58.879274] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-08 (0.31 seconds). [2025-11-04T19:41:59.367043] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-07. [2025-11-04T19:41:59.715451] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-11-04T19:42:00.167060] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-08. [2025-11-04T19:42:00.751049] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-11-04T19:42:00.967030] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-09. [2025-11-04T19:42:01.361709] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-11-04T19:42:01.767069] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-06. [2025-11-04T19:42:02.567072] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-09. [2025-11-04T19:42:02.969351] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-11-04T19:42:03.092466] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T19:42:03.367045] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-10. [2025-11-04T19:42:03.862177] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-11-04T19:42:04.167098] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-07. [2025-11-04T19:42:04.967051] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-11. [2025-11-04T19:42:05.380111] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-11-04T19:42:05.400695] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T19:42:05.401369] [INFO ] [REMOTE ] JHI: Full fetch complete (5887 records). [2025-11-04T19:42:05.401434] [DEBUG] [VALIDATOR ] JHI: Validating primary data. [2025-11-04T19:42:05.750435] [DEBUG] [VALIDATOR ] JHI: All expected trading days are present. [2025-11-04T19:42:05.750509] [DEBUG] [VALIDATOR ] JHI: Primary data validation passed. [2025-11-04T19:42:05.750691] [INFO ] [MAIN ] JHI: Fetched primary data is valid. [2025-11-04T19:42:05.750751] [DEBUG] [LOCAL ] JHI: Saving primary data. [2025-11-04T19:42:05.751124] [DEBUG] [LOCAL ] JHI: Trimmed 5887 records to 4031 (keeping 200 days of data) [2025-11-04T19:42:05.768943] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-01. [2025-11-04T19:42:05.776792] [DEBUG] [LOCAL ] JHI: Primary data saved (4031 records). [2025-11-04T19:42:05.777005] [INFO ] [MAIN ] JHI: Full history saved. [2025-11-04T19:42:05.777093] [INFO ] [MAIN ] Update Progress: 1152/2367 symbols completed. JHI took 27.2s. Estimated remaining: 2h 50m 54s. Rate: 7.1 symbols/min [2025-11-04T19:42:05.955633] [INFO ] [MAIN ] JILL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:42:05.972286] [DEBUG] [LOCAL ] JILL: Loaded 14808 existing records from primary storage. [2025-11-04T19:42:05.972538] [INFO ] [MAIN ] JILL: Loaded 14808 existing records. [2025-11-04T19:42:06.153601] [WARN ] [MAIN ] JILL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:42:06.153856] [INFO ] [MAIN ] JILL: Wiping existing primary data before full refresh. [2025-11-04T19:42:06.153913] [INFO ] [LOCAL ] JILL: Starting deletion of primary data. [2025-11-04T19:42:06.154123] [DEBUG] [LOCAL ] JILL: Deleted primary data file. [2025-11-04T19:42:06.154178] [INFO ] [MAIN ] JILL: Starting full primary refresh. [2025-11-04T19:42:06.574709] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-08. [2025-11-04T19:42:06.598983] [INFO ] [REMOTE ] JILL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:42:06.610593] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-11-04T19:42:07.367063] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-10. [2025-11-04T19:42:07.793735] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-11-04T19:42:07.842125] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T19:42:08.167071] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-01. [2025-11-04T19:42:08.907072] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-11-04T19:42:08.967045] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-02. [2025-11-04T19:42:09.600449] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-11-04T19:42:09.767068] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-03. [2025-11-04T19:42:10.475896] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T19:42:10.567045] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-04. [2025-11-04T19:42:11.340136] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-11-04T19:42:11.367004] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-05. [2025-11-04T19:42:11.991084] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-11-04T19:42:12.167065] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-11. [2025-11-04T19:42:12.537994] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-11 (0.37 seconds). [2025-11-04T19:42:12.538929] [INFO ] [REMOTE ] JHS: Full fetch complete (4611 records). [2025-11-04T19:42:12.539020] [DEBUG] [VALIDATOR ] JHS: Validating primary data. [2025-11-04T19:42:12.971270] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-06. [2025-11-04T19:42:12.977530] [DEBUG] [VALIDATOR ] JHS: All expected trading days are present. [2025-11-04T19:42:12.977618] [DEBUG] [VALIDATOR ] JHS: Primary data validation passed. [2025-11-04T19:42:12.977836] [INFO ] [MAIN ] JHS: Fetched primary data is valid. [2025-11-04T19:42:12.977892] [DEBUG] [LOCAL ] JHS: Saving primary data. [2025-11-04T19:42:12.978278] [DEBUG] [LOCAL ] JHS: Trimmed 4611 records to 2976 (keeping 200 days of data) [2025-11-04T19:42:13.000292] [DEBUG] [LOCAL ] JHS: Primary data saved (2976 records). [2025-11-04T19:42:13.000610] [INFO ] [MAIN ] JHS: Full history saved. [2025-11-04T19:42:13.000751] [INFO ] [MAIN ] Update Progress: 1153/2367 symbols completed. JHS took 30.0s. Estimated remaining: 2h 50m 44s. Rate: 7.1 symbols/min [2025-11-04T19:42:13.203103] [INFO ] [MAIN ] JKS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:42:13.242498] [DEBUG] [LOCAL ] JKS: Loaded 38585 existing records from primary storage. [2025-11-04T19:42:13.242958] [INFO ] [MAIN ] JKS: Loaded 38585 existing records. [2025-11-04T19:42:13.417336] [WARN ] [MAIN ] JKS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:42:13.417559] [INFO ] [MAIN ] JKS: Wiping existing primary data before full refresh. [2025-11-04T19:42:13.417608] [INFO ] [LOCAL ] JKS: Starting deletion of primary data. [2025-11-04T19:42:13.418405] [DEBUG] [LOCAL ] JKS: Deleted primary data file. [2025-11-04T19:42:13.418463] [INFO ] [MAIN ] JKS: Starting full primary refresh. [2025-11-04T19:42:13.782763] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-02. [2025-11-04T19:42:13.788457] [INFO ] [REMOTE ] JKS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:42:13.802851] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-11-04T19:42:14.399245] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-11-04T19:42:14.567088] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-03. [2025-11-04T19:42:15.367067] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-01. [2025-11-04T19:42:15.493287] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T19:42:16.167047] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-09. [2025-11-04T19:42:16.638715] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-11-04T19:42:16.967046] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-02. [2025-11-04T19:42:17.414287] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-11-04T19:42:17.767080] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-07. [2025-11-04T19:42:18.081433] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T19:42:18.402498] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-11-04T19:42:18.567082] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-04. [2025-11-04T19:42:19.367051] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-08. [2025-11-04T19:42:20.006460] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T19:42:20.035846] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T19:42:20.167084] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-09. [2025-11-04T19:42:20.868344] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-11-04T19:42:20.967083] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-10. [2025-11-04T19:42:21.631873] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-11-04T19:42:21.767076] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-10. [2025-11-04T19:42:22.567049] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-11. [2025-11-04T19:42:23.113902] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-11 (0.55 seconds). [2025-11-04T19:42:23.115348] [INFO ] [REMOTE ] JILL: Full fetch complete (22048 records). [2025-11-04T19:42:23.115536] [DEBUG] [VALIDATOR ] JILL: Validating primary data. [2025-11-04T19:42:23.272129] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-11-04T19:42:23.372159] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-05. [2025-11-04T19:42:23.571633] [DEBUG] [VALIDATOR ] JILL: All expected trading days are present. [2025-11-04T19:42:23.571739] [DEBUG] [VALIDATOR ] JILL: Primary data validation passed. [2025-11-04T19:42:23.572184] [INFO ] [MAIN ] JILL: Fetched primary data is valid. [2025-11-04T19:42:23.572224] [DEBUG] [LOCAL ] JILL: Saving primary data. [2025-11-04T19:42:23.572823] [DEBUG] [LOCAL ] JILL: Trimmed 22048 records to 14723 (keeping 200 days of data) [2025-11-04T19:42:23.659943] [DEBUG] [LOCAL ] JILL: Primary data saved (14723 records). [2025-11-04T19:42:23.660079] [INFO ] [MAIN ] JILL: Full history saved. [2025-11-04T19:42:23.660159] [INFO ] [MAIN ] Update Progress: 1154/2367 symbols completed. JILL took 17.9s. Estimated remaining: 2h 50m 38s. Rate: 7.1 symbols/min [2025-11-04T19:42:23.824994] [INFO ] [MAIN ] JLL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:42:23.825356] [DEBUG] [LOCAL ] JLL: No existing data file found in primary storage. [2025-11-04T19:42:23.825393] [WARN ] [MAIN ] JLL: No existing data found, performing full refresh instead. [2025-11-04T19:42:23.825506] [INFO ] [MAIN ] JLL: Wiping existing primary data before full refresh. [2025-11-04T19:42:23.825548] [INFO ] [LOCAL ] JLL: Starting deletion of primary data. [2025-11-04T19:42:23.825588] [DEBUG] [LOCAL ] JLL: No primary data file found to delete. [2025-11-04T19:42:23.825619] [INFO ] [MAIN ] JLL: Starting full primary refresh. [2025-11-04T19:42:24.170104] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-03. [2025-11-04T19:42:24.170011] [INFO ] [REMOTE ] JLL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:42:24.733725] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T19:42:24.967080] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-01. [2025-11-04T19:42:25.208954] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T19:42:25.767052] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-06. [2025-11-04T19:42:25.953592] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T19:42:26.567096] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-04. [2025-11-04T19:42:27.195814] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-11-04T19:42:27.367027] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-02. [2025-11-04T19:42:28.030976] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-11-04T19:42:28.167064] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-11. [2025-11-04T19:42:28.397864] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-11-04T19:42:28.967045] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-07. [2025-11-04T19:42:29.167458] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T19:42:29.169466] [INFO ] [REMOTE ] JHG: Full fetch complete (69043 records). [2025-11-04T19:42:29.169553] [DEBUG] [VALIDATOR ] JHG: Validating primary data. [2025-11-04T19:42:29.546535] [DEBUG] [VALIDATOR ] JHG: All expected trading days are present. [2025-11-04T19:42:29.546612] [DEBUG] [VALIDATOR ] JHG: Primary data validation passed. [2025-11-04T19:42:29.547605] [INFO ] [MAIN ] JHG: Fetched primary data is valid. [2025-11-04T19:42:29.547678] [DEBUG] [LOCAL ] JHG: Saving primary data. [2025-11-04T19:42:29.548990] [DEBUG] [LOCAL ] JHG: Trimmed 69043 records to 47483 (keeping 200 days of data) [2025-11-04T19:42:29.771697] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-05. [2025-11-04T19:42:29.828384] [DEBUG] [LOCAL ] JHG: Primary data saved (47483 records). [2025-11-04T19:42:29.828496] [INFO ] [MAIN ] JHG: Full history saved. [2025-11-04T19:42:29.828572] [INFO ] [MAIN ] Update Progress: 1155/2367 symbols completed. JHG took 59.6s. Estimated remaining: 2h 50m 28s. Rate: 7.1 symbols/min [2025-11-04T19:42:29.994448] [INFO ] [MAIN ] JLS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:42:29.999329] [DEBUG] [LOCAL ] JLS: Loaded 3816 existing records from primary storage. [2025-11-04T19:42:29.999463] [INFO ] [MAIN ] JLS: Loaded 3816 existing records. [2025-11-04T19:42:30.171597] [WARN ] [MAIN ] JLS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:42:30.171856] [INFO ] [MAIN ] JLS: Wiping existing primary data before full refresh. [2025-11-04T19:42:30.171922] [INFO ] [LOCAL ] JLS: Starting deletion of primary data. [2025-11-04T19:42:30.172169] [DEBUG] [LOCAL ] JLS: Deleted primary data file. [2025-11-04T19:42:30.172233] [INFO ] [MAIN ] JLS: Starting full primary refresh. [2025-11-04T19:42:30.538493] [INFO ] [REMOTE ] JLS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:42:30.567486] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-03. [2025-11-04T19:42:30.570512] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-11-04T19:42:30.926000] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T19:42:31.367073] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-06. [2025-11-04T19:42:31.658530] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T19:42:32.167098] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-01. [2025-11-04T19:42:32.509821] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-11-04T19:42:32.724045] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-11-04T19:42:32.967040] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-02. [2025-11-04T19:42:33.336329] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-11-04T19:42:33.767045] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-04. [2025-11-04T19:42:34.567056] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-07. [2025-11-04T19:42:35.166747] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T19:42:35.367091] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-08. [2025-11-04T19:42:35.787739] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-11-04T19:42:36.167045] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-08. [2025-11-04T19:42:36.786447] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T19:42:36.967080] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-09. [2025-11-04T19:42:37.206394] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T19:42:37.767047] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-05. [2025-11-04T19:42:38.383188] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-11-04T19:42:38.567086] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-09. [2025-11-04T19:42:38.838918] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T19:42:39.367035] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-10. [2025-11-04T19:42:39.582428] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-11-04T19:42:40.167009] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-03. [2025-11-04T19:42:40.537510] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-11-04T19:42:40.760328] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-11-04T19:42:40.967014] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-10. [2025-11-04T19:42:41.767042] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-04. [2025-11-04T19:42:41.830545] [DEBUG] [REMOTE ] JLS: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-11-04T19:42:41.830919] [WARN ] [REMOTE ] JLS: No data returned for month 2025-04 [2025-11-04T19:42:41.831019] [FATAL] [REMOTE ] JLS: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:42:41.831094] [FATAL] [MAIN ] JLS: Failed to fetch primary data. [2025-11-04T19:42:41.831157] [WARN ] [MAIN ] JLS: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:42:41.831229] [INFO ] [LOCAL ] JLS: Starting deletion of primary data. [2025-11-04T19:42:41.831293] [DEBUG] [LOCAL ] JLS: No primary data file found to delete. [2025-11-04T19:42:41.831333] [INFO ] [LOCAL ] JLS: Starting deletion of derivative data. [2025-11-04T19:42:41.831487] [DEBUG] [LOCAL ] JLS: Deleted derivative data file. [2025-11-04T19:42:41.831595] [INFO ] [MAIN ] Update Progress: 1156/2367 symbols completed. JLS took 12.0s. Estimated remaining: 2h 50m 23s. Rate: 7.1 symbols/min [2025-11-04T19:42:42.032185] [INFO ] [MAIN ] JMIA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:42:42.090468] [DEBUG] [LOCAL ] JMIA: Loaded 51256 existing records from primary storage. [2025-11-04T19:42:42.090892] [INFO ] [MAIN ] JMIA: Loaded 51256 existing records. [2025-11-04T19:42:42.286932] [WARN ] [MAIN ] JMIA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:42:42.287242] [INFO ] [MAIN ] JMIA: Wiping existing primary data before full refresh. [2025-11-04T19:42:42.287313] [INFO ] [LOCAL ] JMIA: Starting deletion of primary data. [2025-11-04T19:42:42.287691] [DEBUG] [LOCAL ] JMIA: Deleted primary data file. [2025-11-04T19:42:42.287752] [INFO ] [MAIN ] JMIA: Starting full primary refresh. [2025-11-04T19:42:42.505580] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-11-04T19:42:42.570305] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-06. [2025-11-04T19:42:42.706598] [INFO ] [REMOTE ] JMIA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:42:43.367054] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-11. [2025-11-04T19:42:43.581822] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-11-04T19:42:44.167028] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-07. [2025-11-04T19:42:44.450530] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-11 (1.08 seconds). [2025-11-04T19:42:44.452204] [INFO ] [REMOTE ] JKS: Full fetch complete (58439 records). [2025-11-04T19:42:44.452279] [DEBUG] [VALIDATOR ] JKS: Validating primary data. [2025-11-04T19:42:44.833467] [DEBUG] [VALIDATOR ] JKS: All expected trading days are present. [2025-11-04T19:42:44.833552] [DEBUG] [VALIDATOR ] JKS: Primary data validation passed. [2025-11-04T19:42:44.834334] [INFO ] [MAIN ] JKS: Fetched primary data is valid. [2025-11-04T19:42:44.834395] [DEBUG] [LOCAL ] JKS: Saving primary data. [2025-11-04T19:42:44.835410] [DEBUG] [LOCAL ] JKS: Trimmed 58439 records to 38522 (keeping 200 days of data) [2025-11-04T19:42:44.991893] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-11. [2025-11-04T19:42:45.057125] [DEBUG] [LOCAL ] JKS: Primary data saved (38522 records). [2025-11-04T19:42:45.057240] [INFO ] [MAIN ] JKS: Full history saved. [2025-11-04T19:42:45.057314] [INFO ] [MAIN ] Update Progress: 1157/2367 symbols completed. JKS took 32.1s. Estimated remaining: 2h 50m 9s. Rate: 7.1 symbols/min [2025-11-04T19:42:45.238729] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T19:42:45.767088] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-08. [2025-11-04T19:42:46.230155] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T19:42:46.232274] [INFO ] [REMOTE ] JHX: Full fetch complete (64963 records). [2025-11-04T19:42:46.232407] [DEBUG] [VALIDATOR ] JHX: Validating primary data. [2025-11-04T19:42:46.572212] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-01. [2025-11-04T19:42:46.619969] [DEBUG] [VALIDATOR ] JHX: All expected trading days are present. [2025-11-04T19:42:46.620049] [DEBUG] [VALIDATOR ] JHX: Primary data validation passed. [2025-11-04T19:42:46.620914] [INFO ] [MAIN ] JHX: Fetched primary data is valid. [2025-11-04T19:42:46.620975] [DEBUG] [LOCAL ] JHX: Saving primary data. [2025-11-04T19:42:46.622365] [DEBUG] [LOCAL ] JHX: Trimmed 64963 records to 53217 (keeping 200 days of data) [2025-11-04T19:42:46.945322] [DEBUG] [LOCAL ] JHX: Primary data saved (53217 records). [2025-11-04T19:42:46.946123] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-11-04T19:42:46.946379] [INFO ] [MAIN ] JHX: Full history saved. [2025-11-04T19:42:46.953898] [INFO ] [MAIN ] Update Progress: 1158/2367 symbols completed. JHX took 53.9s. Estimated remaining: 2h 49m 54s. Rate: 7.1 symbols/min [2025-11-04T19:42:47.367089] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-09. [2025-11-04T19:42:47.989813] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-11-04T19:42:48.167052] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-02. [2025-11-04T19:42:48.413393] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T19:42:48.967076] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-10. [2025-11-04T19:42:49.515444] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T19:42:49.767074] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-03. [2025-11-04T19:42:49.945490] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-11-04T19:42:50.567105] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-11. [2025-11-04T19:42:51.102323] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T19:42:51.367102] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-04. [2025-11-04T19:42:51.600691] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T19:42:51.602386] [INFO ] [REMOTE ] JLL: Full fetch complete (45261 records). [2025-11-04T19:42:51.602468] [DEBUG] [VALIDATOR ] JLL: Validating primary data. [2025-11-04T19:42:51.988182] [DEBUG] [VALIDATOR ] JLL: All expected trading days are present. [2025-11-04T19:42:51.988258] [DEBUG] [VALIDATOR ] JLL: Primary data validation passed. [2025-11-04T19:42:51.988910] [INFO ] [MAIN ] JLL: Fetched primary data is valid. [2025-11-04T19:42:51.988971] [DEBUG] [LOCAL ] JLL: Saving primary data. [2025-11-04T19:42:51.989890] [DEBUG] [LOCAL ] JLL: Trimmed 45261 records to 30934 (keeping 200 days of data) [2025-11-04T19:42:52.168102] [DEBUG] [LOCAL ] JLL: Primary data saved (30934 records). [2025-11-04T19:42:52.168225] [INFO ] [MAIN ] JLL: Full history saved. [2025-11-04T19:42:52.168296] [INFO ] [MAIN ] Update Progress: 1159/2367 symbols completed. JLL took 28.5s. Estimated remaining: 2h 49m 42s. Rate: 7.1 symbols/min [2025-11-04T19:42:52.739631] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-11-04T19:42:52.749797] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-05. [2025-11-04T19:42:54.231746] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T19:42:54.242223] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-06. [2025-11-04T19:42:55.613273] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-11-04T19:42:55.623465] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-07. [2025-11-04T19:42:56.922630] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T19:42:56.933666] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-08. [2025-11-04T19:42:58.374709] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-11-04T19:42:58.385859] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-09. [2025-11-04T19:42:59.964378] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-11-04T19:42:59.976372] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-10. [2025-11-04T19:43:01.824289] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-11-04T19:43:01.842687] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-11. [2025-11-04T19:43:03.143097] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T19:43:03.145098] [INFO ] [REMOTE ] JMIA: Full fetch complete (73513 records). [2025-11-04T19:43:03.145264] [DEBUG] [VALIDATOR ] JMIA: Validating primary data. [2025-11-04T19:43:03.561906] [DEBUG] [VALIDATOR ] JMIA: All expected trading days are present. [2025-11-04T19:43:03.562013] [DEBUG] [VALIDATOR ] JMIA: Primary data validation passed. [2025-11-04T19:43:03.562967] [INFO ] [MAIN ] JMIA: Fetched primary data is valid. [2025-11-04T19:43:03.563031] [DEBUG] [LOCAL ] JMIA: Saving primary data. [2025-11-04T19:43:03.564463] [DEBUG] [LOCAL ] JMIA: Trimmed 73513 records to 51301 (keeping 200 days of data) [2025-11-04T19:43:03.857829] [DEBUG] [LOCAL ] JMIA: Primary data saved (51301 records). [2025-11-04T19:43:03.857985] [INFO ] [MAIN ] JMIA: Full history saved. [2025-11-04T19:43:03.858064] [INFO ] [MAIN ] Update Progress: 1160/2367 symbols completed. JMIA took 22.0s. Estimated remaining: 2h 49m 37s. Rate: 7.1 symbols/min [2025-11-04T19:43:04.466295] [INFO ] [MAIN ] JOBY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:04.630828] [INFO ] [MAIN ] JOE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:04.610492] [INFO ] [MAIN ] JMM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:04.645225] [DEBUG] [LOCAL ] JMM: No existing data file found in primary storage. [2025-11-04T19:43:04.645501] [WARN ] [MAIN ] JMM: No existing data found, performing full refresh instead. [2025-11-04T19:43:04.645665] [INFO ] [MAIN ] JMM: Wiping existing primary data before full refresh. [2025-11-04T19:43:04.645825] [INFO ] [LOCAL ] JMM: Starting deletion of primary data. [2025-11-04T19:43:04.645890] [DEBUG] [LOCAL ] JMM: No primary data file found to delete. [2025-11-04T19:43:04.645926] [INFO ] [MAIN ] JMM: Starting full primary refresh. [2025-11-04T19:43:04.657541] [INFO ] [MAIN ] JNJ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:04.824200] [DEBUG] [LOCAL ] JOE: Loaded 23209 existing records from primary storage. [2025-11-04T19:43:04.886153] [INFO ] [MAIN ] JOE: Loaded 23209 existing records. [2025-11-04T19:43:05.098088] [DEBUG] [LOCAL ] JOBY: Loaded 53811 existing records from primary storage. [2025-11-04T19:43:05.148858] [INFO ] [MAIN ] JOBY: Loaded 53811 existing records. [2025-11-04T19:43:05.242153] [DEBUG] [LOCAL ] JNJ: Loaded 53753 existing records from primary storage. [2025-11-04T19:43:05.316603] [INFO ] [MAIN ] JNJ: Loaded 53753 existing records. [2025-11-04T19:43:05.311363] [WARN ] [MAIN ] JOE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:43:05.473615] [INFO ] [REMOTE ] JMM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:05.444955] [INFO ] [MAIN ] JOE: Wiping existing primary data before full refresh. [2025-11-04T19:43:05.527930] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-01. [2025-11-04T19:43:05.538925] [INFO ] [LOCAL ] JOE: Starting deletion of primary data. [2025-11-04T19:43:05.578738] [DEBUG] [LOCAL ] JOE: Deleted primary data file. [2025-11-04T19:43:05.636420] [INFO ] [MAIN ] JOE: Starting full primary refresh. [2025-11-04T19:43:05.639627] [WARN ] [MAIN ] JOBY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:43:05.700696] [INFO ] [MAIN ] JOBY: Wiping existing primary data before full refresh. [2025-11-04T19:43:05.789426] [WARN ] [MAIN ] JNJ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:43:05.727146] [INFO ] [LOCAL ] JOBY: Starting deletion of primary data. [2025-11-04T19:43:05.801459] [INFO ] [MAIN ] JNJ: Wiping existing primary data before full refresh. [2025-11-04T19:43:05.812782] [DEBUG] [LOCAL ] JOBY: Deleted primary data file. [2025-11-04T19:43:05.818982] [INFO ] [MAIN ] JOBY: Starting full primary refresh. [2025-11-04T19:43:05.818837] [INFO ] [LOCAL ] JNJ: Starting deletion of primary data. [2025-11-04T19:43:05.873632] [DEBUG] [LOCAL ] JNJ: Deleted primary data file. [2025-11-04T19:43:05.905952] [INFO ] [MAIN ] JNJ: Starting full primary refresh. [2025-11-04T19:43:06.234243] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T19:43:06.361083] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-02. [2025-11-04T19:43:06.632500] [INFO ] [REMOTE ] JOE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:06.659931] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-01. [2025-11-04T19:43:06.785224] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-11-04T19:43:06.817556] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-03. [2025-11-04T19:43:06.841485] [INFO ] [REMOTE ] JOBY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:06.853018] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-01. [2025-11-04T19:43:06.921454] [INFO ] [REMOTE ] JNJ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:06.921769] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-01. [2025-11-04T19:43:07.096865] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-03 (0.28 seconds). [2025-11-04T19:43:07.097818] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-04. [2025-11-04T19:43:07.556883] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-11-04T19:43:07.557914] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-05. [2025-11-04T19:43:07.954057] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-11-04T19:43:07.955847] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-06. [2025-11-04T19:43:07.988054] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T19:43:07.993981] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-02. [2025-11-04T19:43:08.287620] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-11-04T19:43:08.289547] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-07. [2025-11-04T19:43:08.626423] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-11-04T19:43:08.627983] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-08. [2025-11-04T19:43:08.756876] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T19:43:08.764597] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-03. [2025-11-04T19:43:08.842351] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-11-04T19:43:08.853195] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-02. [2025-11-04T19:43:08.866860] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-08 (0.24 seconds). [2025-11-04T19:43:08.919900] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-01 (2.07 seconds). [2025-11-04T19:43:08.967001] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-02. [2025-11-04T19:43:09.747921] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T19:43:09.767041] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-04. [2025-11-04T19:43:10.384812] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-11-04T19:43:10.567056] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-03. [2025-11-04T19:43:10.713680] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-11-04T19:43:10.976289] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-11-04T19:43:11.367058] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-05. [2025-11-04T19:43:12.167015] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-03. [2025-11-04T19:43:12.217495] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-11-04T19:43:12.335317] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-11-04T19:43:12.967084] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-04. [2025-11-04T19:43:13.767045] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-06. [2025-11-04T19:43:14.342154] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-03 (2.18 seconds). [2025-11-04T19:43:14.555082] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T19:43:14.566993] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-07. [2025-11-04T19:43:14.900360] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-11-04T19:43:15.367018] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-04. [2025-11-04T19:43:15.452677] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-11-04T19:43:16.167035] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-08. [2025-11-04T19:43:16.878614] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T19:43:16.967023] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-09. [2025-11-04T19:43:17.509384] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-11-04T19:43:17.767020] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-09. [2025-11-04T19:43:17.800910] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-11-04T19:43:18.036279] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-09 (0.27 seconds). [2025-11-04T19:43:18.567096] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-10. [2025-11-04T19:43:18.831548] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-10 (0.26 seconds). [2025-11-04T19:43:19.367043] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-11. [2025-11-04T19:43:19.694275] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-11 (0.33 seconds). [2025-11-04T19:43:19.695418] [INFO ] [REMOTE ] JMM: Full fetch complete (2335 records). [2025-11-04T19:43:19.695816] [DEBUG] [VALIDATOR ] JMM: Validating primary data. [2025-11-04T19:43:19.899079] [WARN ] [VALIDATOR ] JMM: Validation failed - insufficient historical data. Have 195 unique days, need 200 days [2025-11-04T19:43:19.899360] [FATAL] [MAIN ] JMM: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:43:19.899434] [WARN ] [MAIN ] JMM: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:43:19.899509] [INFO ] [LOCAL ] JMM: Starting deletion of primary data. [2025-11-04T19:43:19.899584] [DEBUG] [LOCAL ] JMM: No primary data file found to delete. [2025-11-04T19:43:19.899623] [INFO ] [LOCAL ] JMM: Starting deletion of derivative data. [2025-11-04T19:43:19.899715] [DEBUG] [LOCAL ] JMM: No derivative data file found to delete. [2025-11-04T19:43:19.899796] [INFO ] [MAIN ] Update Progress: 1161/2367 symbols completed. JMM took 16.0s. Estimated remaining: 2h 49m 36s. Rate: 7.1 symbols/min [2025-11-04T19:43:20.085167] [INFO ] [MAIN ] JOF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:20.095595] [DEBUG] [LOCAL ] JOF: Loaded 6820 existing records from primary storage. [2025-11-04T19:43:20.095825] [INFO ] [MAIN ] JOF: Loaded 6820 existing records. [2025-11-04T19:43:20.172212] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-05. [2025-11-04T19:43:20.276146] [WARN ] [MAIN ] JOF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:43:20.276454] [INFO ] [MAIN ] JOF: Wiping existing primary data before full refresh. [2025-11-04T19:43:20.276514] [INFO ] [LOCAL ] JOF: Starting deletion of primary data. [2025-11-04T19:43:20.276992] [DEBUG] [LOCAL ] JOF: Deleted primary data file. [2025-11-04T19:43:20.277051] [INFO ] [MAIN ] JOF: Starting full primary refresh. [2025-11-04T19:43:20.645749] [INFO ] [REMOTE ] JOF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:20.967069] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-10. [2025-11-04T19:43:21.767032] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-05. [2025-11-04T19:43:21.890279] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-11-04T19:43:22.081596] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-11-04T19:43:22.567070] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-11. [2025-11-04T19:43:23.277568] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T19:43:23.278943] [INFO ] [REMOTE ] JOE: Full fetch complete (35049 records). [2025-11-04T19:43:23.279031] [DEBUG] [VALIDATOR ] JOE: Validating primary data. [2025-11-04T19:43:23.375165] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-01. [2025-11-04T19:43:23.651288] [DEBUG] [VALIDATOR ] JOE: All expected trading days are present. [2025-11-04T19:43:23.651368] [DEBUG] [VALIDATOR ] JOE: Primary data validation passed. [2025-11-04T19:43:23.651988] [INFO ] [MAIN ] JOE: Fetched primary data is valid. [2025-11-04T19:43:23.652051] [DEBUG] [LOCAL ] JOE: Saving primary data. [2025-11-04T19:43:23.652862] [DEBUG] [LOCAL ] JOE: Trimmed 35049 records to 23256 (keeping 200 days of data) [2025-11-04T19:43:23.817441] [DEBUG] [LOCAL ] JOE: Primary data saved (23256 records). [2025-11-04T19:43:23.817725] [INFO ] [MAIN ] JOE: Full history saved. [2025-11-04T19:43:23.817950] [INFO ] [MAIN ] Update Progress: 1162/2367 symbols completed. JOE took 19.9s. Estimated remaining: 2h 49m 23s. Rate: 7.1 symbols/min [2025-11-04T19:43:23.888176] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-11-04T19:43:24.021779] [INFO ] [MAIN ] JPC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:24.060190] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-05 (2.29 seconds). [2025-11-04T19:43:24.104813] [DEBUG] [LOCAL ] JPC: Loaded 36512 existing records from primary storage. [2025-11-04T19:43:24.105198] [INFO ] [MAIN ] JPC: Loaded 36512 existing records. [2025-11-04T19:43:24.172229] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-06. [2025-11-04T19:43:24.306150] [WARN ] [MAIN ] JPC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:43:24.306442] [INFO ] [MAIN ] JPC: Wiping existing primary data before full refresh. [2025-11-04T19:43:24.306496] [INFO ] [LOCAL ] JPC: Starting deletion of primary data. [2025-11-04T19:43:24.307406] [DEBUG] [LOCAL ] JPC: Deleted primary data file. [2025-11-04T19:43:24.307468] [INFO ] [MAIN ] JPC: Starting full primary refresh. [2025-11-04T19:43:24.699990] [INFO ] [REMOTE ] JPC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:24.967039] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-01. [2025-11-04T19:43:25.767024] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-06. [2025-11-04T19:43:25.980920] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-11-04T19:43:26.176120] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T19:43:26.567200] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-07. [2025-11-04T19:43:27.367152] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-02. [2025-11-04T19:43:27.918047] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-06 (2.15 seconds). [2025-11-04T19:43:28.167043] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-02. [2025-11-04T19:43:28.490419] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-11-04T19:43:28.541188] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-11-04T19:43:28.562200] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-11-04T19:43:28.967047] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-03. [2025-11-04T19:43:29.371664] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T19:43:29.767094] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-03. [2025-11-04T19:43:30.567086] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-04. [2025-11-04T19:43:30.867862] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T19:43:31.095175] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-11-04T19:43:31.367062] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-04. [2025-11-04T19:43:32.167061] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-05. [2025-11-04T19:43:32.556427] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T19:43:32.576492] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-11-04T19:43:32.967095] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-06. [2025-11-04T19:43:33.437242] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-11-04T19:43:33.767053] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-07. [2025-11-04T19:43:34.231983] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-11-04T19:43:34.567018] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-08. [2025-11-04T19:43:35.094662] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-11-04T19:43:35.367027] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-05. [2025-11-04T19:43:36.167010] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-08. [2025-11-04T19:43:36.317270] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-11-04T19:43:36.967006] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-06. [2025-11-04T19:43:37.767100] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-07. [2025-11-04T19:43:37.910259] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-11-04T19:43:37.940893] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-11-04T19:43:38.567087] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-07. [2025-11-04T19:43:39.367090] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-09. [2025-11-04T19:43:39.698035] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T19:43:39.935010] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-11-04T19:43:40.113868] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-11-04T19:43:40.167060] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-08. [2025-11-04T19:43:40.967081] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-08. [2025-11-04T19:43:41.767144] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-10. [2025-11-04T19:43:42.059487] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-11-04T19:43:42.128914] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-11-04T19:43:42.310858] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-11-04T19:43:42.567071] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-09. [2025-11-04T19:43:43.367054] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-11. [2025-11-04T19:43:43.677488] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-11-04T19:43:43.795528] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T19:43:43.796286] [INFO ] [REMOTE ] JOF: Full fetch complete (9721 records). [2025-11-04T19:43:43.796354] [DEBUG] [VALIDATOR ] JOF: Validating primary data. [2025-11-04T19:43:44.145339] [DEBUG] [VALIDATOR ] JOF: All expected trading days are present. [2025-11-04T19:43:44.145413] [DEBUG] [VALIDATOR ] JOF: Primary data validation passed. [2025-11-04T19:43:44.145633] [INFO ] [MAIN ] JOF: Fetched primary data is valid. [2025-11-04T19:43:44.145708] [DEBUG] [LOCAL ] JOF: Saving primary data. [2025-11-04T19:43:44.146122] [DEBUG] [LOCAL ] JOF: Trimmed 9721 records to 6772 (keeping 200 days of data) [2025-11-04T19:43:44.175459] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-10. [2025-11-04T19:43:44.188317] [DEBUG] [LOCAL ] JOF: Primary data saved (6772 records). [2025-11-04T19:43:44.189409] [INFO ] [MAIN ] JOF: Full history saved. [2025-11-04T19:43:44.189521] [INFO ] [MAIN ] Update Progress: 1163/2367 symbols completed. JOF took 24.3s. Estimated remaining: 2h 49m 27s. Rate: 7.1 symbols/min [2025-11-04T19:43:44.357619] [INFO ] [MAIN ] JPM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:44.406477] [DEBUG] [LOCAL ] JPM: Loaded 53794 existing records from primary storage. [2025-11-04T19:43:44.406839] [INFO ] [MAIN ] JPM: Loaded 53794 existing records. [2025-11-04T19:43:44.614900] [WARN ] [MAIN ] JPM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:43:44.615117] [INFO ] [MAIN ] JPM: Wiping existing primary data before full refresh. [2025-11-04T19:43:44.615175] [INFO ] [LOCAL ] JPM: Starting deletion of primary data. [2025-11-04T19:43:44.616227] [DEBUG] [LOCAL ] JPM: Deleted primary data file. [2025-11-04T19:43:44.616297] [INFO ] [MAIN ] JPM: Starting full primary refresh. [2025-11-04T19:43:44.972209] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-09. [2025-11-04T19:43:45.046242] [INFO ] [REMOTE ] JPM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:45.497071] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-11-04T19:43:45.767039] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-09. [2025-11-04T19:43:46.567055] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-11. [2025-11-04T19:43:47.367167] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-01. [2025-11-04T19:43:47.400369] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-09 (2.43 seconds). [2025-11-04T19:43:47.448445] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-11-04T19:43:47.637495] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-11 (1.07 seconds). [2025-11-04T19:43:47.639381] [INFO ] [REMOTE ] JPC: Full fetch complete (54853 records). [2025-11-04T19:43:47.639511] [DEBUG] [VALIDATOR ] JPC: Validating primary data. [2025-11-04T19:43:48.082605] [DEBUG] [VALIDATOR ] JPC: All expected trading days are present. [2025-11-04T19:43:48.082710] [DEBUG] [VALIDATOR ] JPC: Primary data validation passed. [2025-11-04T19:43:48.083559] [INFO ] [MAIN ] JPC: Fetched primary data is valid. [2025-11-04T19:43:48.083669] [DEBUG] [LOCAL ] JPC: Saving primary data. [2025-11-04T19:43:48.084783] [DEBUG] [LOCAL ] JPC: Trimmed 54853 records to 36604 (keeping 200 days of data) [2025-11-04T19:43:48.176375] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-10. [2025-11-04T19:43:48.295138] [DEBUG] [LOCAL ] JPC: Primary data saved (36604 records). [2025-11-04T19:43:48.295269] [INFO ] [MAIN ] JPC: Full history saved. [2025-11-04T19:43:48.295344] [INFO ] [MAIN ] Update Progress: 1164/2367 symbols completed. JPC took 24.5s. Estimated remaining: 2h 49m 14s. Rate: 7.1 symbols/min [2025-11-04T19:43:48.467357] [INFO ] [MAIN ] JQC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:48.495893] [DEBUG] [LOCAL ] JQC: Loaded 33199 existing records from primary storage. [2025-11-04T19:43:48.496174] [INFO ] [MAIN ] JQC: Loaded 33199 existing records. [2025-11-04T19:43:48.672991] [WARN ] [MAIN ] JQC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:43:48.673204] [INFO ] [MAIN ] JQC: Wiping existing primary data before full refresh. [2025-11-04T19:43:48.673250] [INFO ] [LOCAL ] JQC: Starting deletion of primary data. [2025-11-04T19:43:48.673893] [DEBUG] [LOCAL ] JQC: Deleted primary data file. [2025-11-04T19:43:48.673955] [INFO ] [MAIN ] JQC: Starting full primary refresh. [2025-11-04T19:43:48.972242] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-10. [2025-11-04T19:43:49.036931] [INFO ] [REMOTE ] JQC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:49.401693] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-01 (2.03 seconds). [2025-11-04T19:43:49.767020] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-02. [2025-11-04T19:43:50.448636] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-10 (2.27 seconds). [2025-11-04T19:43:50.566997] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-11. [2025-11-04T19:43:50.972432] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T19:43:50.974484] [INFO ] [REMOTE ] JNJ: Full fetch complete (77933 records). [2025-11-04T19:43:50.974599] [DEBUG] [VALIDATOR ] JNJ: Validating primary data. [2025-11-04T19:43:51.383397] [DEBUG] [VALIDATOR ] JNJ: All expected trading days are present. [2025-11-04T19:43:51.379696] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-01. [2025-11-04T19:43:51.383586] [DEBUG] [VALIDATOR ] JNJ: Primary data validation passed. [2025-11-04T19:43:51.385531] [INFO ] [MAIN ] JNJ: Fetched primary data is valid. [2025-11-04T19:43:51.385684] [DEBUG] [LOCAL ] JNJ: Saving primary data. [2025-11-04T19:43:51.390459] [DEBUG] [LOCAL ] JNJ: Trimmed 77933 records to 53753 (keeping 200 days of data) [2025-11-04T19:43:51.762148] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-10 (2.79 seconds). [2025-11-04T19:43:51.765379] [DEBUG] [LOCAL ] JNJ: Primary data saved (53753 records). [2025-11-04T19:43:51.777965] [INFO ] [MAIN ] JNJ: Full history saved. [2025-11-04T19:43:51.780437] [INFO ] [MAIN ] Update Progress: 1165/2367 symbols completed. JNJ took 47.9s. Estimated remaining: 2h 49m 1s. Rate: 7.1 symbols/min [2025-11-04T19:43:51.957050] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-02 (2.19 seconds). [2025-11-04T19:43:51.981796] [INFO ] [MAIN ] JRI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:52.009048] [DEBUG] [LOCAL ] JRI: Loaded 15009 existing records from primary storage. [2025-11-04T19:43:52.009291] [INFO ] [MAIN ] JRI: Loaded 15009 existing records. [2025-11-04T19:43:52.167131] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-03. [2025-11-04T19:43:52.186509] [WARN ] [MAIN ] JRI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:43:52.187671] [INFO ] [MAIN ] JRI: Wiping existing primary data before full refresh. [2025-11-04T19:43:52.187735] [INFO ] [LOCAL ] JRI: Starting deletion of primary data. [2025-11-04T19:43:52.188086] [DEBUG] [LOCAL ] JRI: Deleted primary data file. [2025-11-04T19:43:52.188145] [INFO ] [MAIN ] JRI: Starting full primary refresh. [2025-11-04T19:43:52.547514] [INFO ] [REMOTE ] JRI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:52.693400] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-11-04T19:43:52.967035] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-02. [2025-11-04T19:43:53.767071] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-11. [2025-11-04T19:43:53.799079] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T19:43:54.435002] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-03 (2.27 seconds). [2025-11-04T19:43:54.567058] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-03. [2025-11-04T19:43:55.367051] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-04. [2025-11-04T19:43:55.661353] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T19:43:56.167084] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-04. [2025-11-04T19:43:56.275153] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-11 (2.51 seconds). [2025-11-04T19:43:56.277174] [INFO ] [REMOTE ] JOBY: Full fetch complete (77974 records). [2025-11-04T19:43:56.277388] [DEBUG] [VALIDATOR ] JOBY: Validating primary data. [2025-11-04T19:43:56.693412] [DEBUG] [VALIDATOR ] JOBY: All expected trading days are present. [2025-11-04T19:43:56.693507] [DEBUG] [VALIDATOR ] JOBY: Primary data validation passed. [2025-11-04T19:43:56.694635] [INFO ] [MAIN ] JOBY: Fetched primary data is valid. [2025-11-04T19:43:56.694744] [DEBUG] [LOCAL ] JOBY: Saving primary data. [2025-11-04T19:43:56.696269] [DEBUG] [LOCAL ] JOBY: Trimmed 77974 records to 53811 (keeping 200 days of data) [2025-11-04T19:43:56.968127] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-01. [2025-11-04T19:43:57.013797] [DEBUG] [LOCAL ] JOBY: Primary data saved (53811 records). [2025-11-04T19:43:57.014375] [INFO ] [MAIN ] JOBY: Full history saved. [2025-11-04T19:43:57.014481] [INFO ] [MAIN ] Update Progress: 1166/2367 symbols completed. JOBY took 53.2s. Estimated remaining: 2h 48m 49s. Rate: 7.1 symbols/min [2025-11-04T19:43:57.195868] [INFO ] [MAIN ] JRS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:43:57.224744] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-11-04T19:43:57.238070] [DEBUG] [LOCAL ] JRS: Loaded 11456 existing records from primary storage. [2025-11-04T19:43:57.238751] [INFO ] [MAIN ] JRS: Loaded 11456 existing records. [2025-11-04T19:43:57.418460] [WARN ] [MAIN ] JRS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:43:57.418942] [INFO ] [MAIN ] JRS: Wiping existing primary data before full refresh. [2025-11-04T19:43:57.419019] [INFO ] [LOCAL ] JRS: Starting deletion of primary data. [2025-11-04T19:43:57.419373] [DEBUG] [LOCAL ] JRS: Deleted primary data file. [2025-11-04T19:43:57.430550] [INFO ] [MAIN ] JRS: Starting full primary refresh. [2025-11-04T19:43:57.538131] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-11-04T19:43:57.767206] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-05. [2025-11-04T19:43:57.915575] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T19:43:57.965935] [INFO ] [REMOTE ] JRS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:43:58.567038] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-05. [2025-11-04T19:43:58.710934] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T19:43:59.367048] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-06. [2025-11-04T19:44:00.167123] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-01. [2025-11-04T19:44:00.231908] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-11-04T19:44:00.233386] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T19:44:00.876740] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-11-04T19:44:00.967043] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-06. [2025-11-04T19:44:01.767043] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-07. [2025-11-04T19:44:02.567064] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-02. [2025-11-04T19:44:02.684706] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-11-04T19:44:02.908722] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-11-04T19:44:03.279932] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T19:44:03.367044] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-03. [2025-11-04T19:44:04.010639] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T19:44:04.167052] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-02. [2025-11-04T19:44:04.705101] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-11-04T19:44:04.967113] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-07. [2025-11-04T19:44:05.767067] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-04. [2025-11-04T19:44:06.382355] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-11-04T19:44:06.567093] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-08. [2025-11-04T19:44:06.795023] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-11-04T19:44:07.367043] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-08. [2025-11-04T19:44:07.593777] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-11-04T19:44:08.167043] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-09. [2025-11-04T19:44:08.967019] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-05. [2025-11-04T19:44:09.075416] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-11-04T19:44:09.201316] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-11-04T19:44:09.601357] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-11-04T19:44:09.767036] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-06. [2025-11-04T19:44:10.466683] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-11-04T19:44:10.567032] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-09. [2025-11-04T19:44:11.367056] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-03. [2025-11-04T19:44:11.846719] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-11-04T19:44:12.167047] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-07. [2025-11-04T19:44:12.828874] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-09 (2.26 seconds). [2025-11-04T19:44:12.935371] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T19:44:12.966998] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-08. [2025-11-04T19:44:13.557019] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-11-04T19:44:13.767041] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-10. [2025-11-04T19:44:14.567093] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-09. [2025-11-04T19:44:15.274161] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-11-04T19:44:15.367015] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-10. [2025-11-04T19:44:16.056017] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-11-04T19:44:16.103760] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-10 (2.34 seconds). [2025-11-04T19:44:16.167134] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-04. [2025-11-04T19:44:16.738721] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-11-04T19:44:16.967051] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-11. [2025-11-04T19:44:17.348137] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-11 (0.38 seconds). [2025-11-04T19:44:17.350236] [INFO ] [REMOTE ] JPM: Full fetch complete (77973 records). [2025-11-04T19:44:17.350349] [DEBUG] [VALIDATOR ] JPM: Validating primary data. [2025-11-04T19:44:17.782503] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-11. [2025-11-04T19:44:17.810878] [DEBUG] [VALIDATOR ] JPM: All expected trading days are present. [2025-11-04T19:44:17.810985] [DEBUG] [VALIDATOR ] JPM: Primary data validation passed. [2025-11-04T19:44:17.812151] [INFO ] [MAIN ] JPM: Fetched primary data is valid. [2025-11-04T19:44:17.812230] [DEBUG] [LOCAL ] JPM: Saving primary data. [2025-11-04T19:44:17.813851] [DEBUG] [LOCAL ] JPM: Trimmed 77973 records to 53794 (keeping 200 days of data) [2025-11-04T19:44:18.190553] [DEBUG] [LOCAL ] JPM: Primary data saved (53794 records). [2025-11-04T19:44:18.190743] [INFO ] [MAIN ] JPM: Full history saved. [2025-11-04T19:44:18.190846] [INFO ] [MAIN ] Update Progress: 1167/2367 symbols completed. JPM took 34.0s. Estimated remaining: 2h 48m 54s. Rate: 7.1 symbols/min [2025-11-04T19:44:18.394317] [INFO ] [MAIN ] JXN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:44:18.434137] [DEBUG] [LOCAL ] JXN: Loaded 43379 existing records from primary storage. [2025-11-04T19:44:18.434532] [INFO ] [MAIN ] JXN: Loaded 43379 existing records. [2025-11-04T19:44:18.564151] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T19:44:18.568381] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-05. [2025-11-04T19:44:18.570820] [INFO ] [REMOTE ] JRI: Full fetch complete (22492 records). [2025-11-04T19:44:18.581839] [DEBUG] [VALIDATOR ] JRI: Validating primary data. [2025-11-04T19:44:18.750362] [WARN ] [MAIN ] JXN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:44:18.767037] [INFO ] [MAIN ] JXN: Wiping existing primary data before full refresh. [2025-11-04T19:44:18.777483] [INFO ] [LOCAL ] JXN: Starting deletion of primary data. [2025-11-04T19:44:18.783593] [DEBUG] [LOCAL ] JXN: Deleted primary data file. [2025-11-04T19:44:18.788807] [INFO ] [MAIN ] JXN: Starting full primary refresh. [2025-11-04T19:44:19.284968] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-11-04T19:44:19.369534] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-10. [2025-11-04T19:44:19.483112] [INFO ] [REMOTE ] JXN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:44:19.483692] [DEBUG] [VALIDATOR ] JRI: All expected trading days are present. [2025-11-04T19:44:19.483803] [DEBUG] [VALIDATOR ] JRI: Primary data validation passed. [2025-11-04T19:44:19.484467] [INFO ] [MAIN ] JRI: Fetched primary data is valid. [2025-11-04T19:44:19.484516] [DEBUG] [LOCAL ] JRI: Saving primary data. [2025-11-04T19:44:19.485378] [DEBUG] [LOCAL ] JRI: Trimmed 22492 records to 15081 (keeping 200 days of data) [2025-11-04T19:44:19.577729] [DEBUG] [LOCAL ] JRI: Primary data saved (15081 records). [2025-11-04T19:44:19.577885] [INFO ] [MAIN ] JRI: Full history saved. [2025-11-04T19:44:19.577977] [INFO ] [MAIN ] Update Progress: 1168/2367 symbols completed. JRI took 27.8s. Estimated remaining: 2h 48m 38s. Rate: 7.1 symbols/min [2025-11-04T19:44:19.748222] [INFO ] [MAIN ] K: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:44:19.797146] [DEBUG] [LOCAL ] K: Loaded 50314 existing records from primary storage. [2025-11-04T19:44:19.797553] [INFO ] [MAIN ] K: Loaded 50314 existing records. [2025-11-04T19:44:19.978167] [WARN ] [MAIN ] K: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:44:19.978371] [INFO ] [MAIN ] K: Wiping existing primary data before full refresh. [2025-11-04T19:44:19.978429] [INFO ] [LOCAL ] K: Starting deletion of primary data. [2025-11-04T19:44:19.978776] [DEBUG] [LOCAL ] K: Deleted primary data file. [2025-11-04T19:44:19.978843] [INFO ] [MAIN ] K: Starting full primary refresh. [2025-11-04T19:44:20.179396] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-06. [2025-11-04T19:44:20.326588] [INFO ] [REMOTE ] K: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:44:20.569106] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-11-04T19:44:20.823502] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-11-04T19:44:20.967055] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-11. [2025-11-04T19:44:21.767047] [INFO ] [REMOTE ] K: Fetching primary data for 2025-01. [2025-11-04T19:44:21.945179] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-11 (0.98 seconds). [2025-11-04T19:44:21.946834] [INFO ] [REMOTE ] JQC: Full fetch complete (49493 records). [2025-11-04T19:44:21.946912] [DEBUG] [VALIDATOR ] JQC: Validating primary data. [2025-11-04T19:44:22.318785] [DEBUG] [VALIDATOR ] JQC: All expected trading days are present. [2025-11-04T19:44:22.318857] [DEBUG] [VALIDATOR ] JQC: Primary data validation passed. [2025-11-04T19:44:22.319563] [INFO ] [MAIN ] JQC: Fetched primary data is valid. [2025-11-04T19:44:22.319602] [DEBUG] [LOCAL ] JQC: Saving primary data. [2025-11-04T19:44:22.320742] [DEBUG] [LOCAL ] JQC: Trimmed 49493 records to 33193 (keeping 200 days of data) [2025-11-04T19:44:22.503957] [DEBUG] [LOCAL ] JQC: Primary data saved (33193 records). [2025-11-04T19:44:22.504080] [INFO ] [MAIN ] JQC: Full history saved. [2025-11-04T19:44:22.504154] [INFO ] [MAIN ] Update Progress: 1169/2367 symbols completed. JQC took 34.2s. Estimated remaining: 2h 48m 24s. Rate: 7.1 symbols/min [2025-11-04T19:44:22.572240] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-01. [2025-11-04T19:44:22.672350] [INFO ] [MAIN ] KAI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:44:22.686824] [DEBUG] [LOCAL ] KAI: Loaded 13458 existing records from primary storage. [2025-11-04T19:44:22.687054] [INFO ] [MAIN ] KAI: Loaded 13458 existing records. [2025-11-04T19:44:22.867613] [WARN ] [MAIN ] KAI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:44:22.867857] [INFO ] [MAIN ] KAI: Wiping existing primary data before full refresh. [2025-11-04T19:44:22.867903] [INFO ] [LOCAL ] KAI: Starting deletion of primary data. [2025-11-04T19:44:22.868058] [DEBUG] [LOCAL ] KAI: Deleted primary data file. [2025-11-04T19:44:22.868094] [INFO ] [MAIN ] KAI: Starting full primary refresh. [2025-11-04T19:44:23.265381] [INFO ] [REMOTE ] KAI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:44:23.291557] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-11-04T19:44:23.367071] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-01. [2025-11-04T19:44:23.666732] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-11-04T19:44:23.892903] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T19:44:24.167020] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-02. [2025-11-04T19:44:24.967051] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-02. [2025-11-04T19:44:25.220554] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T19:44:25.680985] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T19:44:25.767075] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-03. [2025-11-04T19:44:26.438482] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-11-04T19:44:26.567090] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-07. [2025-11-04T19:44:27.276025] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-11-04T19:44:27.367054] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-04. [2025-11-04T19:44:28.133739] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-11-04T19:44:28.167082] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-05. [2025-11-04T19:44:28.827017] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T19:44:28.967063] [INFO ] [REMOTE ] K: Fetching primary data for 2025-02. [2025-11-04T19:44:29.767078] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-08. [2025-11-04T19:44:30.171971] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T19:44:30.302786] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-11-04T19:44:30.567065] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-06. [2025-11-04T19:44:31.172333] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T19:44:31.367008] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-07. [2025-11-04T19:44:31.981925] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-11-04T19:44:32.167062] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-03. [2025-11-04T19:44:32.967077] [INFO ] [REMOTE ] K: Fetching primary data for 2025-03. [2025-11-04T19:44:33.502495] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T19:44:33.767109] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-04. [2025-11-04T19:44:34.141615] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-11-04T19:44:34.567059] [INFO ] [REMOTE ] K: Fetching primary data for 2025-04. [2025-11-04T19:44:35.000629] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-11-04T19:44:35.367039] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-08. [2025-11-04T19:44:36.014772] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-11-04T19:44:36.036291] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-11-04T19:44:36.167073] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-09. [2025-11-04T19:44:36.750263] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-11-04T19:44:36.967065] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-05. [2025-11-04T19:44:37.767135] [INFO ] [REMOTE ] K: Fetching primary data for 2025-05. [2025-11-04T19:44:38.138911] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T19:44:38.567040] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-06. [2025-11-04T19:44:39.000135] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T19:44:39.367043] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-09. [2025-11-04T19:44:39.720868] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T19:44:39.843172] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-11-04T19:44:40.167068] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-07. [2025-11-04T19:44:40.967013] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-10. [2025-11-04T19:44:41.209707] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T19:44:41.524854] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-11-04T19:44:41.767060] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-08. [2025-11-04T19:44:42.567083] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-10. [2025-11-04T19:44:42.983431] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T19:44:43.356675] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-11-04T19:44:43.367003] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-11. [2025-11-04T19:44:43.426390] [DEBUG] [REMOTE ] JRS: Error fetching primary data for 2025-11: 'API call returned an empty response.' [2025-11-04T19:44:43.426623] [WARN ] [REMOTE ] JRS: No data returned for month 2025-11 [2025-11-04T19:44:43.426769] [FATAL] [REMOTE ] JRS: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:44:43.426842] [FATAL] [MAIN ] JRS: Failed to fetch primary data. [2025-11-04T19:44:43.426902] [WARN ] [MAIN ] JRS: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:44:43.426975] [INFO ] [LOCAL ] JRS: Starting deletion of primary data. [2025-11-04T19:44:43.427033] [DEBUG] [LOCAL ] JRS: No primary data file found to delete. [2025-11-04T19:44:43.427113] [INFO ] [LOCAL ] JRS: Starting deletion of derivative data. [2025-11-04T19:44:43.427256] [DEBUG] [LOCAL ] JRS: Deleted derivative data file. [2025-11-04T19:44:43.427336] [INFO ] [MAIN ] Update Progress: 1170/2367 symbols completed. JRS took 46.4s. Estimated remaining: 2h 48m 28s. Rate: 7.1 symbols/min [2025-11-04T19:44:43.597618] [INFO ] [MAIN ] KAR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:44:43.631108] [DEBUG] [LOCAL ] KAR: Loaded 39582 existing records from primary storage. [2025-11-04T19:44:43.631404] [INFO ] [MAIN ] KAR: Loaded 39582 existing records. [2025-11-04T19:44:43.809077] [WARN ] [MAIN ] KAR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:44:43.809284] [INFO ] [MAIN ] KAR: Wiping existing primary data before full refresh. [2025-11-04T19:44:43.809332] [INFO ] [LOCAL ] KAR: Starting deletion of primary data. [2025-11-04T19:44:43.810068] [DEBUG] [LOCAL ] KAR: Deleted primary data file. [2025-11-04T19:44:43.810129] [INFO ] [MAIN ] KAR: Starting full primary refresh. [2025-11-04T19:44:44.172092] [INFO ] [REMOTE ] KAR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:44:44.172154] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-09. [2025-11-04T19:44:44.967100] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-11. [2025-11-04T19:44:45.378557] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-11-04T19:44:45.767141] [INFO ] [REMOTE ] K: Fetching primary data for 2025-06. [2025-11-04T19:44:45.845362] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-11 (0.88 seconds). [2025-11-04T19:44:45.847112] [INFO ] [REMOTE ] KAI: Full fetch complete (19961 records). [2025-11-04T19:44:45.847203] [DEBUG] [VALIDATOR ] KAI: Validating primary data. [2025-11-04T19:44:46.216223] [DEBUG] [VALIDATOR ] KAI: All expected trading days are present. [2025-11-04T19:44:46.216302] [DEBUG] [VALIDATOR ] KAI: Primary data validation passed. [2025-11-04T19:44:46.216891] [INFO ] [MAIN ] KAI: Fetched primary data is valid. [2025-11-04T19:44:46.216955] [DEBUG] [LOCAL ] KAI: Saving primary data. [2025-11-04T19:44:46.217512] [DEBUG] [LOCAL ] KAI: Trimmed 19961 records to 13561 (keeping 200 days of data) [2025-11-04T19:44:46.298754] [DEBUG] [LOCAL ] KAI: Primary data saved (13561 records). [2025-11-04T19:44:46.298882] [INFO ] [MAIN ] KAI: Full history saved. [2025-11-04T19:44:46.298957] [INFO ] [MAIN ] Update Progress: 1171/2367 symbols completed. KAI took 23.8s. Estimated remaining: 2h 48m 14s. Rate: 7.1 symbols/min [2025-11-04T19:44:46.464399] [INFO ] [MAIN ] KB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:44:46.484912] [DEBUG] [LOCAL ] KB: Loaded 22203 existing records from primary storage. [2025-11-04T19:44:46.485157] [INFO ] [MAIN ] KB: Loaded 22203 existing records. [2025-11-04T19:44:46.573206] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-10. [2025-11-04T19:44:46.669348] [WARN ] [MAIN ] KB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:44:46.669566] [INFO ] [MAIN ] KB: Wiping existing primary data before full refresh. [2025-11-04T19:44:46.669615] [INFO ] [LOCAL ] KB: Starting deletion of primary data. [2025-11-04T19:44:46.669824] [DEBUG] [LOCAL ] KB: Deleted primary data file. [2025-11-04T19:44:46.669876] [INFO ] [MAIN ] KB: Starting full primary refresh. [2025-11-04T19:44:47.038704] [INFO ] [REMOTE ] KB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:44:47.069482] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T19:44:47.367077] [INFO ] [REMOTE ] K: Fetching primary data for 2025-07. [2025-11-04T19:44:47.815836] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-11-04T19:44:48.167060] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-01. [2025-11-04T19:44:48.767955] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T19:44:48.967121] [INFO ] [REMOTE ] K: Fetching primary data for 2025-08. [2025-11-04T19:44:49.008165] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-11-04T19:44:49.767083] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-11. [2025-11-04T19:44:50.269794] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T19:44:50.567055] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-01. [2025-11-04T19:44:50.901300] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:44:50.903015] [INFO ] [REMOTE ] JXN: Full fetch complete (64988 records). [2025-11-04T19:44:50.903128] [DEBUG] [VALIDATOR ] JXN: Validating primary data. [2025-11-04T19:44:51.279805] [DEBUG] [VALIDATOR ] JXN: All expected trading days are present. [2025-11-04T19:44:51.279973] [DEBUG] [VALIDATOR ] JXN: Primary data validation passed. [2025-11-04T19:44:51.280876] [INFO ] [MAIN ] JXN: Fetched primary data is valid. [2025-11-04T19:44:51.280939] [DEBUG] [LOCAL ] JXN: Saving primary data. [2025-11-04T19:44:51.282203] [DEBUG] [LOCAL ] JXN: Trimmed 64988 records to 43433 (keeping 200 days of data) [2025-11-04T19:44:51.369712] [INFO ] [REMOTE ] K: Fetching primary data for 2025-09. [2025-11-04T19:44:51.515503] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T19:44:51.574558] [DEBUG] [LOCAL ] JXN: Primary data saved (43433 records). [2025-11-04T19:44:51.574735] [INFO ] [MAIN ] JXN: Full history saved. [2025-11-04T19:44:51.574819] [INFO ] [MAIN ] Update Progress: 1172/2367 symbols completed. JXN took 33.4s. Estimated remaining: 2h 48m 2s. Rate: 7.1 symbols/min [2025-11-04T19:44:51.743791] [INFO ] [MAIN ] KBDC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:44:51.766325] [DEBUG] [LOCAL ] KBDC: Loaded 22797 existing records from primary storage. [2025-11-04T19:44:51.766559] [INFO ] [MAIN ] KBDC: Loaded 22797 existing records. [2025-11-04T19:44:51.940223] [WARN ] [MAIN ] KBDC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:44:51.940450] [INFO ] [MAIN ] KBDC: Wiping existing primary data before full refresh. [2025-11-04T19:44:51.940496] [INFO ] [LOCAL ] KBDC: Starting deletion of primary data. [2025-11-04T19:44:51.940693] [DEBUG] [LOCAL ] KBDC: Deleted primary data file. [2025-11-04T19:44:51.940749] [INFO ] [MAIN ] KBDC: Starting full primary refresh. [2025-11-04T19:44:52.177466] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-02. [2025-11-04T19:44:52.294789] [INFO ] [REMOTE ] KBDC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:44:52.668803] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T19:44:52.967044] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-02. [2025-11-04T19:44:53.373669] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T19:44:53.767049] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-03. [2025-11-04T19:44:53.814579] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-11-04T19:44:54.567013] [INFO ] [REMOTE ] K: Fetching primary data for 2025-10. [2025-11-04T19:44:54.765557] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-11-04T19:44:55.367051] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-03. [2025-11-04T19:44:56.150182] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-11-04T19:44:56.167005] [INFO ] [REMOTE ] K: Fetching primary data for 2025-11. [2025-11-04T19:44:56.264433] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-11-04T19:44:56.967033] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-04. [2025-11-04T19:44:57.376310] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-11 (1.21 seconds). [2025-11-04T19:44:57.378985] [INFO ] [REMOTE ] K: Full fetch complete (72302 records). [2025-11-04T19:44:57.379134] [DEBUG] [VALIDATOR ] K: Validating primary data. [2025-11-04T19:44:57.777173] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-04. [2025-11-04T19:44:57.842746] [DEBUG] [VALIDATOR ] K: All expected trading days are present. [2025-11-04T19:44:57.842880] [DEBUG] [VALIDATOR ] K: Primary data validation passed. [2025-11-04T19:44:57.844349] [INFO ] [MAIN ] K: Fetched primary data is valid. [2025-11-04T19:44:57.844774] [DEBUG] [LOCAL ] K: Saving primary data. [2025-11-04T19:44:57.847168] [DEBUG] [LOCAL ] K: Trimmed 72302 records to 50294 (keeping 200 days of data) [2025-11-04T19:44:58.178364] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T19:44:58.205130] [DEBUG] [LOCAL ] K: Primary data saved (50294 records). [2025-11-04T19:44:58.205679] [INFO ] [MAIN ] K: Full history saved. [2025-11-04T19:44:58.205811] [INFO ] [MAIN ] Update Progress: 1173/2367 symbols completed. K took 38.6s. Estimated remaining: 2h 47m 52s. Rate: 7.1 symbols/min [2025-11-04T19:44:58.406839] [INFO ] [MAIN ] KBH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:44:58.455372] [DEBUG] [LOCAL ] KBH: Loaded 48395 existing records from primary storage. [2025-11-04T19:44:58.455832] [INFO ] [MAIN ] KBH: Loaded 48395 existing records. [2025-11-04T19:44:58.568020] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-01. [2025-11-04T19:44:58.651042] [WARN ] [MAIN ] KBH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:44:58.651276] [INFO ] [MAIN ] KBH: Wiping existing primary data before full refresh. [2025-11-04T19:44:58.651327] [INFO ] [LOCAL ] KBH: Starting deletion of primary data. [2025-11-04T19:44:58.651813] [DEBUG] [LOCAL ] KBH: Deleted primary data file. [2025-11-04T19:44:58.651871] [INFO ] [MAIN ] KBH: Starting full primary refresh. [2025-11-04T19:44:59.044222] [INFO ] [REMOTE ] KBH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:44:59.074682] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T19:44:59.372235] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-05. [2025-11-04T19:44:59.375399] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T19:45:00.167088] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-01. [2025-11-04T19:45:00.686489] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-11-04T19:45:00.967117] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-05. [2025-11-04T19:45:01.767044] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-02. [2025-11-04T19:45:01.887040] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-11-04T19:45:01.913118] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-11-04T19:45:02.278957] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T19:45:02.567114] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-03. [2025-11-04T19:45:03.239804] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-11-04T19:45:03.367053] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-04. [2025-11-04T19:45:04.167055] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-06. [2025-11-04T19:45:04.228088] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-11-04T19:45:04.967073] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-06. [2025-11-04T19:45:05.202120] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T19:45:05.767018] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-07. [2025-11-04T19:45:05.816803] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T19:45:06.567131] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-05. [2025-11-04T19:45:07.008314] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-11-04T19:45:07.340911] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-11-04T19:45:07.367020] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-06. [2025-11-04T19:45:08.050954] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-11-04T19:45:08.167022] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-07. [2025-11-04T19:45:08.916895] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-11-04T19:45:08.967040] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-02. [2025-11-04T19:45:09.767140] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-08. [2025-11-04T19:45:10.146227] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T19:45:10.567054] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-03. [2025-11-04T19:45:10.813283] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-11-04T19:45:11.367058] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-08. [2025-11-04T19:45:12.063170] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T19:45:12.095828] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-11-04T19:45:12.167101] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-09. [2025-11-04T19:45:12.967089] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-09. [2025-11-04T19:45:13.090786] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T19:45:13.767185] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-04. [2025-11-04T19:45:14.018143] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-11-04T19:45:14.567094] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-07. [2025-11-04T19:45:15.367060] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-10. [2025-11-04T19:45:15.562260] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-11-04T19:45:15.675476] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-11-04T19:45:16.167053] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-08. [2025-11-04T19:45:16.575899] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-11-04T19:45:16.961408] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-11-04T19:45:16.966989] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-09. [2025-11-04T19:45:17.685916] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-11-04T19:45:17.767067] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-05. [2025-11-04T19:45:18.567035] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-10. [2025-11-04T19:45:18.980005] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T19:45:19.367045] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-06. [2025-11-04T19:45:19.641005] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T19:45:20.167037] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-11. [2025-11-04T19:45:20.475092] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T19:45:20.967031] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-07. [2025-11-04T19:45:21.101286] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T19:45:21.102537] [INFO ] [REMOTE ] KBDC: Full fetch complete (31915 records). [2025-11-04T19:45:21.102605] [DEBUG] [VALIDATOR ] KBDC: Validating primary data. [2025-11-04T19:45:21.471333] [DEBUG] [VALIDATOR ] KBDC: All expected trading days are present. [2025-11-04T19:45:21.471408] [DEBUG] [VALIDATOR ] KBDC: Primary data validation passed. [2025-11-04T19:45:21.471999] [INFO ] [MAIN ] KBDC: Fetched primary data is valid. [2025-11-04T19:45:21.472062] [DEBUG] [LOCAL ] KBDC: Saving primary data. [2025-11-04T19:45:21.472853] [DEBUG] [LOCAL ] KBDC: Trimmed 31915 records to 22829 (keeping 200 days of data) [2025-11-04T19:45:21.604856] [DEBUG] [LOCAL ] KBDC: Primary data saved (22829 records). [2025-11-04T19:45:21.604970] [INFO ] [MAIN ] KBDC: Full history saved. [2025-11-04T19:45:21.605042] [INFO ] [MAIN ] Update Progress: 1174/2367 symbols completed. KBDC took 30.0s. Estimated remaining: 2h 47m 59s. Rate: 7.1 symbols/min [2025-11-04T19:45:21.773249] [INFO ] [MAIN ] KBR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:45:21.773318] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-10. [2025-11-04T19:45:21.818374] [DEBUG] [LOCAL ] KBR: Loaded 48067 existing records from primary storage. [2025-11-04T19:45:21.818853] [INFO ] [MAIN ] KBR: Loaded 48067 existing records. [2025-11-04T19:45:22.003823] [WARN ] [MAIN ] KBR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:45:22.004099] [INFO ] [MAIN ] KBR: Wiping existing primary data before full refresh. [2025-11-04T19:45:22.004167] [INFO ] [LOCAL ] KBR: Starting deletion of primary data. [2025-11-04T19:45:22.005075] [DEBUG] [LOCAL ] KBR: Deleted primary data file. [2025-11-04T19:45:22.005133] [INFO ] [MAIN ] KBR: Starting full primary refresh. [2025-11-04T19:45:22.380713] [INFO ] [REMOTE ] KBR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:45:22.412278] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-11-04T19:45:22.436000] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-11-04T19:45:22.567026] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-01. [2025-11-04T19:45:23.367032] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-08. [2025-11-04T19:45:24.023750] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-11-04T19:45:24.167004] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-02. [2025-11-04T19:45:24.743357] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-11-04T19:45:24.967064] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-11. [2025-11-04T19:45:25.311849] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T19:45:25.614912] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-11 (0.65 seconds). [2025-11-04T19:45:25.616299] [INFO ] [REMOTE ] KB: Full fetch complete (33026 records). [2025-11-04T19:45:25.616403] [DEBUG] [VALIDATOR ] KB: Validating primary data. [2025-11-04T19:45:25.772109] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-09. [2025-11-04T19:45:26.070305] [DEBUG] [VALIDATOR ] KB: All expected trading days are present. [2025-11-04T19:45:26.070394] [DEBUG] [VALIDATOR ] KB: Primary data validation passed. [2025-11-04T19:45:26.071167] [INFO ] [MAIN ] KB: Fetched primary data is valid. [2025-11-04T19:45:26.071232] [DEBUG] [LOCAL ] KB: Saving primary data. [2025-11-04T19:45:26.072072] [DEBUG] [LOCAL ] KB: Trimmed 33026 records to 22130 (keeping 200 days of data) [2025-11-04T19:45:26.215136] [DEBUG] [LOCAL ] KB: Primary data saved (22130 records). [2025-11-04T19:45:26.215297] [INFO ] [MAIN ] KB: Full history saved. [2025-11-04T19:45:26.215422] [INFO ] [MAIN ] Update Progress: 1175/2367 symbols completed. KB took 39.9s. Estimated remaining: 2h 47m 46s. Rate: 7.1 symbols/min [2025-11-04T19:45:26.393316] [INFO ] [MAIN ] KD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:45:26.438859] [DEBUG] [LOCAL ] KD: Loaded 52286 existing records from primary storage. [2025-11-04T19:45:26.439179] [INFO ] [MAIN ] KD: Loaded 52286 existing records. [2025-11-04T19:45:26.582122] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-11. [2025-11-04T19:45:26.624191] [WARN ] [MAIN ] KD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:45:26.624547] [INFO ] [MAIN ] KD: Wiping existing primary data before full refresh. [2025-11-04T19:45:26.624602] [INFO ] [LOCAL ] KD: Starting deletion of primary data. [2025-11-04T19:45:26.625409] [DEBUG] [LOCAL ] KD: Deleted primary data file. [2025-11-04T19:45:26.625468] [INFO ] [MAIN ] KD: Starting full primary refresh. [2025-11-04T19:45:26.980861] [INFO ] [REMOTE ] KD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:45:27.145711] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-11-04T19:45:27.367089] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-03. [2025-11-04T19:45:27.879411] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T19:45:27.881149] [INFO ] [REMOTE ] KAR: Full fetch complete (57354 records). [2025-11-04T19:45:27.881236] [DEBUG] [VALIDATOR ] KAR: Validating primary data. [2025-11-04T19:45:28.172186] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-01. [2025-11-04T19:45:28.258799] [DEBUG] [VALIDATOR ] KAR: All expected trading days are present. [2025-11-04T19:45:28.258939] [DEBUG] [VALIDATOR ] KAR: Primary data validation passed. [2025-11-04T19:45:28.259823] [INFO ] [MAIN ] KAR: Fetched primary data is valid. [2025-11-04T19:45:28.259886] [DEBUG] [LOCAL ] KAR: Saving primary data. [2025-11-04T19:45:28.261058] [DEBUG] [LOCAL ] KAR: Trimmed 57354 records to 39631 (keeping 200 days of data) [2025-11-04T19:45:28.490371] [DEBUG] [LOCAL ] KAR: Primary data saved (39631 records). [2025-11-04T19:45:28.490554] [INFO ] [MAIN ] KAR: Full history saved. [2025-11-04T19:45:28.490671] [INFO ] [MAIN ] Update Progress: 1176/2367 symbols completed. KAR took 45.1s. Estimated remaining: 2h 47m 32s. Rate: 7.1 symbols/min [2025-11-04T19:45:28.671973] [INFO ] [MAIN ] KEN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:45:28.699988] [DEBUG] [LOCAL ] KEN: Loaded 3972 existing records from primary storage. [2025-11-04T19:45:28.702923] [INFO ] [MAIN ] KEN: Loaded 3972 existing records. [2025-11-04T19:45:28.739719] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-11-04T19:45:28.919369] [WARN ] [MAIN ] KEN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:45:28.919579] [INFO ] [MAIN ] KEN: Wiping existing primary data before full refresh. [2025-11-04T19:45:28.919666] [INFO ] [LOCAL ] KEN: Starting deletion of primary data. [2025-11-04T19:45:28.919924] [DEBUG] [LOCAL ] KEN: Deleted primary data file. [2025-11-04T19:45:28.919993] [INFO ] [MAIN ] KEN: Starting full primary refresh. [2025-11-04T19:45:28.974760] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-04. [2025-11-04T19:45:29.279434] [INFO ] [REMOTE ] KEN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:45:29.670409] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T19:45:29.767101] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-10. [2025-11-04T19:45:30.331078] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:45:30.567045] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-05. [2025-11-04T19:45:31.032992] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T19:45:31.367101] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-11. [2025-11-04T19:45:31.829118] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T19:45:32.167016] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-02. [2025-11-04T19:45:32.427182] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T19:45:32.428977] [INFO ] [REMOTE ] KBH: Full fetch complete (70715 records). [2025-11-04T19:45:32.429063] [DEBUG] [VALIDATOR ] KBH: Validating primary data. [2025-11-04T19:45:32.813249] [DEBUG] [VALIDATOR ] KBH: All expected trading days are present. [2025-11-04T19:45:32.813338] [DEBUG] [VALIDATOR ] KBH: Primary data validation passed. [2025-11-04T19:45:32.814323] [INFO ] [MAIN ] KBH: Fetched primary data is valid. [2025-11-04T19:45:32.814391] [DEBUG] [LOCAL ] KBH: Saving primary data. [2025-11-04T19:45:32.815849] [DEBUG] [LOCAL ] KBH: Trimmed 70715 records to 48338 (keeping 200 days of data) [2025-11-04T19:45:32.984439] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-01. [2025-11-04T19:45:33.101576] [DEBUG] [LOCAL ] KBH: Primary data saved (48338 records). [2025-11-04T19:45:33.101730] [INFO ] [MAIN ] KBH: Full history saved. [2025-11-04T19:45:33.101817] [INFO ] [MAIN ] Update Progress: 1177/2367 symbols completed. KBH took 34.9s. Estimated remaining: 2h 47m 19s. Rate: 7.1 symbols/min [2025-11-04T19:45:33.271797] [INFO ] [MAIN ] KEP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:45:33.302727] [DEBUG] [LOCAL ] KEP: Loaded 32968 existing records from primary storage. [2025-11-04T19:45:33.303025] [INFO ] [MAIN ] KEP: Loaded 32968 existing records. [2025-11-04T19:45:33.465561] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-01 (0.48 seconds). [2025-11-04T19:45:33.487307] [WARN ] [MAIN ] KEP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:45:33.489669] [INFO ] [MAIN ] KEP: Wiping existing primary data before full refresh. [2025-11-04T19:45:33.489782] [INFO ] [LOCAL ] KEP: Starting deletion of primary data. [2025-11-04T19:45:33.490436] [DEBUG] [LOCAL ] KEP: Deleted primary data file. [2025-11-04T19:45:33.490498] [INFO ] [MAIN ] KEP: Starting full primary refresh. [2025-11-04T19:45:33.771844] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-02. [2025-11-04T19:45:33.877225] [INFO ] [REMOTE ] KEP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:45:33.879016] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-11-04T19:45:34.166454] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-11-04T19:45:34.567027] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-03. [2025-11-04T19:45:35.367040] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-03. [2025-11-04T19:45:35.746075] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-11-04T19:45:36.167018] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-01. [2025-11-04T19:45:36.217579] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-11-04T19:45:36.967040] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-04. [2025-11-04T19:45:37.328794] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T19:45:37.767130] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-02. [2025-11-04T19:45:38.495410] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-11-04T19:45:38.525337] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-11-04T19:45:38.567014] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-03. [2025-11-04T19:45:39.222231] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-11-04T19:45:39.367060] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-04. [2025-11-04T19:45:40.118529] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T19:45:40.167002] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-05. [2025-11-04T19:45:40.967028] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-05. [2025-11-04T19:45:41.101939] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-11-04T19:45:41.767087] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-04. [2025-11-04T19:45:42.416827] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-11-04T19:45:42.416548] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T19:45:42.567063] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-06. [2025-11-04T19:45:43.367056] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-05. [2025-11-04T19:45:43.720021] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-11-04T19:45:43.854158] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-11-04T19:45:44.167035] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-06. [2025-11-04T19:45:44.564865] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-11-04T19:45:44.967069] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-06. [2025-11-04T19:45:45.767047] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-07. [2025-11-04T19:45:46.059493] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-11-04T19:45:46.206809] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-11-04T19:45:46.567043] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-06. [2025-11-04T19:45:47.367060] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-08. [2025-11-04T19:45:47.809252] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T19:45:47.880580] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-11-04T19:45:48.167045] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-07. [2025-11-04T19:45:48.967044] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-09. [2025-11-04T19:45:49.314277] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-11-04T19:45:49.332077] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-11-04T19:45:49.767030] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-08. [2025-11-04T19:45:50.567036] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-07. [2025-11-04T19:45:50.810086] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T19:45:51.367034] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-07. [2025-11-04T19:45:52.129107] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-11-04T19:45:52.166999] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-08. [2025-11-04T19:45:52.831174] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T19:45:52.967018] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-10. [2025-11-04T19:45:53.315410] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-10 (0.35 seconds). [2025-11-04T19:45:53.751220] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-11-04T19:45:53.767010] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-09. [2025-11-04T19:45:54.567020] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-08. [2025-11-04T19:45:54.649032] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T19:45:55.367069] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-10. [2025-11-04T19:45:55.691520] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-11-04T19:45:56.167039] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-09. [2025-11-04T19:45:56.342022] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T19:45:56.967036] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-11. [2025-11-04T19:45:57.640234] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-11-04T19:45:57.767053] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-10. [2025-11-04T19:45:58.098133] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:45:58.099781] [INFO ] [REMOTE ] KEP: Full fetch complete (43047 records). [2025-11-04T19:45:58.099895] [DEBUG] [VALIDATOR ] KEP: Validating primary data. [2025-11-04T19:45:58.477385] [DEBUG] [VALIDATOR ] KEP: All expected trading days are present. [2025-11-04T19:45:58.477474] [DEBUG] [VALIDATOR ] KEP: Primary data validation passed. [2025-11-04T19:45:58.478175] [INFO ] [MAIN ] KEP: Fetched primary data is valid. [2025-11-04T19:45:58.478239] [DEBUG] [LOCAL ] KEP: Saving primary data. [2025-11-04T19:45:58.479274] [DEBUG] [LOCAL ] KEP: Trimmed 43047 records to 33069 (keeping 200 days of data) [2025-11-04T19:45:58.568015] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-09. [2025-11-04T19:45:58.672526] [DEBUG] [LOCAL ] KEP: Primary data saved (33069 records). [2025-11-04T19:45:58.672694] [INFO ] [MAIN ] KEP: Full history saved. [2025-11-04T19:45:58.672790] [INFO ] [MAIN ] Update Progress: 1178/2367 symbols completed. KEP took 25.6s. Estimated remaining: 2h 47m 28s. Rate: 7.1 symbols/min [2025-11-04T19:45:58.840109] [INFO ] [MAIN ] KEX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:45:58.874570] [DEBUG] [LOCAL ] KEX: Loaded 39011 existing records from primary storage. [2025-11-04T19:45:58.874880] [INFO ] [MAIN ] KEX: Loaded 39011 existing records. [2025-11-04T19:45:59.055484] [WARN ] [MAIN ] KEX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:45:59.055704] [INFO ] [MAIN ] KEX: Wiping existing primary data before full refresh. [2025-11-04T19:45:59.055767] [INFO ] [LOCAL ] KEX: Starting deletion of primary data. [2025-11-04T19:45:59.056391] [DEBUG] [LOCAL ] KEX: Deleted primary data file. [2025-11-04T19:45:59.056452] [INFO ] [MAIN ] KEX: Starting full primary refresh. [2025-11-04T19:45:59.371956] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-11. [2025-11-04T19:45:59.414069] [INFO ] [REMOTE ] KEX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:45:59.468972] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-11-04T19:45:59.819990] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-11 (0.45 seconds). [2025-11-04T19:45:59.820755] [INFO ] [REMOTE ] KEN: Full fetch complete (5785 records). [2025-11-04T19:45:59.820826] [DEBUG] [VALIDATOR ] KEN: Validating primary data. [2025-11-04T19:46:00.171525] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-11. [2025-11-04T19:46:00.213341] [DEBUG] [VALIDATOR ] KEN: All expected trading days are present. [2025-11-04T19:46:00.213517] [DEBUG] [VALIDATOR ] KEN: Primary data validation passed. [2025-11-04T19:46:00.215621] [INFO ] [MAIN ] KEN: Fetched primary data is valid. [2025-11-04T19:46:00.215409] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-11-04T19:46:00.215977] [DEBUG] [LOCAL ] KEN: Saving primary data. [2025-11-04T19:46:00.226454] [DEBUG] [LOCAL ] KEN: Trimmed 5785 records to 3940 (keeping 200 days of data) [2025-11-04T19:46:00.251252] [DEBUG] [LOCAL ] KEN: Primary data saved (3940 records). [2025-11-04T19:46:00.251377] [INFO ] [MAIN ] KEN: Full history saved. [2025-11-04T19:46:00.251460] [INFO ] [MAIN ] Update Progress: 1179/2367 symbols completed. KEN took 31.8s. Estimated remaining: 2h 47m 13s. Rate: 7.1 symbols/min [2025-11-04T19:46:00.421092] [INFO ] [MAIN ] KEY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:46:00.421290] [DEBUG] [LOCAL ] KEY: No existing data file found in primary storage. [2025-11-04T19:46:00.421327] [WARN ] [MAIN ] KEY: No existing data found, performing full refresh instead. [2025-11-04T19:46:00.421443] [INFO ] [MAIN ] KEY: Wiping existing primary data before full refresh. [2025-11-04T19:46:00.421520] [INFO ] [LOCAL ] KEY: Starting deletion of primary data. [2025-11-04T19:46:00.421570] [DEBUG] [LOCAL ] KEY: No primary data file found to delete. [2025-11-04T19:46:00.421603] [INFO ] [MAIN ] KEY: Starting full primary refresh. [2025-11-04T19:46:00.809745] [INFO ] [REMOTE ] KEY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:46:00.967036] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-10. [2025-11-04T19:46:01.574689] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-11 (1.40 seconds). [2025-11-04T19:46:01.577068] [INFO ] [REMOTE ] KD: Full fetch complete (76067 records). [2025-11-04T19:46:01.577234] [DEBUG] [VALIDATOR ] KD: Validating primary data. [2025-11-04T19:46:01.772234] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-01. [2025-11-04T19:46:02.052685] [DEBUG] [VALIDATOR ] KD: All expected trading days are present. [2025-11-04T19:46:02.052773] [DEBUG] [VALIDATOR ] KD: Primary data validation passed. [2025-11-04T19:46:02.053707] [INFO ] [MAIN ] KD: Fetched primary data is valid. [2025-11-04T19:46:02.053769] [DEBUG] [LOCAL ] KD: Saving primary data. [2025-11-04T19:46:02.055127] [DEBUG] [LOCAL ] KD: Trimmed 76067 records to 52289 (keeping 200 days of data) [2025-11-04T19:46:02.368463] [DEBUG] [LOCAL ] KD: Primary data saved (52289 records). [2025-11-04T19:46:02.368681] [INFO ] [MAIN ] KD: Full history saved. [2025-11-04T19:46:02.368818] [INFO ] [MAIN ] Update Progress: 1180/2367 symbols completed. KD took 36.2s. Estimated remaining: 2h 46m 58s. Rate: 7.1 symbols/min [2025-11-04T19:46:02.537444] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-11-04T19:46:02.570145] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-01. [2025-11-04T19:46:02.624072] [INFO ] [MAIN ] KEYS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:46:02.662637] [DEBUG] [LOCAL ] KEYS: Loaded 42970 existing records from primary storage. [2025-11-04T19:46:02.662992] [INFO ] [MAIN ] KEYS: Loaded 42970 existing records. [2025-11-04T19:46:02.837597] [WARN ] [MAIN ] KEYS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:46:02.837889] [INFO ] [MAIN ] KEYS: Wiping existing primary data before full refresh. [2025-11-04T19:46:02.838053] [INFO ] [LOCAL ] KEYS: Starting deletion of primary data. [2025-11-04T19:46:02.838954] [DEBUG] [LOCAL ] KEYS: Deleted primary data file. [2025-11-04T19:46:02.839013] [INFO ] [MAIN ] KEYS: Starting full primary refresh. [2025-11-04T19:46:03.181674] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T19:46:03.228964] [INFO ] [REMOTE ] KEYS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:46:03.367012] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-02. [2025-11-04T19:46:03.924423] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T19:46:04.167060] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-01. [2025-11-04T19:46:04.507132] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T19:46:04.967050] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-11. [2025-11-04T19:46:05.405559] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-11-04T19:46:05.767048] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-03. [2025-11-04T19:46:06.147631] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-11 (1.18 seconds). [2025-11-04T19:46:06.149784] [INFO ] [REMOTE ] KBR: Full fetch complete (70335 records). [2025-11-04T19:46:06.149920] [DEBUG] [VALIDATOR ] KBR: Validating primary data. [2025-11-04T19:46:06.572127] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-02. [2025-11-04T19:46:06.635536] [DEBUG] [VALIDATOR ] KBR: All expected trading days are present. [2025-11-04T19:46:06.635625] [DEBUG] [VALIDATOR ] KBR: Primary data validation passed. [2025-11-04T19:46:06.636766] [INFO ] [MAIN ] KBR: Fetched primary data is valid. [2025-11-04T19:46:06.636919] [DEBUG] [LOCAL ] KBR: Saving primary data. [2025-11-04T19:46:06.638488] [DEBUG] [LOCAL ] KBR: Trimmed 70335 records to 48166 (keeping 200 days of data) [2025-11-04T19:46:06.961704] [DEBUG] [LOCAL ] KBR: Primary data saved (48166 records). [2025-11-04T19:46:06.961910] [INFO ] [MAIN ] KBR: Full history saved. [2025-11-04T19:46:06.962118] [INFO ] [MAIN ] Update Progress: 1181/2367 symbols completed. KBR took 45.4s. Estimated remaining: 2h 46m 46s. Rate: 7.1 symbols/min [2025-11-04T19:46:07.052509] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-11-04T19:46:07.184293] [INFO ] [MAIN ] KF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:46:07.189301] [DEBUG] [LOCAL ] KF: Loaded 2226 existing records from primary storage. [2025-11-04T19:46:07.189535] [INFO ] [MAIN ] KF: Loaded 2226 existing records. [2025-11-04T19:46:07.372365] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-04. [2025-11-04T19:46:07.377972] [WARN ] [MAIN ] KF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:46:07.379136] [INFO ] [MAIN ] KF: Wiping existing primary data before full refresh. [2025-11-04T19:46:07.379213] [INFO ] [LOCAL ] KF: Starting deletion of primary data. [2025-11-04T19:46:07.379430] [DEBUG] [LOCAL ] KF: Deleted primary data file. [2025-11-04T19:46:07.379508] [INFO ] [MAIN ] KF: Starting full primary refresh. [2025-11-04T19:46:07.746281] [INFO ] [REMOTE ] KF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:46:08.043187] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-11-04T19:46:08.167018] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-02. [2025-11-04T19:46:08.608432] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T19:46:08.967046] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-01. [2025-11-04T19:46:09.518628] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T19:46:09.648207] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-11-04T19:46:09.767084] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-05. [2025-11-04T19:46:10.567038] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-03. [2025-11-04T19:46:10.981140] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-11-04T19:46:11.367059] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-03. [2025-11-04T19:46:11.915543] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-11-04T19:46:12.167070] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-04. [2025-11-04T19:46:12.952946] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-11-04T19:46:12.966993] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-04. [2025-11-04T19:46:13.555935] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-11-04T19:46:13.767085] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-05. [2025-11-04T19:46:14.568406] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-06. [2025-11-04T19:46:14.571405] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-11-04T19:46:14.968203] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T19:46:15.367097] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-06. [2025-11-04T19:46:15.488076] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-11-04T19:46:16.167040] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-02. [2025-11-04T19:46:16.227596] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-11-04T19:46:16.471992] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-02 (0.30 seconds). [2025-11-04T19:46:16.967088] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-05. [2025-11-04T19:46:17.767040] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-07. [2025-11-04T19:46:18.442483] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T19:46:18.567046] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-06. [2025-11-04T19:46:18.920736] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-11-04T19:46:19.367059] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-07. [2025-11-04T19:46:20.172572] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-08. [2025-11-04T19:46:20.191086] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-11-04T19:46:20.686462] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-11-04T19:46:20.967103] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-08. [2025-11-04T19:46:21.438823] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T19:46:21.767052] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-09. [2025-11-04T19:46:22.302667] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T19:46:22.567077] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-03. [2025-11-04T19:46:22.901972] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-11-04T19:46:22.905252] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T19:46:23.367048] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-04. [2025-11-04T19:46:23.745509] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-11-04T19:46:24.167078] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-07. [2025-11-04T19:46:24.967125] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-09. [2025-11-04T19:46:25.772214] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-05. [2025-11-04T19:46:25.790705] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-11-04T19:46:26.105122] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T19:46:26.172473] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-11-04T19:46:26.567074] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-06. [2025-11-04T19:46:26.926879] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-06 (0.36 seconds). [2025-11-04T19:46:27.367051] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-07. [2025-11-04T19:46:27.700675] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-11-04T19:46:28.167064] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-08. [2025-11-04T19:46:28.967049] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-10. [2025-11-04T19:46:29.561747] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-11-04T19:46:29.767042] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-09. [2025-11-04T19:46:30.192354] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-11-04T19:46:30.567074] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-11. [2025-11-04T19:46:31.028843] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-11-04T19:46:31.367050] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-10. [2025-11-04T19:46:31.901536] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-11 (1.33 seconds). [2025-11-04T19:46:31.903886] [INFO ] [REMOTE ] KEYS: Full fetch complete (63549 records). [2025-11-04T19:46:31.904111] [DEBUG] [VALIDATOR ] KEYS: Validating primary data. [2025-11-04T19:46:32.178772] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-08. [2025-11-04T19:46:32.351426] [DEBUG] [VALIDATOR ] KEYS: All expected trading days are present. [2025-11-04T19:46:32.351540] [DEBUG] [VALIDATOR ] KEYS: Primary data validation passed. [2025-11-04T19:46:32.352499] [INFO ] [MAIN ] KEYS: Fetched primary data is valid. [2025-11-04T19:46:32.352565] [DEBUG] [LOCAL ] KEYS: Saving primary data. [2025-11-04T19:46:32.353897] [DEBUG] [LOCAL ] KEYS: Trimmed 63549 records to 43016 (keeping 200 days of data) [2025-11-04T19:46:32.572308] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-11-04T19:46:32.627070] [DEBUG] [LOCAL ] KEYS: Primary data saved (43016 records). [2025-11-04T19:46:32.627295] [INFO ] [MAIN ] KEYS: Full history saved. [2025-11-04T19:46:32.627624] [INFO ] [MAIN ] Update Progress: 1182/2367 symbols completed. KEYS took 30.3s. Estimated remaining: 2h 46m 55s. Rate: 7.1 symbols/min [2025-11-04T19:46:32.766110] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-11-04T19:46:32.838340] [INFO ] [MAIN ] KFRC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:46:32.857208] [DEBUG] [LOCAL ] KFRC: Loaded 21345 existing records from primary storage. [2025-11-04T19:46:32.857450] [INFO ] [MAIN ] KFRC: Loaded 21345 existing records. [2025-11-04T19:46:32.972274] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-10. [2025-11-04T19:46:33.038466] [WARN ] [MAIN ] KFRC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:46:33.038697] [INFO ] [MAIN ] KFRC: Wiping existing primary data before full refresh. [2025-11-04T19:46:33.038759] [INFO ] [LOCAL ] KFRC: Starting deletion of primary data. [2025-11-04T19:46:33.039144] [DEBUG] [LOCAL ] KFRC: Deleted primary data file. [2025-11-04T19:46:33.039184] [INFO ] [MAIN ] KFRC: Starting full primary refresh. [2025-11-04T19:46:33.388962] [INFO ] [REMOTE ] KFRC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:46:33.767054] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-11. [2025-11-04T19:46:34.567060] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-01. [2025-11-04T19:46:34.647145] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-11-04T19:46:34.918823] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-11 (1.15 seconds). [2025-11-04T19:46:34.920589] [INFO ] [REMOTE ] KEX: Full fetch complete (56437 records). [2025-11-04T19:46:34.920745] [DEBUG] [VALIDATOR ] KEX: Validating primary data. [2025-11-04T19:46:35.197140] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-11-04T19:46:35.333982] [DEBUG] [VALIDATOR ] KEX: All expected trading days are present. [2025-11-04T19:46:35.334059] [DEBUG] [VALIDATOR ] KEX: Primary data validation passed. [2025-11-04T19:46:35.334868] [INFO ] [MAIN ] KEX: Fetched primary data is valid. [2025-11-04T19:46:35.334928] [DEBUG] [LOCAL ] KEX: Saving primary data. [2025-11-04T19:46:35.336097] [DEBUG] [LOCAL ] KEX: Trimmed 56437 records to 38960 (keeping 200 days of data) [2025-11-04T19:46:35.384521] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-09. [2025-11-04T19:46:35.561714] [DEBUG] [LOCAL ] KEX: Primary data saved (38960 records). [2025-11-04T19:46:35.561842] [INFO ] [MAIN ] KEX: Full history saved. [2025-11-04T19:46:35.561920] [INFO ] [MAIN ] Update Progress: 1183/2367 symbols completed. KEX took 36.9s. Estimated remaining: 2h 46m 41s. Rate: 7.1 symbols/min [2025-11-04T19:46:35.708410] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-11-04T19:46:35.741388] [INFO ] [MAIN ] KFS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:46:35.751023] [DEBUG] [LOCAL ] KFS: Loaded 8711 existing records from primary storage. [2025-11-04T19:46:35.751245] [INFO ] [MAIN ] KFS: Loaded 8711 existing records. [2025-11-04T19:46:35.931156] [WARN ] [MAIN ] KFS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:46:35.931380] [INFO ] [MAIN ] KFS: Wiping existing primary data before full refresh. [2025-11-04T19:46:35.931428] [INFO ] [LOCAL ] KFS: Starting deletion of primary data. [2025-11-04T19:46:35.931720] [DEBUG] [LOCAL ] KFS: Deleted primary data file. [2025-11-04T19:46:35.931779] [INFO ] [MAIN ] KFS: Starting full primary refresh. [2025-11-04T19:46:36.174104] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-02. [2025-11-04T19:46:36.296838] [INFO ] [REMOTE ] KFS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:46:36.846709] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-11-04T19:46:36.967067] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-03. [2025-11-04T19:46:37.605732] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T19:46:37.767089] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-04. [2025-11-04T19:46:38.525431] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-11-04T19:46:38.567082] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-10. [2025-11-04T19:46:38.916478] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-10 (0.35 seconds). [2025-11-04T19:46:39.367044] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-05. [2025-11-04T19:46:40.040943] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-11-04T19:46:40.167043] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-01. [2025-11-04T19:46:40.703182] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-11-04T19:46:40.967052] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-02. [2025-11-04T19:46:41.378317] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-11-04T19:46:41.767053] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-03. [2025-11-04T19:46:42.213461] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-11-04T19:46:42.567040] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-11. [2025-11-04T19:46:42.908879] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T19:46:42.910944] [INFO ] [REMOTE ] KEY: Full fetch complete (77426 records). [2025-11-04T19:46:42.911069] [DEBUG] [VALIDATOR ] KEY: Validating primary data. [2025-11-04T19:46:43.130283] [WARN ] [VALIDATOR ] KEY: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-11-04T19:46:43.131495] [FATAL] [MAIN ] KEY: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:46:43.131589] [WARN ] [MAIN ] KEY: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:46:43.131741] [INFO ] [LOCAL ] KEY: Starting deletion of primary data. [2025-11-04T19:46:43.131828] [DEBUG] [LOCAL ] KEY: No primary data file found to delete. [2025-11-04T19:46:43.131869] [INFO ] [LOCAL ] KEY: Starting deletion of derivative data. [2025-11-04T19:46:43.131911] [DEBUG] [LOCAL ] KEY: No derivative data file found to delete. [2025-11-04T19:46:43.131986] [INFO ] [MAIN ] Update Progress: 1184/2367 symbols completed. KEY took 42.9s. Estimated remaining: 2h 46m 31s. Rate: 7.1 symbols/min [2025-11-04T19:46:43.312161] [INFO ] [MAIN ] KFY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:46:43.339999] [DEBUG] [LOCAL ] KFY: Loaded 29178 existing records from primary storage. [2025-11-04T19:46:43.340285] [INFO ] [MAIN ] KFY: Loaded 29178 existing records. [2025-11-04T19:46:43.372183] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-04. [2025-11-04T19:46:43.525619] [WARN ] [MAIN ] KFY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:46:43.525886] [INFO ] [MAIN ] KFY: Wiping existing primary data before full refresh. [2025-11-04T19:46:43.525935] [INFO ] [LOCAL ] KFY: Starting deletion of primary data. [2025-11-04T19:46:43.526463] [DEBUG] [LOCAL ] KFY: Deleted primary data file. [2025-11-04T19:46:43.526521] [INFO ] [MAIN ] KFY: Starting full primary refresh. [2025-11-04T19:46:43.872410] [INFO ] [REMOTE ] KFY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:46:44.021584] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-11-04T19:46:44.167043] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-11. [2025-11-04T19:46:44.573461] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-11 (0.41 seconds). [2025-11-04T19:46:44.574164] [INFO ] [REMOTE ] KF: Full fetch complete (3423 records). [2025-11-04T19:46:44.574230] [DEBUG] [VALIDATOR ] KF: Validating primary data. [2025-11-04T19:46:44.957330] [DEBUG] [VALIDATOR ] KF: All expected trading days are present. [2025-11-04T19:46:44.957409] [DEBUG] [VALIDATOR ] KF: Primary data validation passed. [2025-11-04T19:46:44.957559] [INFO ] [MAIN ] KF: Fetched primary data is valid. [2025-11-04T19:46:44.957599] [DEBUG] [LOCAL ] KF: Saving primary data. [2025-11-04T19:46:44.957975] [DEBUG] [LOCAL ] KF: Trimmed 3423 records to 2230 (keeping 200 days of data) [2025-11-04T19:46:44.968183] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-06. [2025-11-04T19:46:44.972783] [DEBUG] [LOCAL ] KF: Primary data saved (2230 records). [2025-11-04T19:46:44.972848] [INFO ] [MAIN ] KF: Full history saved. [2025-11-04T19:46:44.972919] [INFO ] [MAIN ] Update Progress: 1185/2367 symbols completed. KF took 38.0s. Estimated remaining: 2h 46m 16s. Rate: 7.1 symbols/min [2025-11-04T19:46:45.161129] [INFO ] [MAIN ] KGC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:46:45.216052] [DEBUG] [LOCAL ] KGC: Loaded 53699 existing records from primary storage. [2025-11-04T19:46:45.216458] [INFO ] [MAIN ] KGC: Loaded 53699 existing records. [2025-11-04T19:46:45.406719] [WARN ] [MAIN ] KGC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:46:45.406920] [INFO ] [MAIN ] KGC: Wiping existing primary data before full refresh. [2025-11-04T19:46:45.406966] [INFO ] [LOCAL ] KGC: Starting deletion of primary data. [2025-11-04T19:46:45.407454] [DEBUG] [LOCAL ] KGC: Deleted primary data file. [2025-11-04T19:46:45.407491] [INFO ] [MAIN ] KGC: Starting full primary refresh. [2025-11-04T19:46:45.755126] [INFO ] [REMOTE ] KGC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:46:45.764202] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-11-04T19:46:45.767324] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-05. [2025-11-04T19:46:46.303030] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-11-04T19:46:46.567047] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-01. [2025-11-04T19:46:47.367019] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-07. [2025-11-04T19:46:48.037716] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-11-04T19:46:48.132512] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-11-04T19:46:48.167009] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-08. [2025-11-04T19:46:48.967017] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-06. [2025-11-04T19:46:48.988052] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-11-04T19:46:49.518196] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-11-04T19:46:49.767042] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-02. [2025-11-04T19:46:50.567039] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-07. [2025-11-04T19:46:51.157800] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-11-04T19:46:51.174389] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-11-04T19:46:51.367032] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-03. [2025-11-04T19:46:52.167032] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-08. [2025-11-04T19:46:52.655366] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-11-04T19:46:52.915547] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-11-04T19:46:52.966996] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-04. [2025-11-04T19:46:53.767023] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-01. [2025-11-04T19:46:54.575510] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-09. [2025-11-04T19:46:54.581627] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T19:46:54.806408] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-11-04T19:46:55.367020] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-02. [2025-11-04T19:46:55.459452] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T19:46:56.167021] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-10. [2025-11-04T19:46:56.315997] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-11-04T19:46:56.967006] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-09. [2025-11-04T19:46:57.132460] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-11-04T19:46:57.367290] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-11-04T19:46:57.767088] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-10. [2025-11-04T19:46:58.368887] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-11-04T19:46:58.567047] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-11. [2025-11-04T19:46:59.108507] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T19:46:59.109321] [INFO ] [REMOTE ] KFS: Full fetch complete (11453 records). [2025-11-04T19:46:59.109389] [DEBUG] [VALIDATOR ] KFS: Validating primary data. [2025-11-04T19:46:59.372180] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-03. [2025-11-04T19:46:59.471905] [DEBUG] [VALIDATOR ] KFS: All expected trading days are present. [2025-11-04T19:46:59.471985] [DEBUG] [VALIDATOR ] KFS: Primary data validation passed. [2025-11-04T19:46:59.472252] [INFO ] [MAIN ] KFS: Fetched primary data is valid. [2025-11-04T19:46:59.472312] [DEBUG] [LOCAL ] KFS: Saving primary data. [2025-11-04T19:46:59.472797] [DEBUG] [LOCAL ] KFS: Trimmed 11453 records to 8716 (keeping 200 days of data) [2025-11-04T19:46:59.522147] [DEBUG] [LOCAL ] KFS: Primary data saved (8716 records). [2025-11-04T19:46:59.522279] [INFO ] [MAIN ] KFS: Full history saved. [2025-11-04T19:46:59.522357] [INFO ] [MAIN ] Update Progress: 1186/2367 symbols completed. KFS took 24.0s. Estimated remaining: 2h 46m 14s. Rate: 7.1 symbols/min [2025-11-04T19:46:59.685855] [INFO ] [MAIN ] KGS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:46:59.725108] [DEBUG] [LOCAL ] KGS: Loaded 45854 existing records from primary storage. [2025-11-04T19:46:59.725401] [INFO ] [MAIN ] KGS: Loaded 45854 existing records. [2025-11-04T19:46:59.899981] [WARN ] [MAIN ] KGS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:46:59.900221] [INFO ] [MAIN ] KGS: Wiping existing primary data before full refresh. [2025-11-04T19:46:59.900271] [INFO ] [LOCAL ] KGS: Starting deletion of primary data. [2025-11-04T19:46:59.900913] [DEBUG] [LOCAL ] KGS: Deleted primary data file. [2025-11-04T19:46:59.900969] [INFO ] [MAIN ] KGS: Starting full primary refresh. [2025-11-04T19:47:00.172279] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-11. [2025-11-04T19:47:00.300145] [INFO ] [REMOTE ] KGS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:47:00.463577] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-11-04T19:47:00.967058] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-01. [2025-11-04T19:47:01.767135] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-05. [2025-11-04T19:47:02.567099] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-04. [2025-11-04T19:47:03.276688] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-11-04T19:47:03.367006] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-06. [2025-11-04T19:47:03.459257] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-11-04T19:47:04.167117] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-05. [2025-11-04T19:47:04.519755] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-11 (4.35 seconds). [2025-11-04T19:47:04.521236] [INFO ] [REMOTE ] KFRC: Full fetch complete (28232 records). [2025-11-04T19:47:04.521303] [DEBUG] [VALIDATOR ] KFRC: Validating primary data. [2025-11-04T19:47:04.888833] [DEBUG] [VALIDATOR ] KFRC: All expected trading days are present. [2025-11-04T19:47:04.888936] [DEBUG] [VALIDATOR ] KFRC: Primary data validation passed. [2025-11-04T19:47:04.889563] [INFO ] [MAIN ] KFRC: Fetched primary data is valid. [2025-11-04T19:47:04.889696] [DEBUG] [LOCAL ] KFRC: Saving primary data. [2025-11-04T19:47:04.891102] [DEBUG] [LOCAL ] KFRC: Trimmed 28232 records to 21551 (keeping 200 days of data) [2025-11-04T19:47:05.024308] [DEBUG] [LOCAL ] KFRC: Primary data saved (21551 records). [2025-11-04T19:47:05.024540] [INFO ] [MAIN ] KFRC: Full history saved. [2025-11-04T19:47:05.024846] [INFO ] [MAIN ] Update Progress: 1187/2367 symbols completed. KFRC took 32.4s. Estimated remaining: 2h 46m 3s. Rate: 7.1 symbols/min [2025-11-04T19:47:05.143682] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-11-04T19:47:05.239636] [INFO ] [MAIN ] KIM: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:47:05.240614] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-07. [2025-11-04T19:47:05.258100] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-11-04T19:47:05.292669] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-01 (4.33 seconds). [2025-11-04T19:47:05.340479] [DEBUG] [LOCAL ] KIM: Loaded 52990 existing records from primary storage. [2025-11-04T19:47:05.340876] [INFO ] [MAIN ] KIM: Loaded 52990 existing records. [2025-11-04T19:47:05.525867] [WARN ] [MAIN ] KIM: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:47:05.526063] [INFO ] [MAIN ] KIM: Wiping existing primary data before full refresh. [2025-11-04T19:47:05.526112] [INFO ] [LOCAL ] KIM: Starting deletion of primary data. [2025-11-04T19:47:05.527070] [DEBUG] [LOCAL ] KIM: Deleted primary data file. [2025-11-04T19:47:05.527129] [INFO ] [MAIN ] KIM: Starting full primary refresh. [2025-11-04T19:47:05.772634] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-02. [2025-11-04T19:47:05.893206] [INFO ] [REMOTE ] KIM: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:47:06.567019] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-06. [2025-11-04T19:47:06.772315] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T19:47:06.956263] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-11-04T19:47:07.367011] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-08. [2025-11-04T19:47:07.551826] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-11-04T19:47:08.167042] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-01. [2025-11-04T19:47:08.804367] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-11-04T19:47:08.967070] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-07. [2025-11-04T19:47:09.519432] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T19:47:09.767048] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-02. [2025-11-04T19:47:09.999310] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-11-04T19:47:10.567055] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-09. [2025-11-04T19:47:10.980524] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-11-04T19:47:11.367040] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-03. [2025-11-04T19:47:12.154601] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-11-04T19:47:12.166995] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-10. [2025-11-04T19:47:12.746312] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-11-04T19:47:12.967044] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-03. [2025-11-04T19:47:13.767096] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-08. [2025-11-04T19:47:13.969809] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-11-04T19:47:14.165116] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T19:47:14.443361] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-11-04T19:47:14.567013] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-04. [2025-11-04T19:47:15.367074] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-04. [2025-11-04T19:47:15.653748] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T19:47:16.167091] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-05. [2025-11-04T19:47:16.630778] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T19:47:16.967060] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-05. [2025-11-04T19:47:17.275722] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T19:47:17.767081] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-06. [2025-11-04T19:47:18.260198] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-11-04T19:47:18.567058] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-09. [2025-11-04T19:47:18.779007] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-11-04T19:47:19.367018] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-06. [2025-11-04T19:47:19.721323] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-11-04T19:47:20.167112] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-11. [2025-11-04T19:47:20.837187] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-11-04T19:47:20.967018] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-10. [2025-11-04T19:47:21.767012] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-07. [2025-11-04T19:47:21.857144] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-11-04T19:47:22.090560] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-11 (1.92 seconds). [2025-11-04T19:47:22.093367] [INFO ] [REMOTE ] KGC: Full fetch complete (77875 records). [2025-11-04T19:47:22.093554] [DEBUG] [VALIDATOR ] KGC: Validating primary data. [2025-11-04T19:47:22.517367] [DEBUG] [VALIDATOR ] KGC: All expected trading days are present. [2025-11-04T19:47:22.517489] [DEBUG] [VALIDATOR ] KGC: Primary data validation passed. [2025-11-04T19:47:22.519072] [INFO ] [MAIN ] KGC: Fetched primary data is valid. [2025-11-04T19:47:22.519150] [DEBUG] [LOCAL ] KGC: Saving primary data. [2025-11-04T19:47:22.521385] [DEBUG] [LOCAL ] KGC: Trimmed 77875 records to 53699 (keeping 200 days of data) [2025-11-04T19:47:22.580522] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-11. [2025-11-04T19:47:22.883955] [DEBUG] [LOCAL ] KGC: Primary data saved (53699 records). [2025-11-04T19:47:22.884159] [INFO ] [MAIN ] KGC: Full history saved. [2025-11-04T19:47:22.884263] [INFO ] [MAIN ] Update Progress: 1188/2367 symbols completed. KGC took 37.9s. Estimated remaining: 2h 46m 4s. Rate: 7.1 symbols/min [2025-11-04T19:47:23.065774] [INFO ] [MAIN ] KIO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:47:23.092859] [DEBUG] [LOCAL ] KIO: Loaded 21770 existing records from primary storage. [2025-11-04T19:47:23.093252] [INFO ] [MAIN ] KIO: Loaded 21770 existing records. [2025-11-04T19:47:23.292473] [WARN ] [MAIN ] KIO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:47:23.292896] [INFO ] [MAIN ] KIO: Wiping existing primary data before full refresh. [2025-11-04T19:47:23.293014] [INFO ] [LOCAL ] KIO: Starting deletion of primary data. [2025-11-04T19:47:23.293445] [DEBUG] [LOCAL ] KIO: Deleted primary data file. [2025-11-04T19:47:23.293531] [INFO ] [MAIN ] KIO: Starting full primary refresh. [2025-11-04T19:47:23.415570] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-07. [2025-11-04T19:47:23.448365] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-11-04T19:47:23.618050] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T19:47:23.642130] [INFO ] [REMOTE ] KFY: Full fetch complete (43412 records). [2025-11-04T19:47:23.647468] [DEBUG] [VALIDATOR ] KFY: Validating primary data. [2025-11-04T19:47:23.903384] [INFO ] [REMOTE ] KIO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:47:24.168221] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-01. [2025-11-04T19:47:24.183896] [DEBUG] [VALIDATOR ] KFY: All expected trading days are present. [2025-11-04T19:47:24.183996] [DEBUG] [VALIDATOR ] KFY: Primary data validation passed. [2025-11-04T19:47:24.184831] [INFO ] [MAIN ] KFY: Fetched primary data is valid. [2025-11-04T19:47:24.185046] [DEBUG] [LOCAL ] KFY: Saving primary data. [2025-11-04T19:47:24.186040] [DEBUG] [LOCAL ] KFY: Trimmed 43412 records to 29103 (keeping 200 days of data) [2025-11-04T19:47:24.362717] [DEBUG] [LOCAL ] KFY: Primary data saved (29103 records). [2025-11-04T19:47:24.362880] [INFO ] [MAIN ] KFY: Full history saved. [2025-11-04T19:47:24.362988] [INFO ] [MAIN ] Update Progress: 1189/2367 symbols completed. KFY took 41.2s. Estimated remaining: 2h 45m 48s. Rate: 7.1 symbols/min [2025-11-04T19:47:24.533687] [INFO ] [MAIN ] KKR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:47:24.579150] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-11-04T19:47:24.628047] [DEBUG] [LOCAL ] KKR: Loaded 53492 existing records from primary storage. [2025-11-04T19:47:24.628403] [INFO ] [MAIN ] KKR: Loaded 53492 existing records. [2025-11-04T19:47:24.801202] [WARN ] [MAIN ] KKR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:47:24.801418] [INFO ] [MAIN ] KKR: Wiping existing primary data before full refresh. [2025-11-04T19:47:24.801466] [INFO ] [LOCAL ] KKR: Starting deletion of primary data. [2025-11-04T19:47:24.801811] [DEBUG] [LOCAL ] KKR: Deleted primary data file. [2025-11-04T19:47:24.801867] [INFO ] [MAIN ] KKR: Starting full primary refresh. [2025-11-04T19:47:24.971201] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-08. [2025-11-04T19:47:25.169782] [INFO ] [REMOTE ] KKR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:47:25.189384] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-11-04T19:47:25.767083] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-01. [2025-11-04T19:47:26.164681] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T19:47:26.567080] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-08. [2025-11-04T19:47:27.320805] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-11-04T19:47:27.367007] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-02. [2025-11-04T19:47:27.819631] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-11-04T19:47:28.167041] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-09. [2025-11-04T19:47:28.631583] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T19:47:28.967088] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-03. [2025-11-04T19:47:29.474349] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-11-04T19:47:29.767100] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-10. [2025-11-04T19:47:30.567055] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-09. [2025-11-04T19:47:30.693259] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-11-04T19:47:31.115058] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T19:47:31.367022] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-11. [2025-11-04T19:47:31.976038] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-11-04T19:47:32.167004] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-10. [2025-11-04T19:47:32.455902] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T19:47:32.458120] [INFO ] [REMOTE ] KIM: Full fetch complete (76986 records). [2025-11-04T19:47:32.458226] [DEBUG] [VALIDATOR ] KIM: Validating primary data. [2025-11-04T19:47:32.951782] [DEBUG] [VALIDATOR ] KIM: All expected trading days are present. [2025-11-04T19:47:32.951909] [DEBUG] [VALIDATOR ] KIM: Primary data validation passed. [2025-11-04T19:47:32.953836] [INFO ] [MAIN ] KIM: Fetched primary data is valid. [2025-11-04T19:47:32.953969] [DEBUG] [LOCAL ] KIM: Saving primary data. [2025-11-04T19:47:32.955999] [DEBUG] [LOCAL ] KIM: Trimmed 76986 records to 52990 (keeping 200 days of data) [2025-11-04T19:47:33.032280] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-02. [2025-11-04T19:47:33.383066] [DEBUG] [LOCAL ] KIM: Primary data saved (52990 records). [2025-11-04T19:47:33.383262] [INFO ] [MAIN ] KIM: Full history saved. [2025-11-04T19:47:33.383354] [INFO ] [MAIN ] Update Progress: 1190/2367 symbols completed. KIM took 28.4s. Estimated remaining: 2h 45m 40s. Rate: 7.1 symbols/min [2025-11-04T19:47:33.590734] [INFO ] [MAIN ] KKRS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:47:33.614873] [DEBUG] [LOCAL ] KKRS: Loaded 5491 existing records from primary storage. [2025-11-04T19:47:33.630903] [INFO ] [MAIN ] KKRS: Loaded 5491 existing records. [2025-11-04T19:47:33.638779] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-11-04T19:47:33.767497] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-11. [2025-11-04T19:47:33.842470] [WARN ] [MAIN ] KKRS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:47:33.842797] [INFO ] [MAIN ] KKRS: Wiping existing primary data before full refresh. [2025-11-04T19:47:33.842862] [INFO ] [LOCAL ] KKRS: Starting deletion of primary data. [2025-11-04T19:47:33.848406] [DEBUG] [LOCAL ] KKRS: Deleted primary data file. [2025-11-04T19:47:33.856206] [INFO ] [MAIN ] KKRS: Starting full primary refresh. [2025-11-04T19:47:33.856624] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-11-04T19:47:34.226064] [INFO ] [REMOTE ] KKRS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:47:34.567052] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-01. [2025-11-04T19:47:34.775818] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-11 (1.01 seconds). [2025-11-04T19:47:34.777593] [INFO ] [REMOTE ] KGS: Full fetch complete (66615 records). [2025-11-04T19:47:34.777703] [DEBUG] [VALIDATOR ] KGS: Validating primary data. [2025-11-04T19:47:35.085946] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-11-04T19:47:35.181805] [DEBUG] [VALIDATOR ] KGS: All expected trading days are present. [2025-11-04T19:47:35.181892] [DEBUG] [VALIDATOR ] KGS: Primary data validation passed. [2025-11-04T19:47:35.182761] [INFO ] [MAIN ] KGS: Fetched primary data is valid. [2025-11-04T19:47:35.182824] [DEBUG] [LOCAL ] KGS: Saving primary data. [2025-11-04T19:47:35.184149] [DEBUG] [LOCAL ] KGS: Trimmed 66615 records to 45870 (keeping 200 days of data) [2025-11-04T19:47:35.388046] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-02. [2025-11-04T19:47:35.461242] [DEBUG] [LOCAL ] KGS: Primary data saved (45870 records). [2025-11-04T19:47:35.461382] [INFO ] [MAIN ] KGS: Full history saved. [2025-11-04T19:47:35.461463] [INFO ] [MAIN ] Update Progress: 1191/2367 symbols completed. KGS took 35.9s. Estimated remaining: 2h 45m 26s. Rate: 7.1 symbols/min [2025-11-04T19:47:35.632638] [INFO ] [MAIN ] KKRT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:47:35.632849] [DEBUG] [LOCAL ] KKRT: No existing data file found in primary storage. [2025-11-04T19:47:35.632885] [WARN ] [MAIN ] KKRT: No existing data found, performing full refresh instead. [2025-11-04T19:47:35.632992] [INFO ] [MAIN ] KKRT: Wiping existing primary data before full refresh. [2025-11-04T19:47:35.633031] [INFO ] [LOCAL ] KKRT: Starting deletion of primary data. [2025-11-04T19:47:35.633072] [DEBUG] [LOCAL ] KKRT: No primary data file found to delete. [2025-11-04T19:47:35.633145] [INFO ] [MAIN ] KKRT: Starting full primary refresh. [2025-11-04T19:47:35.913393] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-11-04T19:47:36.015513] [INFO ] [REMOTE ] KKRT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:47:36.167066] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-04. [2025-11-04T19:47:36.967088] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-03. [2025-11-04T19:47:37.639317] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-11-04T19:47:37.744258] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-11-04T19:47:37.767069] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-03. [2025-11-04T19:47:38.071244] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-03 (0.30 seconds). [2025-11-04T19:47:38.567059] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-05. [2025-11-04T19:47:39.367042] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-01. [2025-11-04T19:47:39.514848] [DEBUG] [REMOTE ] KKRT: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T19:47:39.515397] [WARN ] [REMOTE ] KKRT: No data returned for month 2025-01 [2025-11-04T19:47:39.515850] [FATAL] [REMOTE ] KKRT: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:47:39.515958] [FATAL] [MAIN ] KKRT: Failed to fetch primary data. [2025-11-04T19:47:39.516002] [WARN ] [MAIN ] KKRT: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:47:39.516045] [INFO ] [LOCAL ] KKRT: Starting deletion of primary data. [2025-11-04T19:47:39.516114] [DEBUG] [LOCAL ] KKRT: No primary data file found to delete. [2025-11-04T19:47:39.516160] [INFO ] [LOCAL ] KKRT: Starting deletion of derivative data. [2025-11-04T19:47:39.516237] [DEBUG] [LOCAL ] KKRT: No derivative data file found to delete. [2025-11-04T19:47:39.516299] [INFO ] [MAIN ] Update Progress: 1192/2367 symbols completed. KKRT took 4.1s. Estimated remaining: 2h 45m 13s. Rate: 7.1 symbols/min [2025-11-04T19:47:39.690260] [INFO ] [MAIN ] KLAR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:47:39.690462] [DEBUG] [LOCAL ] KLAR: No existing data file found in primary storage. [2025-11-04T19:47:39.690505] [WARN ] [MAIN ] KLAR: No existing data found, performing full refresh instead. [2025-11-04T19:47:39.690617] [INFO ] [MAIN ] KLAR: Wiping existing primary data before full refresh. [2025-11-04T19:47:39.690690] [INFO ] [LOCAL ] KLAR: Starting deletion of primary data. [2025-11-04T19:47:39.690749] [DEBUG] [LOCAL ] KLAR: No primary data file found to delete. [2025-11-04T19:47:39.690783] [INFO ] [MAIN ] KLAR: Starting full primary refresh. [2025-11-04T19:47:40.063117] [INFO ] [REMOTE ] KLAR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:47:40.098282] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-11-04T19:47:40.167053] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-04. [2025-11-04T19:47:40.671931] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-11-04T19:47:40.967083] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-06. [2025-11-04T19:47:41.767028] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-05. [2025-11-04T19:47:42.234411] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-11-04T19:47:42.239217] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T19:47:42.567047] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-01. [2025-11-04T19:47:42.712724] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T19:47:42.713031] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-01 [2025-11-04T19:47:42.713149] [FATAL] [REMOTE ] KLAR: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:47:42.713250] [FATAL] [MAIN ] KLAR: Failed to fetch primary data. [2025-11-04T19:47:42.713331] [WARN ] [MAIN ] KLAR: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:47:42.713423] [INFO ] [LOCAL ] KLAR: Starting deletion of primary data. [2025-11-04T19:47:42.713502] [DEBUG] [LOCAL ] KLAR: No primary data file found to delete. [2025-11-04T19:47:42.713539] [INFO ] [LOCAL ] KLAR: Starting deletion of derivative data. [2025-11-04T19:47:42.713578] [DEBUG] [LOCAL ] KLAR: No derivative data file found to delete. [2025-11-04T19:47:42.713623] [INFO ] [MAIN ] Update Progress: 1193/2367 symbols completed. KLAR took 3.2s. Estimated remaining: 2h 44m 59s. Rate: 7.1 symbols/min [2025-11-04T19:47:42.882240] [INFO ] [MAIN ] KLC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:47:42.916134] [DEBUG] [LOCAL ] KLC: Loaded 35768 existing records from primary storage. [2025-11-04T19:47:42.916397] [INFO ] [MAIN ] KLC: Loaded 35768 existing records. [2025-11-04T19:47:43.088725] [WARN ] [MAIN ] KLC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:47:43.089068] [INFO ] [MAIN ] KLC: Wiping existing primary data before full refresh. [2025-11-04T19:47:43.089153] [INFO ] [LOCAL ] KLC: Starting deletion of primary data. [2025-11-04T19:47:43.089779] [DEBUG] [LOCAL ] KLC: Deleted primary data file. [2025-11-04T19:47:43.089836] [INFO ] [MAIN ] KLC: Starting full primary refresh. [2025-11-04T19:47:43.372176] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-04. [2025-11-04T19:47:43.445421] [INFO ] [REMOTE ] KLC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:47:44.167024] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-07. [2025-11-04T19:47:44.188854] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-11-04T19:47:44.967047] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-06. [2025-11-04T19:47:45.362306] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-11-04T19:47:45.519092] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-11-04T19:47:45.767049] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-08. [2025-11-04T19:47:46.567033] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-05. [2025-11-04T19:47:47.093977] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-11-04T19:47:47.218571] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-11-04T19:47:47.367038] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-06. [2025-11-04T19:47:48.014211] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-11-04T19:47:48.167042] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-07. [2025-11-04T19:47:48.967016] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-07. [2025-11-04T19:47:49.015443] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-11-04T19:47:49.256420] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-07 (0.29 seconds). [2025-11-04T19:47:49.767048] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-01. [2025-11-04T19:47:50.567046] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-08. [2025-11-04T19:47:50.831474] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T19:47:51.374022] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-09. [2025-11-04T19:47:51.377975] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-11-04T19:47:52.167035] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-02. [2025-11-04T19:47:52.725296] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-11-04T19:47:52.765629] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-11-04T19:47:52.967041] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-09. [2025-11-04T19:47:53.723785] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-11-04T19:47:53.767009] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-08. [2025-11-04T19:47:54.146931] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-11-04T19:47:54.567036] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-10. [2025-11-04T19:47:55.367019] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-03. [2025-11-04T19:47:55.419152] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-11-04T19:47:56.167051] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-11. [2025-11-04T19:47:56.294080] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T19:47:56.934700] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-11 (0.77 seconds). [2025-11-04T19:47:56.935979] [INFO ] [REMOTE ] KIO: Full fetch complete (32715 records). [2025-11-04T19:47:56.936064] [DEBUG] [VALIDATOR ] KIO: Validating primary data. [2025-11-04T19:47:56.972933] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-10. [2025-11-04T19:47:57.320025] [DEBUG] [VALIDATOR ] KIO: All expected trading days are present. [2025-11-04T19:47:57.320091] [DEBUG] [VALIDATOR ] KIO: Primary data validation passed. [2025-11-04T19:47:57.320762] [INFO ] [MAIN ] KIO: Fetched primary data is valid. [2025-11-04T19:47:57.320823] [DEBUG] [LOCAL ] KIO: Saving primary data. [2025-11-04T19:47:57.321686] [DEBUG] [LOCAL ] KIO: Trimmed 32715 records to 21823 (keeping 200 days of data) [2025-11-04T19:47:57.450637] [DEBUG] [LOCAL ] KIO: Primary data saved (21823 records). [2025-11-04T19:47:57.450785] [INFO ] [MAIN ] KIO: Full history saved. [2025-11-04T19:47:57.450862] [INFO ] [MAIN ] Update Progress: 1194/2367 symbols completed. KIO took 34.6s. Estimated remaining: 2h 44m 57s. Rate: 7.1 symbols/min [2025-11-04T19:47:57.615203] [INFO ] [MAIN ] KMI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:47:57.663447] [DEBUG] [LOCAL ] KMI: Loaded 53693 existing records from primary storage. [2025-11-04T19:47:57.663777] [INFO ] [MAIN ] KMI: Loaded 53693 existing records. [2025-11-04T19:47:57.772236] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-04. [2025-11-04T19:47:57.842741] [WARN ] [MAIN ] KMI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:47:57.842938] [INFO ] [MAIN ] KMI: Wiping existing primary data before full refresh. [2025-11-04T19:47:57.842985] [INFO ] [LOCAL ] KMI: Starting deletion of primary data. [2025-11-04T19:47:57.843370] [DEBUG] [LOCAL ] KMI: Deleted primary data file. [2025-11-04T19:47:57.843426] [INFO ] [MAIN ] KMI: Starting full primary refresh. [2025-11-04T19:47:58.188257] [INFO ] [REMOTE ] KMI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:47:58.451496] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-11-04T19:47:58.567052] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-11. [2025-11-04T19:47:58.981344] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T19:47:59.367095] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-01. [2025-11-04T19:47:59.817535] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-11 (1.25 seconds). [2025-11-04T19:47:59.819613] [INFO ] [REMOTE ] KKR: Full fetch complete (77655 records). [2025-11-04T19:47:59.819732] [DEBUG] [VALIDATOR ] KKR: Validating primary data. [2025-11-04T19:48:00.172209] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-09. [2025-11-04T19:48:00.252855] [DEBUG] [VALIDATOR ] KKR: All expected trading days are present. [2025-11-04T19:48:00.252969] [DEBUG] [VALIDATOR ] KKR: Primary data validation passed. [2025-11-04T19:48:00.253963] [INFO ] [MAIN ] KKR: Fetched primary data is valid. [2025-11-04T19:48:00.254029] [DEBUG] [LOCAL ] KKR: Saving primary data. [2025-11-04T19:48:00.255738] [DEBUG] [LOCAL ] KKR: Trimmed 77655 records to 53491 (keeping 200 days of data) [2025-11-04T19:48:00.581163] [DEBUG] [LOCAL ] KKR: Primary data saved (53491 records). [2025-11-04T19:48:00.581316] [INFO ] [MAIN ] KKR: Full history saved. [2025-11-04T19:48:00.581399] [INFO ] [MAIN ] Update Progress: 1195/2367 symbols completed. KKR took 36.2s. Estimated remaining: 2h 44m 43s. Rate: 7.1 symbols/min [2025-11-04T19:48:00.753856] [INFO ] [MAIN ] KMPB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:48:00.759437] [DEBUG] [LOCAL ] KMPB: Loaded 3376 existing records from primary storage. [2025-11-04T19:48:00.759594] [INFO ] [MAIN ] KMPB: Loaded 3376 existing records. [2025-11-04T19:48:00.841921] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-11-04T19:48:00.938429] [WARN ] [MAIN ] KMPB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:48:00.938678] [INFO ] [MAIN ] KMPB: Wiping existing primary data before full refresh. [2025-11-04T19:48:00.938748] [INFO ] [LOCAL ] KMPB: Starting deletion of primary data. [2025-11-04T19:48:00.938925] [DEBUG] [LOCAL ] KMPB: Deleted primary data file. [2025-11-04T19:48:00.938980] [INFO ] [MAIN ] KMPB: Starting full primary refresh. [2025-11-04T19:48:00.972184] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-05. [2025-11-04T19:48:01.292564] [INFO ] [REMOTE ] KMPB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:48:01.385399] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-11-04T19:48:01.767058] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-02. [2025-11-04T19:48:02.018182] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-11-04T19:48:02.567060] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-01. [2025-11-04T19:48:02.991263] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-11-04T19:48:03.372127] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-10. [2025-11-04T19:48:03.389345] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-11-04T19:48:03.736766] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-11-04T19:48:04.167044] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-03. [2025-11-04T19:48:04.967101] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-02. [2025-11-04T19:48:05.223295] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-02 (0.26 seconds). [2025-11-04T19:48:05.778678] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-03. [2025-11-04T19:48:05.815914] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-11-04T19:48:06.180577] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-11-04T19:48:06.567092] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-04. [2025-11-04T19:48:06.896169] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-11-04T19:48:07.367024] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-05. [2025-11-04T19:48:07.674580] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-11-04T19:48:08.167061] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-04. [2025-11-04T19:48:08.967029] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-06. [2025-11-04T19:48:09.343797] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-11-04T19:48:09.550107] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T19:48:09.767059] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-06. [2025-11-04T19:48:10.543746] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T19:48:10.567032] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-07. [2025-11-04T19:48:11.367082] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-07. [2025-11-04T19:48:11.526980] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T19:48:11.621064] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-07 (0.25 seconds). [2025-11-04T19:48:12.167062] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-05. [2025-11-04T19:48:12.967070] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-08. [2025-11-04T19:48:13.522729] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T19:48:13.767075] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-11. [2025-11-04T19:48:14.026513] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-11-04T19:48:14.193719] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-11 (0.43 seconds). [2025-11-04T19:48:14.194736] [INFO ] [REMOTE ] KKRS: Full fetch complete (7766 records). [2025-11-04T19:48:14.194805] [DEBUG] [VALIDATOR ] KKRS: Validating primary data. [2025-11-04T19:48:14.546019] [DEBUG] [VALIDATOR ] KKRS: All expected trading days are present. [2025-11-04T19:48:14.546101] [DEBUG] [VALIDATOR ] KKRS: Primary data validation passed. [2025-11-04T19:48:14.546314] [INFO ] [MAIN ] KKRS: Fetched primary data is valid. [2025-11-04T19:48:14.546354] [DEBUG] [LOCAL ] KKRS: Saving primary data. [2025-11-04T19:48:14.546843] [DEBUG] [LOCAL ] KKRS: Trimmed 7766 records to 5536 (keeping 200 days of data) [2025-11-04T19:48:14.571149] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-09. [2025-11-04T19:48:14.581213] [DEBUG] [LOCAL ] KKRS: Primary data saved (5536 records). [2025-11-04T19:48:14.581289] [INFO ] [MAIN ] KKRS: Full history saved. [2025-11-04T19:48:14.581361] [INFO ] [MAIN ] Update Progress: 1196/2367 symbols completed. KKRS took 41.2s. Estimated remaining: 2h 44m 40s. Rate: 7.1 symbols/min [2025-11-04T19:48:14.755527] [INFO ] [MAIN ] KMPR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:48:14.789743] [DEBUG] [LOCAL ] KMPR: Loaded 40646 existing records from primary storage. [2025-11-04T19:48:14.790025] [INFO ] [MAIN ] KMPR: Loaded 40646 existing records. [2025-11-04T19:48:14.962962] [WARN ] [MAIN ] KMPR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:48:14.963111] [INFO ] [MAIN ] KMPR: Wiping existing primary data before full refresh. [2025-11-04T19:48:14.963157] [INFO ] [LOCAL ] KMPR: Starting deletion of primary data. [2025-11-04T19:48:14.963387] [DEBUG] [LOCAL ] KMPR: Deleted primary data file. [2025-11-04T19:48:14.963424] [INFO ] [MAIN ] KMPR: Starting full primary refresh. [2025-11-04T19:48:15.310884] [INFO ] [REMOTE ] KMPR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:48:15.367001] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-01. [2025-11-04T19:48:15.709139] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T19:48:16.107519] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-11-04T19:48:16.167052] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-02. [2025-11-04T19:48:16.967021] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-06. [2025-11-04T19:48:17.089455] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-11-04T19:48:17.767079] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-10. [2025-11-04T19:48:18.241529] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-11-04T19:48:18.567054] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-03. [2025-11-04T19:48:18.889057] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-11-04T19:48:19.359970] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T19:48:19.366987] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-04. [2025-11-04T19:48:20.167124] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-11. [2025-11-04T19:48:20.423769] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-11-04T19:48:20.967064] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-08. [2025-11-04T19:48:21.289049] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-11-04T19:48:21.294714] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-11 (1.13 seconds). [2025-11-04T19:48:21.296193] [INFO ] [REMOTE ] KLC: Full fetch complete (51047 records). [2025-11-04T19:48:21.296271] [DEBUG] [VALIDATOR ] KLC: Validating primary data. [2025-11-04T19:48:21.681603] [DEBUG] [VALIDATOR ] KLC: All expected trading days are present. [2025-11-04T19:48:21.681715] [DEBUG] [VALIDATOR ] KLC: Primary data validation passed. [2025-11-04T19:48:21.682456] [INFO ] [MAIN ] KLC: Fetched primary data is valid. [2025-11-04T19:48:21.682516] [DEBUG] [LOCAL ] KLC: Saving primary data. [2025-11-04T19:48:21.683485] [DEBUG] [LOCAL ] KLC: Trimmed 51047 records to 35816 (keeping 200 days of data) [2025-11-04T19:48:21.772687] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-09. [2025-11-04T19:48:21.885126] [DEBUG] [LOCAL ] KLC: Primary data saved (35816 records). [2025-11-04T19:48:21.885253] [INFO ] [MAIN ] KLC: Full history saved. [2025-11-04T19:48:21.885330] [INFO ] [MAIN ] Update Progress: 1197/2367 symbols completed. KLC took 39.2s. Estimated remaining: 2h 44m 31s. Rate: 7.1 symbols/min [2025-11-04T19:48:22.192939] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-11-04T19:48:22.567067] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-10. [2025-11-04T19:48:22.829480] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-10 (0.26 seconds). [2025-11-04T19:48:23.367038] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-07. [2025-11-04T19:48:24.167101] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-05. [2025-11-04T19:48:24.967056] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-11. [2025-11-04T19:48:25.062849] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-11-04T19:48:25.141761] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-11-04T19:48:25.285865] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-11 (0.32 seconds). [2025-11-04T19:48:25.286540] [INFO ] [REMOTE ] KMPB: Full fetch complete (4897 records). [2025-11-04T19:48:25.286609] [DEBUG] [VALIDATOR ] KMPB: Validating primary data. [2025-11-04T19:48:25.663912] [DEBUG] [VALIDATOR ] KMPB: All expected trading days are present. [2025-11-04T19:48:25.663990] [DEBUG] [VALIDATOR ] KMPB: Primary data validation passed. [2025-11-04T19:48:25.664170] [INFO ] [MAIN ] KMPB: Fetched primary data is valid. [2025-11-04T19:48:25.664212] [DEBUG] [LOCAL ] KMPB: Saving primary data. [2025-11-04T19:48:25.664832] [DEBUG] [LOCAL ] KMPB: Trimmed 4897 records to 3374 (keeping 200 days of data) [2025-11-04T19:48:25.687019] [DEBUG] [LOCAL ] KMPB: Primary data saved (3374 records). [2025-11-04T19:48:25.687094] [INFO ] [MAIN ] KMPB: Full history saved. [2025-11-04T19:48:25.687166] [INFO ] [MAIN ] Update Progress: 1198/2367 symbols completed. KMPB took 25.1s. Estimated remaining: 2h 44m 18s. Rate: 7.1 symbols/min [2025-11-04T19:48:25.767088] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-06. [2025-11-04T19:48:26.567075] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-08. [2025-11-04T19:48:26.692391] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T19:48:27.367143] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-07. [2025-11-04T19:48:27.912372] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-11-04T19:48:28.167087] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-09. [2025-11-04T19:48:28.483780] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-11-04T19:48:28.967076] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-08. [2025-11-04T19:48:29.535536] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-11-04T19:48:29.767080] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-10. [2025-11-04T19:48:30.192535] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T19:48:30.567055] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-09. [2025-11-04T19:48:31.413764] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-11-04T19:48:31.427740] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-11. [2025-11-04T19:48:31.655222] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-11-04T19:48:31.879187] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-11 (0.45 seconds). [2025-11-04T19:48:31.882302] [INFO ] [REMOTE ] KMI: Full fetch complete (77873 records). [2025-11-04T19:48:31.882582] [DEBUG] [VALIDATOR ] KMI: Validating primary data. [2025-11-04T19:48:32.172268] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-10. [2025-11-04T19:48:32.321483] [DEBUG] [VALIDATOR ] KMI: All expected trading days are present. [2025-11-04T19:48:32.321607] [DEBUG] [VALIDATOR ] KMI: Primary data validation passed. [2025-11-04T19:48:32.322959] [INFO ] [MAIN ] KMI: Fetched primary data is valid. [2025-11-04T19:48:32.323031] [DEBUG] [LOCAL ] KMI: Saving primary data. [2025-11-04T19:48:32.325156] [DEBUG] [LOCAL ] KMI: Trimmed 77873 records to 53693 (keeping 200 days of data) [2025-11-04T19:48:32.681051] [DEBUG] [LOCAL ] KMI: Primary data saved (53693 records). [2025-11-04T19:48:32.681296] [INFO ] [MAIN ] KMI: Full history saved. [2025-11-04T19:48:32.681423] [INFO ] [MAIN ] Update Progress: 1199/2367 symbols completed. KMI took 35.2s. Estimated remaining: 2h 44m 8s. Rate: 7.1 symbols/min [2025-11-04T19:48:33.281827] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-11-04T19:48:33.297480] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-11. [2025-11-04T19:48:34.654814] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-11 (1.36 seconds). [2025-11-04T19:48:34.656506] [INFO ] [REMOTE ] KMPR: Full fetch complete (54637 records). [2025-11-04T19:48:34.656582] [DEBUG] [VALIDATOR ] KMPR: Validating primary data. [2025-11-04T19:48:35.037970] [DEBUG] [VALIDATOR ] KMPR: All expected trading days are present. [2025-11-04T19:48:35.038048] [DEBUG] [VALIDATOR ] KMPR: Primary data validation passed. [2025-11-04T19:48:35.038847] [INFO ] [MAIN ] KMPR: Fetched primary data is valid. [2025-11-04T19:48:35.038909] [DEBUG] [LOCAL ] KMPR: Saving primary data. [2025-11-04T19:48:35.040035] [DEBUG] [LOCAL ] KMPR: Trimmed 54637 records to 40749 (keeping 200 days of data) [2025-11-04T19:48:35.275376] [DEBUG] [LOCAL ] KMPR: Primary data saved (40749 records). [2025-11-04T19:48:35.275520] [INFO ] [MAIN ] KMPR: Full history saved. [2025-11-04T19:48:35.275602] [INFO ] [MAIN ] Update Progress: 1200/2367 symbols completed. KMPR took 20.7s. Estimated remaining: 2h 43m 54s. Rate: 7.1 symbols/min [2025-11-04T19:48:35.844228] [INFO ] [MAIN ] KN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:48:35.975929] [INFO ] [MAIN ] KMT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:48:36.001754] [INFO ] [MAIN ] KMX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:48:36.023509] [INFO ] [MAIN ] KNF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:48:36.068539] [DEBUG] [LOCAL ] KN: Loaded 38038 existing records from primary storage. [2025-11-04T19:48:36.068912] [INFO ] [MAIN ] KN: Loaded 38038 existing records. [2025-11-04T19:48:36.189602] [DEBUG] [LOCAL ] KMX: Loaded 53286 existing records from primary storage. [2025-11-04T19:48:36.196195] [INFO ] [MAIN ] KMX: Loaded 53286 existing records. [2025-11-04T19:48:36.317969] [DEBUG] [LOCAL ] KNF: Loaded 36837 existing records from primary storage. [2025-11-04T19:48:36.383151] [INFO ] [MAIN ] KNF: Loaded 36837 existing records. [2025-11-04T19:48:36.400137] [DEBUG] [LOCAL ] KMT: Loaded 42219 existing records from primary storage. [2025-11-04T19:48:36.488626] [WARN ] [MAIN ] KN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:48:36.456809] [INFO ] [MAIN ] KMT: Loaded 42219 existing records. [2025-11-04T19:48:36.533809] [INFO ] [MAIN ] KN: Wiping existing primary data before full refresh. [2025-11-04T19:48:36.571244] [INFO ] [LOCAL ] KN: Starting deletion of primary data. [2025-11-04T19:48:36.671758] [DEBUG] [LOCAL ] KN: Deleted primary data file. [2025-11-04T19:48:36.717439] [INFO ] [MAIN ] KN: Starting full primary refresh. [2025-11-04T19:48:36.738985] [WARN ] [MAIN ] KMX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:48:36.776841] [INFO ] [MAIN ] KMX: Wiping existing primary data before full refresh. [2025-11-04T19:48:36.846607] [INFO ] [LOCAL ] KMX: Starting deletion of primary data. [2025-11-04T19:48:36.976235] [WARN ] [MAIN ] KNF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:48:36.979318] [WARN ] [MAIN ] KMT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:48:36.989927] [INFO ] [MAIN ] KNF: Wiping existing primary data before full refresh. [2025-11-04T19:48:36.990014] [INFO ] [LOCAL ] KNF: Starting deletion of primary data. [2025-11-04T19:48:36.995268] [DEBUG] [LOCAL ] KMX: Deleted primary data file. [2025-11-04T19:48:37.010304] [INFO ] [MAIN ] KMT: Wiping existing primary data before full refresh. [2025-11-04T19:48:37.010403] [DEBUG] [LOCAL ] KNF: Deleted primary data file. [2025-11-04T19:48:37.011051] [INFO ] [MAIN ] KNF: Starting full primary refresh. [2025-11-04T19:48:37.010692] [INFO ] [MAIN ] KMX: Starting full primary refresh. [2025-11-04T19:48:37.010582] [INFO ] [LOCAL ] KMT: Starting deletion of primary data. [2025-11-04T19:48:37.114140] [DEBUG] [LOCAL ] KMT: Deleted primary data file. [2025-11-04T19:48:37.209343] [INFO ] [MAIN ] KMT: Starting full primary refresh. [2025-11-04T19:48:37.821012] [INFO ] [REMOTE ] KN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:48:37.875476] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-01. [2025-11-04T19:48:38.295055] [INFO ] [REMOTE ] KMX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:48:38.354318] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-01. [2025-11-04T19:48:38.384950] [INFO ] [REMOTE ] KNF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:48:38.410828] [INFO ] [REMOTE ] KMT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:48:38.411227] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-01. [2025-11-04T19:48:38.411494] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-01. [2025-11-04T19:48:39.097100] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-11-04T19:48:39.105340] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-02. [2025-11-04T19:48:39.458552] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-11-04T19:48:39.465090] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-02. [2025-11-04T19:48:39.562883] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-11-04T19:48:39.573547] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-02. [2025-11-04T19:48:39.652538] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-11-04T19:48:39.660472] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-02. [2025-11-04T19:48:40.193120] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T19:48:40.201128] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-03. [2025-11-04T19:48:40.265322] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-11-04T19:48:40.473853] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-11-04T19:48:40.864825] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T19:48:40.967036] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-03. [2025-11-04T19:48:41.111419] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T19:48:41.767073] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-03. [2025-11-04T19:48:42.064999] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-11-04T19:48:42.567050] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-03. [2025-11-04T19:48:42.710779] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-11-04T19:48:43.367088] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-04. [2025-11-04T19:48:43.763922] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T19:48:44.167049] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-04. [2025-11-04T19:48:44.363583] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-11-04T19:48:44.967059] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-04. [2025-11-04T19:48:45.090702] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T19:48:45.767040] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-05. [2025-11-04T19:48:45.826611] [DEBUG] [REMOTE ] KNF: Error fetching primary data for 2025-05: 'API call returned an empty response.' [2025-11-04T19:48:45.826913] [WARN ] [REMOTE ] KNF: No data returned for month 2025-05 [2025-11-04T19:48:45.827002] [FATAL] [REMOTE ] KNF: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:48:45.827112] [FATAL] [MAIN ] KNF: Failed to fetch primary data. [2025-11-04T19:48:45.827184] [WARN ] [MAIN ] KNF: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:48:45.827255] [INFO ] [LOCAL ] KNF: Starting deletion of primary data. [2025-11-04T19:48:45.827318] [DEBUG] [LOCAL ] KNF: No primary data file found to delete. [2025-11-04T19:48:45.827354] [INFO ] [LOCAL ] KNF: Starting deletion of derivative data. [2025-11-04T19:48:45.827522] [DEBUG] [LOCAL ] KNF: Deleted derivative data file. [2025-11-04T19:48:45.827606] [INFO ] [MAIN ] Update Progress: 1201/2367 symbols completed. KNF took 10.5s. Estimated remaining: 2h 43m 48s. Rate: 7.1 symbols/min [2025-11-04T19:48:46.036897] [INFO ] [MAIN ] KNOP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:48:46.048696] [DEBUG] [LOCAL ] KNOP: Loaded 9943 existing records from primary storage. [2025-11-04T19:48:46.048908] [INFO ] [MAIN ] KNOP: Loaded 9943 existing records. [2025-11-04T19:48:46.304991] [WARN ] [MAIN ] KNOP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:48:46.306994] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-11-04T19:48:46.307177] [INFO ] [MAIN ] KNOP: Wiping existing primary data before full refresh. [2025-11-04T19:48:46.316331] [INFO ] [LOCAL ] KNOP: Starting deletion of primary data. [2025-11-04T19:48:46.317083] [DEBUG] [LOCAL ] KNOP: Deleted primary data file. [2025-11-04T19:48:46.317156] [INFO ] [MAIN ] KNOP: Starting full primary refresh. [2025-11-04T19:48:46.572289] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-05. [2025-11-04T19:48:46.744676] [INFO ] [REMOTE ] KNOP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:48:47.367062] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-04. [2025-11-04T19:48:47.733527] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T19:48:48.167038] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-06. [2025-11-04T19:48:48.770571] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-11-04T19:48:48.967036] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-01. [2025-11-04T19:48:49.200381] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-11-04T19:48:49.519998] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-11-04T19:48:49.767053] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-02. [2025-11-04T19:48:50.244759] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-11-04T19:48:50.567063] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-07. [2025-11-04T19:48:51.367045] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-05. [2025-11-04T19:48:51.702313] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T19:48:52.167030] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-05. [2025-11-04T19:48:52.327236] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-11-04T19:48:52.967046] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-06. [2025-11-04T19:48:53.317145] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T19:48:53.767010] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-06. [2025-11-04T19:48:53.986218] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T19:48:54.567043] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-07. [2025-11-04T19:48:55.067149] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-11-04T19:48:55.367043] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-08. [2025-11-04T19:48:55.670123] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T19:48:56.167085] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-03. [2025-11-04T19:48:56.507497] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T19:48:56.679103] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-11-04T19:48:56.967167] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-04. [2025-11-04T19:48:57.558170] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-11-04T19:48:57.767085] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-05. [2025-11-04T19:48:58.348427] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-11-04T19:48:58.567056] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-09. [2025-11-04T19:48:59.367047] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-08. [2025-11-04T19:48:59.608912] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-11-04T19:49:00.167089] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-10. [2025-11-04T19:49:00.729591] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-11-04T19:49:00.967101] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-09. [2025-11-04T19:49:01.267405] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T19:49:01.767098] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-11. [2025-11-04T19:49:02.027634] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T19:49:02.567050] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-07. [2025-11-04T19:49:02.819874] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T19:49:02.821599] [INFO ] [REMOTE ] KMT: Full fetch complete (61811 records). [2025-11-04T19:49:02.821724] [DEBUG] [VALIDATOR ] KMT: Validating primary data. [2025-11-04T19:49:03.220934] [DEBUG] [VALIDATOR ] KMT: All expected trading days are present. [2025-11-04T19:49:03.221016] [DEBUG] [VALIDATOR ] KMT: Primary data validation passed. [2025-11-04T19:49:03.221849] [INFO ] [MAIN ] KMT: Fetched primary data is valid. [2025-11-04T19:49:03.221913] [DEBUG] [LOCAL ] KMT: Saving primary data. [2025-11-04T19:49:03.223166] [DEBUG] [LOCAL ] KMT: Trimmed 61811 records to 42230 (keeping 200 days of data) [2025-11-04T19:49:03.391633] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-10. [2025-11-04T19:49:03.477241] [DEBUG] [LOCAL ] KMT: Primary data saved (42230 records). [2025-11-04T19:49:03.477362] [INFO ] [MAIN ] KMT: Full history saved. [2025-11-04T19:49:03.477437] [INFO ] [MAIN ] Update Progress: 1202/2367 symbols completed. KMT took 28.2s. Estimated remaining: 2h 43m 48s. Rate: 7.1 symbols/min [2025-11-04T19:49:03.650198] [INFO ] [MAIN ] KNSL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:49:03.669682] [DEBUG] [LOCAL ] KNSL: Loaded 19035 existing records from primary storage. [2025-11-04T19:49:03.669928] [INFO ] [MAIN ] KNSL: Loaded 19035 existing records. [2025-11-04T19:49:03.859209] [WARN ] [MAIN ] KNSL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:49:03.859423] [INFO ] [MAIN ] KNSL: Wiping existing primary data before full refresh. [2025-11-04T19:49:03.859488] [INFO ] [LOCAL ] KNSL: Starting deletion of primary data. [2025-11-04T19:49:03.859745] [DEBUG] [LOCAL ] KNSL: Deleted primary data file. [2025-11-04T19:49:03.859812] [INFO ] [MAIN ] KNSL: Starting full primary refresh. [2025-11-04T19:49:04.170436] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-06. [2025-11-04T19:49:04.235467] [INFO ] [REMOTE ] KNSL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:49:04.280632] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-11-04T19:49:04.405770] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-11-04T19:49:04.712020] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-11-04T19:49:04.967021] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-11. [2025-11-04T19:49:05.767089] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-01. [2025-11-04T19:49:06.004737] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-11 (1.04 seconds). [2025-11-04T19:49:06.006412] [INFO ] [REMOTE ] KN: Full fetch complete (57087 records). [2025-11-04T19:49:06.006492] [DEBUG] [VALIDATOR ] KN: Validating primary data. [2025-11-04T19:49:06.386962] [DEBUG] [VALIDATOR ] KN: All expected trading days are present. [2025-11-04T19:49:06.387063] [DEBUG] [VALIDATOR ] KN: Primary data validation passed. [2025-11-04T19:49:06.388072] [INFO ] [MAIN ] KN: Fetched primary data is valid. [2025-11-04T19:49:06.388152] [DEBUG] [LOCAL ] KN: Saving primary data. [2025-11-04T19:49:06.389521] [DEBUG] [LOCAL ] KN: Trimmed 57087 records to 37994 (keeping 200 days of data) [2025-11-04T19:49:06.574073] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-07. [2025-11-04T19:49:06.604505] [DEBUG] [LOCAL ] KN: Primary data saved (37994 records). [2025-11-04T19:49:06.604992] [INFO ] [MAIN ] KN: Full history saved. [2025-11-04T19:49:06.605108] [INFO ] [MAIN ] Update Progress: 1203/2367 symbols completed. KN took 31.3s. Estimated remaining: 2h 43m 34s. Rate: 7.1 symbols/min [2025-11-04T19:49:06.721070] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-11-04T19:49:06.794094] [INFO ] [MAIN ] KNTK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:49:06.835261] [DEBUG] [LOCAL ] KNTK: Loaded 47182 existing records from primary storage. [2025-11-04T19:49:06.835568] [INFO ] [MAIN ] KNTK: Loaded 47182 existing records. [2025-11-04T19:49:07.008561] [WARN ] [MAIN ] KNTK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:49:07.009355] [INFO ] [MAIN ] KNTK: Wiping existing primary data before full refresh. [2025-11-04T19:49:07.009405] [INFO ] [LOCAL ] KNTK: Starting deletion of primary data. [2025-11-04T19:49:07.009853] [DEBUG] [LOCAL ] KNTK: Deleted primary data file. [2025-11-04T19:49:07.009909] [INFO ] [MAIN ] KNTK: Starting full primary refresh. [2025-11-04T19:49:07.244262] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-11-04T19:49:07.376393] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-08. [2025-11-04T19:49:07.381048] [INFO ] [REMOTE ] KNTK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:49:07.807215] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-11-04T19:49:08.167043] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-09. [2025-11-04T19:49:08.832318] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-11-04T19:49:08.967075] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-08. [2025-11-04T19:49:09.767026] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-02. [2025-11-04T19:49:10.422035] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-11-04T19:49:10.446894] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-11-04T19:49:10.567041] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-03. [2025-11-04T19:49:11.358044] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T19:49:11.366988] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-04. [2025-11-04T19:49:12.167105] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-01. [2025-11-04T19:49:12.262211] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-11-04T19:49:12.967089] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-05. [2025-11-04T19:49:13.263351] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-11-04T19:49:13.622378] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-11-04T19:49:13.767084] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-06. [2025-11-04T19:49:14.536341] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T19:49:14.567017] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-10. [2025-11-04T19:49:15.195208] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-11-04T19:49:15.367093] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-09. [2025-11-04T19:49:16.167082] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-07. [2025-11-04T19:49:16.802934] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-11-04T19:49:16.923133] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-11-04T19:49:16.967008] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-11. [2025-11-04T19:49:17.593152] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T19:49:17.594320] [INFO ] [REMOTE ] KNOP: Full fetch complete (14957 records). [2025-11-04T19:49:17.594385] [DEBUG] [VALIDATOR ] KNOP: Validating primary data. [2025-11-04T19:49:17.776154] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-10. [2025-11-04T19:49:17.964148] [DEBUG] [VALIDATOR ] KNOP: All expected trading days are present. [2025-11-04T19:49:17.964221] [DEBUG] [VALIDATOR ] KNOP: Primary data validation passed. [2025-11-04T19:49:17.964569] [INFO ] [MAIN ] KNOP: Fetched primary data is valid. [2025-11-04T19:49:17.964612] [DEBUG] [LOCAL ] KNOP: Saving primary data. [2025-11-04T19:49:17.965112] [DEBUG] [LOCAL ] KNOP: Trimmed 14957 records to 10042 (keeping 200 days of data) [2025-11-04T19:49:18.021900] [DEBUG] [LOCAL ] KNOP: Primary data saved (10042 records). [2025-11-04T19:49:18.022016] [INFO ] [MAIN ] KNOP: Full history saved. [2025-11-04T19:49:18.022091] [INFO ] [MAIN ] Update Progress: 1204/2367 symbols completed. KNOP took 32.2s. Estimated remaining: 2h 43m 29s. Rate: 7.1 symbols/min [2025-11-04T19:49:18.201558] [INFO ] [MAIN ] KNX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:49:18.248326] [DEBUG] [LOCAL ] KNX: Loaded 52646 existing records from primary storage. [2025-11-04T19:49:18.248625] [INFO ] [MAIN ] KNX: Loaded 52646 existing records. [2025-11-04T19:49:18.439002] [WARN ] [MAIN ] KNX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:49:18.439154] [INFO ] [MAIN ] KNX: Wiping existing primary data before full refresh. [2025-11-04T19:49:18.439202] [INFO ] [LOCAL ] KNX: Starting deletion of primary data. [2025-11-04T19:49:18.439375] [DEBUG] [LOCAL ] KNX: Deleted primary data file. [2025-11-04T19:49:18.439414] [INFO ] [MAIN ] KNX: Starting full primary refresh. [2025-11-04T19:49:18.572156] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-02. [2025-11-04T19:49:18.824814] [INFO ] [REMOTE ] KNX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:49:19.367016] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-08. [2025-11-04T19:49:19.408974] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-11-04T19:49:19.535238] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T19:49:20.023180] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-11-04T19:49:20.167045] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-09. [2025-11-04T19:49:20.859249] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-11-04T19:49:20.967046] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-10. [2025-11-04T19:49:21.762736] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T19:49:21.767808] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-11. [2025-11-04T19:49:22.225513] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-11 (0.46 seconds). [2025-11-04T19:49:22.227517] [INFO ] [REMOTE ] KMX: Full fetch complete (76518 records). [2025-11-04T19:49:22.227616] [DEBUG] [VALIDATOR ] KMX: Validating primary data. [2025-11-04T19:49:22.572225] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-11. [2025-11-04T19:49:22.618905] [DEBUG] [VALIDATOR ] KMX: All expected trading days are present. [2025-11-04T19:49:22.619100] [DEBUG] [VALIDATOR ] KMX: Primary data validation passed. [2025-11-04T19:49:22.620088] [INFO ] [MAIN ] KMX: Fetched primary data is valid. [2025-11-04T19:49:22.620169] [DEBUG] [LOCAL ] KMX: Saving primary data. [2025-11-04T19:49:22.621539] [DEBUG] [LOCAL ] KMX: Trimmed 76518 records to 53289 (keeping 200 days of data) [2025-11-04T19:49:22.921166] [DEBUG] [LOCAL ] KMX: Primary data saved (53289 records). [2025-11-04T19:49:22.921288] [INFO ] [MAIN ] KMX: Full history saved. [2025-11-04T19:49:22.921377] [INFO ] [MAIN ] Update Progress: 1205/2367 symbols completed. KMX took 47.6s. Estimated remaining: 2h 43m 17s. Rate: 7.1 symbols/min [2025-11-04T19:49:23.086756] [INFO ] [MAIN ] KO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:49:23.086952] [DEBUG] [LOCAL ] KO: No existing data file found in primary storage. [2025-11-04T19:49:23.086988] [WARN ] [MAIN ] KO: No existing data found, performing full refresh instead. [2025-11-04T19:49:23.087100] [INFO ] [MAIN ] KO: Wiping existing primary data before full refresh. [2025-11-04T19:49:23.087140] [INFO ] [LOCAL ] KO: Starting deletion of primary data. [2025-11-04T19:49:23.087179] [DEBUG] [LOCAL ] KO: No primary data file found to delete. [2025-11-04T19:49:23.087210] [INFO ] [MAIN ] KO: Starting full primary refresh. [2025-11-04T19:49:23.372140] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-01. [2025-11-04T19:49:23.442741] [INFO ] [REMOTE ] KO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:49:23.462880] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T19:49:23.464363] [INFO ] [REMOTE ] KNSL: Full fetch complete (27733 records). [2025-11-04T19:49:23.464433] [DEBUG] [VALIDATOR ] KNSL: Validating primary data. [2025-11-04T19:49:23.822206] [DEBUG] [VALIDATOR ] KNSL: All expected trading days are present. [2025-11-04T19:49:23.822282] [DEBUG] [VALIDATOR ] KNSL: Primary data validation passed. [2025-11-04T19:49:23.822885] [INFO ] [MAIN ] KNSL: Fetched primary data is valid. [2025-11-04T19:49:23.822947] [DEBUG] [LOCAL ] KNSL: Saving primary data. [2025-11-04T19:49:23.823600] [DEBUG] [LOCAL ] KNSL: Trimmed 27733 records to 19233 (keeping 200 days of data) [2025-11-04T19:49:23.939922] [DEBUG] [LOCAL ] KNSL: Primary data saved (19233 records). [2025-11-04T19:49:23.940051] [INFO ] [MAIN ] KNSL: Full history saved. [2025-11-04T19:49:23.940125] [INFO ] [MAIN ] Update Progress: 1206/2367 symbols completed. KNSL took 20.5s. Estimated remaining: 2h 43m 2s. Rate: 7.1 symbols/min [2025-11-04T19:49:24.107109] [INFO ] [MAIN ] KODK: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:49:24.145277] [DEBUG] [LOCAL ] KODK: Loaded 43194 existing records from primary storage. [2025-11-04T19:49:24.145584] [INFO ] [MAIN ] KODK: Loaded 43194 existing records. [2025-11-04T19:49:24.176779] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-03. [2025-11-04T19:49:24.330735] [WARN ] [MAIN ] KODK: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:49:24.330991] [INFO ] [MAIN ] KODK: Wiping existing primary data before full refresh. [2025-11-04T19:49:24.331048] [INFO ] [LOCAL ] KODK: Starting deletion of primary data. [2025-11-04T19:49:24.331266] [DEBUG] [LOCAL ] KODK: Deleted primary data file. [2025-11-04T19:49:24.331320] [INFO ] [MAIN ] KODK: Starting full primary refresh. [2025-11-04T19:49:24.699730] [INFO ] [REMOTE ] KODK: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:49:24.727066] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-11-04T19:49:24.967068] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-02. [2025-11-04T19:49:25.330134] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-11-04T19:49:25.767037] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-04. [2025-11-04T19:49:26.237085] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-11-04T19:49:26.567051] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-01. [2025-11-04T19:49:26.948686] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-11-04T19:49:27.367048] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-03. [2025-11-04T19:49:28.167051] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-05. [2025-11-04T19:49:28.539690] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-11-04T19:49:28.841755] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-11-04T19:49:28.967105] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-04. [2025-11-04T19:49:29.335257] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T19:49:29.767089] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-06. [2025-11-04T19:49:30.351540] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-11-04T19:49:30.567140] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-01. [2025-11-04T19:49:30.932357] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T19:49:31.367049] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-02. [2025-11-04T19:49:32.065830] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-11-04T19:49:32.167028] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-02. [2025-11-04T19:49:32.967036] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-07. [2025-11-04T19:49:33.307149] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T19:49:33.480828] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-02 (2.11 seconds). [2025-11-04T19:49:33.767023] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-05. [2025-11-04T19:49:34.237571] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-11-04T19:49:34.567106] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-08. [2025-11-04T19:49:34.985577] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-11-04T19:49:35.367084] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-06. [2025-11-04T19:49:35.794527] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-11-04T19:49:36.167041] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-09. [2025-11-04T19:49:36.585817] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-11-04T19:49:36.967027] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-07. [2025-11-04T19:49:37.358961] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T19:49:37.767064] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-03. [2025-11-04T19:49:38.169540] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T19:49:38.567026] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-03. [2025-11-04T19:49:39.367119] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-10. [2025-11-04T19:49:39.668866] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-11-04T19:49:39.702194] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-11-04T19:49:40.167144] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-04. [2025-11-04T19:49:40.635942] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-11-04T19:49:40.967037] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-04. [2025-11-04T19:49:41.767039] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-08. [2025-11-04T19:49:42.092409] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-11-04T19:49:42.181697] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-11-04T19:49:42.567018] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-05. [2025-11-04T19:49:43.038753] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T19:49:43.367019] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-09. [2025-11-04T19:49:44.167061] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-05. [2025-11-04T19:49:44.585570] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T19:49:44.717064] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-11-04T19:49:44.967043] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-10. [2025-11-04T19:49:45.429227] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-11-04T19:49:45.767044] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-06. [2025-11-04T19:49:46.425365] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-11-04T19:49:46.567016] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-11. [2025-11-04T19:49:46.809727] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-11-04T19:49:46.931714] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T19:49:46.933662] [INFO ] [REMOTE ] KNX: Full fetch complete (76383 records). [2025-11-04T19:49:46.933762] [DEBUG] [VALIDATOR ] KNX: Validating primary data. [2025-11-04T19:49:47.310330] [DEBUG] [VALIDATOR ] KNX: All expected trading days are present. [2025-11-04T19:49:47.310408] [DEBUG] [VALIDATOR ] KNX: Primary data validation passed. [2025-11-04T19:49:47.311337] [INFO ] [MAIN ] KNX: Fetched primary data is valid. [2025-11-04T19:49:47.311399] [DEBUG] [LOCAL ] KNX: Saving primary data. [2025-11-04T19:49:47.312751] [DEBUG] [LOCAL ] KNX: Trimmed 76383 records to 52652 (keeping 200 days of data) [2025-11-04T19:49:47.372374] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-11. [2025-11-04T19:49:47.623931] [DEBUG] [LOCAL ] KNX: Primary data saved (52652 records). [2025-11-04T19:49:47.624079] [INFO ] [MAIN ] KNX: Full history saved. [2025-11-04T19:49:47.624170] [INFO ] [MAIN ] Update Progress: 1207/2367 symbols completed. KNX took 29.6s. Estimated remaining: 2h 43m 8s. Rate: 7.1 symbols/min [2025-11-04T19:49:47.792402] [INFO ] [MAIN ] KOF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:49:47.817261] [DEBUG] [LOCAL ] KOF: Loaded 25971 existing records from primary storage. [2025-11-04T19:49:47.817509] [INFO ] [MAIN ] KOF: Loaded 25971 existing records. [2025-11-04T19:49:47.990924] [WARN ] [MAIN ] KOF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:49:47.991142] [INFO ] [MAIN ] KOF: Wiping existing primary data before full refresh. [2025-11-04T19:49:47.991187] [INFO ] [LOCAL ] KOF: Starting deletion of primary data. [2025-11-04T19:49:47.991429] [DEBUG] [LOCAL ] KOF: Deleted primary data file. [2025-11-04T19:49:47.991485] [INFO ] [MAIN ] KOF: Starting full primary refresh. [2025-11-04T19:49:48.174414] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-07. [2025-11-04T19:49:48.344495] [INFO ] [REMOTE ] KOF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:49:48.389120] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-11 (1.02 seconds). [2025-11-04T19:49:48.390792] [INFO ] [REMOTE ] KNTK: Full fetch complete (66317 records). [2025-11-04T19:49:48.390885] [DEBUG] [VALIDATOR ] KNTK: Validating primary data. [2025-11-04T19:49:48.794812] [DEBUG] [VALIDATOR ] KNTK: All expected trading days are present. [2025-11-04T19:49:48.794890] [DEBUG] [VALIDATOR ] KNTK: Primary data validation passed. [2025-11-04T19:49:48.795818] [INFO ] [MAIN ] KNTK: Fetched primary data is valid. [2025-11-04T19:49:48.795900] [DEBUG] [LOCAL ] KNTK: Saving primary data. [2025-11-04T19:49:48.797163] [DEBUG] [LOCAL ] KNTK: Trimmed 66317 records to 47232 (keeping 200 days of data) [2025-11-04T19:49:48.976980] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-06. [2025-11-04T19:49:49.094759] [DEBUG] [LOCAL ] KNTK: Primary data saved (47232 records). [2025-11-04T19:49:49.094883] [INFO ] [MAIN ] KNTK: Full history saved. [2025-11-04T19:49:49.094958] [INFO ] [MAIN ] Update Progress: 1208/2367 symbols completed. KNTK took 42.5s. Estimated remaining: 2h 42m 53s. Rate: 7.1 symbols/min [2025-11-04T19:49:49.280971] [INFO ] [MAIN ] KOP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:49:49.297944] [DEBUG] [LOCAL ] KOP: Loaded 17218 existing records from primary storage. [2025-11-04T19:49:49.298178] [INFO ] [MAIN ] KOP: Loaded 17218 existing records. [2025-11-04T19:49:49.500811] [WARN ] [MAIN ] KOP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:49:49.501081] [INFO ] [MAIN ] KOP: Wiping existing primary data before full refresh. [2025-11-04T19:49:49.501158] [INFO ] [LOCAL ] KOP: Starting deletion of primary data. [2025-11-04T19:49:49.501415] [DEBUG] [LOCAL ] KOP: Deleted primary data file. [2025-11-04T19:49:49.501481] [INFO ] [MAIN ] KOP: Starting full primary refresh. [2025-11-04T19:49:49.770321] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-01. [2025-11-04T19:49:49.798141] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-11-04T19:49:49.901049] [INFO ] [REMOTE ] KOP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:49:50.567080] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-01. [2025-11-04T19:49:50.909811] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T19:49:50.995701] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-06 (2.02 seconds). [2025-11-04T19:49:51.225251] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-11-04T19:49:51.367063] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-08. [2025-11-04T19:49:52.167024] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-07. [2025-11-04T19:49:52.575187] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T19:49:52.967005] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-02. [2025-11-04T19:49:53.519968] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-11-04T19:49:53.767019] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-03. [2025-11-04T19:49:54.260346] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-11-04T19:49:54.456387] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-11-04T19:49:54.567021] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-09. [2025-11-04T19:49:55.367043] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-08. [2025-11-04T19:49:55.691856] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T19:49:56.167033] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-02. [2025-11-04T19:49:56.954069] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T19:49:56.967001] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-03. [2025-11-04T19:49:57.267325] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-11-04T19:49:57.713981] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-11-04T19:49:57.766998] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-04. [2025-11-04T19:49:58.567007] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-10. [2025-11-04T19:49:58.690948] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T19:49:59.367067] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-04. [2025-11-04T19:49:59.943005] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-11-04T19:50:00.137029] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-11-04T19:50:00.167097] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-05. [2025-11-04T19:50:00.967151] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-05. [2025-11-04T19:50:01.001204] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-11-04T19:50:01.767086] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-06. [2025-11-04T19:50:01.944411] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-11-04T19:50:02.553099] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-11-04T19:50:02.567027] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-06. [2025-11-04T19:50:03.367023] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-07. [2025-11-04T19:50:03.474709] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T19:50:03.921303] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-11-04T19:50:04.167064] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-08. [2025-11-04T19:50:04.967051] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-09. [2025-11-04T19:50:05.008280] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-11-04T19:50:05.767098] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-09. [2025-11-04T19:50:06.399521] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-11-04T19:50:06.567040] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-10. [2025-11-04T19:50:06.786277] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-11-04T19:50:07.230399] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-11-04T19:50:07.367046] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-10. [2025-11-04T19:50:08.167089] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-11. [2025-11-04T19:50:08.805836] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-11 (0.64 seconds). [2025-11-04T19:50:08.806978] [INFO ] [REMOTE ] KOP: Full fetch complete (25536 records). [2025-11-04T19:50:08.807066] [DEBUG] [VALIDATOR ] KOP: Validating primary data. [2025-11-04T19:50:08.980713] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-07. [2025-11-04T19:50:09.211258] [DEBUG] [VALIDATOR ] KOP: All expected trading days are present. [2025-11-04T19:50:09.211346] [DEBUG] [VALIDATOR ] KOP: Primary data validation passed. [2025-11-04T19:50:09.211996] [INFO ] [MAIN ] KOP: Fetched primary data is valid. [2025-11-04T19:50:09.212079] [DEBUG] [LOCAL ] KOP: Saving primary data. [2025-11-04T19:50:09.213095] [DEBUG] [LOCAL ] KOP: Trimmed 25536 records to 17214 (keeping 200 days of data) [2025-11-04T19:50:09.325814] [DEBUG] [LOCAL ] KOP: Primary data saved (17214 records). [2025-11-04T19:50:09.325961] [INFO ] [MAIN ] KOP: Full history saved. [2025-11-04T19:50:09.326043] [INFO ] [MAIN ] Update Progress: 1209/2367 symbols completed. KOP took 20.2s. Estimated remaining: 2h 42m 56s. Rate: 7.1 symbols/min [2025-11-04T19:50:09.544624] [INFO ] [MAIN ] KORE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:50:09.549466] [DEBUG] [LOCAL ] KORE: Loaded 2342 existing records from primary storage. [2025-11-04T19:50:09.549748] [INFO ] [MAIN ] KORE: Loaded 2342 existing records. [2025-11-04T19:50:09.730785] [WARN ] [MAIN ] KORE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:50:09.731180] [INFO ] [MAIN ] KORE: Wiping existing primary data before full refresh. [2025-11-04T19:50:09.731255] [INFO ] [LOCAL ] KORE: Starting deletion of primary data. [2025-11-04T19:50:09.750040] [DEBUG] [LOCAL ] KORE: Deleted primary data file. [2025-11-04T19:50:09.769763] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-11. [2025-11-04T19:50:09.770209] [INFO ] [MAIN ] KORE: Starting full primary refresh. [2025-11-04T19:50:09.804223] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-10 (2.44 seconds). [2025-11-04T19:50:10.097170] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-11-04T19:50:10.169162] [INFO ] [REMOTE ] KORE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:50:10.567051] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-11. [2025-11-04T19:50:10.957465] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-11 (0.39 seconds). [2025-11-04T19:50:10.959516] [INFO ] [REMOTE ] KO: Full fetch complete (77218 records). [2025-11-04T19:50:10.959626] [DEBUG] [VALIDATOR ] KO: Validating primary data. [2025-11-04T19:50:11.067441] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T19:50:11.095992] [INFO ] [REMOTE ] KODK: Full fetch complete (62941 records). [2025-11-04T19:50:11.121634] [DEBUG] [VALIDATOR ] KODK: Validating primary data. [2025-11-04T19:50:11.170674] [WARN ] [VALIDATOR ] KO: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-11-04T19:50:11.215710] [FATAL] [MAIN ] KO: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:50:11.242442] [WARN ] [MAIN ] KO: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:50:11.253505] [INFO ] [LOCAL ] KO: Starting deletion of primary data. [2025-11-04T19:50:11.274884] [DEBUG] [LOCAL ] KO: No primary data file found to delete. [2025-11-04T19:50:11.274997] [INFO ] [LOCAL ] KO: Starting deletion of derivative data. [2025-11-04T19:50:11.275324] [DEBUG] [LOCAL ] KO: No derivative data file found to delete. [2025-11-04T19:50:11.285916] [INFO ] [MAIN ] Update Progress: 1210/2367 symbols completed. KO took 48.4s. Estimated remaining: 2h 42m 41s. Rate: 7.1 symbols/min [2025-11-04T19:50:11.372111] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-08. [2025-11-04T19:50:11.667825] [INFO ] [MAIN ] KOS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:50:11.739717] [DEBUG] [VALIDATOR ] KODK: All expected trading days are present. [2025-11-04T19:50:11.739875] [DEBUG] [VALIDATOR ] KODK: Primary data validation passed. [2025-11-04T19:50:11.740741] [INFO ] [MAIN ] KODK: Fetched primary data is valid. [2025-11-04T19:50:11.740900] [DEBUG] [LOCAL ] KODK: Saving primary data. [2025-11-04T19:50:11.742158] [DEBUG] [LOCAL ] KODK: Trimmed 62941 records to 43142 (keeping 200 days of data) [2025-11-04T19:50:11.935707] [DEBUG] [LOCAL ] KOS: Loaded 53079 existing records from primary storage. [2025-11-04T19:50:11.972857] [INFO ] [MAIN ] KOS: Loaded 53079 existing records. [2025-11-04T19:50:12.116824] [DEBUG] [LOCAL ] KODK: Primary data saved (43142 records). [2025-11-04T19:50:12.122776] [INFO ] [MAIN ] KODK: Full history saved. [2025-11-04T19:50:12.167412] [INFO ] [MAIN ] Update Progress: 1211/2367 symbols completed. KODK took 48.2s. Estimated remaining: 2h 42m 25s. Rate: 7.1 symbols/min [2025-11-04T19:50:12.167486] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-01. [2025-11-04T19:50:12.202863] [WARN ] [MAIN ] KOS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:50:12.219684] [INFO ] [MAIN ] KOS: Wiping existing primary data before full refresh. [2025-11-04T19:50:12.229113] [INFO ] [LOCAL ] KOS: Starting deletion of primary data. [2025-11-04T19:50:12.274023] [DEBUG] [LOCAL ] KOS: Deleted primary data file. [2025-11-04T19:50:12.279395] [INFO ] [MAIN ] KOS: Starting full primary refresh. [2025-11-04T19:50:12.479093] [INFO ] [MAIN ] KR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:50:12.700031] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-11-04T19:50:12.787124] [INFO ] [REMOTE ] KOS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:50:12.810351] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-11-04T19:50:12.810079] [DEBUG] [LOCAL ] KR: Loaded 53681 existing records from primary storage. [2025-11-04T19:50:12.813365] [INFO ] [MAIN ] KR: Loaded 53681 existing records. [2025-11-04T19:50:12.967095] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-09. [2025-11-04T19:50:12.989270] [WARN ] [MAIN ] KR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:50:12.989601] [INFO ] [MAIN ] KR: Wiping existing primary data before full refresh. [2025-11-04T19:50:12.989676] [INFO ] [LOCAL ] KR: Starting deletion of primary data. [2025-11-04T19:50:12.989886] [DEBUG] [LOCAL ] KR: Deleted primary data file. [2025-11-04T19:50:12.989924] [INFO ] [MAIN ] KR: Starting full primary refresh. [2025-11-04T19:50:13.338562] [INFO ] [REMOTE ] KR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:50:13.753397] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-11-04T19:50:13.766999] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-10. [2025-11-04T19:50:14.568109] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-01. [2025-11-04T19:50:14.571044] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-11-04T19:50:15.367115] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-01. [2025-11-04T19:50:16.084020] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-11-04T19:50:16.167002] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-02. [2025-11-04T19:50:16.906364] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-11-04T19:50:16.967016] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-02. [2025-11-04T19:50:17.365005] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T19:50:17.767045] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-11. [2025-11-04T19:50:18.226209] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T19:50:18.550659] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T19:50:18.551913] [INFO ] [REMOTE ] KOF: Full fetch complete (37897 records). [2025-11-04T19:50:18.551983] [DEBUG] [VALIDATOR ] KOF: Validating primary data. [2025-11-04T19:50:18.572383] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-03. [2025-11-04T19:50:18.932691] [DEBUG] [VALIDATOR ] KOF: All expected trading days are present. [2025-11-04T19:50:18.932771] [DEBUG] [VALIDATOR ] KOF: Primary data validation passed. [2025-11-04T19:50:18.933397] [INFO ] [MAIN ] KOF: Fetched primary data is valid. [2025-11-04T19:50:18.933434] [DEBUG] [LOCAL ] KOF: Saving primary data. [2025-11-04T19:50:18.934216] [DEBUG] [LOCAL ] KOF: Trimmed 37897 records to 26018 (keeping 200 days of data) [2025-11-04T19:50:19.085816] [DEBUG] [LOCAL ] KOF: Primary data saved (26018 records). [2025-11-04T19:50:19.085938] [INFO ] [MAIN ] KOF: Full history saved. [2025-11-04T19:50:19.086013] [INFO ] [MAIN ] Update Progress: 1212/2367 symbols completed. KOF took 31.5s. Estimated remaining: 2h 42m 15s. Rate: 7.1 symbols/min [2025-11-04T19:50:19.252354] [INFO ] [MAIN ] KRC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:50:19.252507] [DEBUG] [LOCAL ] KRC: No existing data file found in primary storage. [2025-11-04T19:50:19.252564] [WARN ] [MAIN ] KRC: No existing data found, performing full refresh instead. [2025-11-04T19:50:19.252697] [INFO ] [MAIN ] KRC: Wiping existing primary data before full refresh. [2025-11-04T19:50:19.252749] [INFO ] [LOCAL ] KRC: Starting deletion of primary data. [2025-11-04T19:50:19.252791] [DEBUG] [LOCAL ] KRC: No primary data file found to delete. [2025-11-04T19:50:19.252821] [INFO ] [MAIN ] KRC: Starting full primary refresh. [2025-11-04T19:50:19.372115] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-03. [2025-11-04T19:50:19.626185] [INFO ] [REMOTE ] KRC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:50:20.135819] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-11-04T19:50:20.167060] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-02. [2025-11-04T19:50:20.521259] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-11-04T19:50:20.691565] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-11-04T19:50:20.967054] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-04. [2025-11-04T19:50:21.767049] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-03. [2025-11-04T19:50:22.119767] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-11-04T19:50:22.293284] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T19:50:22.567044] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-04. [2025-11-04T19:50:22.892082] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-11-04T19:50:23.367061] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-05. [2025-11-04T19:50:24.167062] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-05. [2025-11-04T19:50:24.529780] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-11-04T19:50:24.671142] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-11-04T19:50:24.967020] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-06. [2025-11-04T19:50:25.249698] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-06 (0.28 seconds). [2025-11-04T19:50:25.767053] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-06. [2025-11-04T19:50:26.567055] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-01. [2025-11-04T19:50:27.123379] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-11-04T19:50:27.367043] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-07. [2025-11-04T19:50:27.723103] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-11-04T19:50:27.970217] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-11-04T19:50:28.167067] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-02. [2025-11-04T19:50:28.967077] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-08. [2025-11-04T19:50:29.257549] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-11-04T19:50:29.524696] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T19:50:29.767041] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-03. [2025-11-04T19:50:30.567084] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-04. [2025-11-04T19:50:30.803508] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-11-04T19:50:31.367010] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-09. [2025-11-04T19:50:31.672827] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-09 (0.31 seconds). [2025-11-04T19:50:32.011713] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T19:50:32.167030] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-10. [2025-11-04T19:50:32.538045] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-11-04T19:50:32.967036] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-11. [2025-11-04T19:50:33.483773] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-11 (0.52 seconds). [2025-11-04T19:50:33.484748] [INFO ] [REMOTE ] KORE: Full fetch complete (3829 records). [2025-11-04T19:50:33.484815] [DEBUG] [VALIDATOR ] KORE: Validating primary data. [2025-11-04T19:50:33.772176] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-05. [2025-11-04T19:50:33.881900] [DEBUG] [VALIDATOR ] KORE: All expected trading days are present. [2025-11-04T19:50:33.881973] [DEBUG] [VALIDATOR ] KORE: Primary data validation passed. [2025-11-04T19:50:33.882102] [INFO ] [MAIN ] KORE: Fetched primary data is valid. [2025-11-04T19:50:33.882140] [DEBUG] [LOCAL ] KORE: Saving primary data. [2025-11-04T19:50:33.882478] [DEBUG] [LOCAL ] KORE: Trimmed 3829 records to 2569 (keeping 200 days of data) [2025-11-04T19:50:33.897151] [DEBUG] [LOCAL ] KORE: Primary data saved (2569 records). [2025-11-04T19:50:33.897222] [INFO ] [MAIN ] KORE: Full history saved. [2025-11-04T19:50:33.897290] [INFO ] [MAIN ] Update Progress: 1213/2367 symbols completed. KORE took 24.6s. Estimated remaining: 2h 42m 13s. Rate: 7.1 symbols/min [2025-11-04T19:50:34.072368] [INFO ] [MAIN ] KREF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:50:34.107602] [DEBUG] [LOCAL ] KREF: Loaded 37831 existing records from primary storage. [2025-11-04T19:50:34.107914] [INFO ] [MAIN ] KREF: Loaded 37831 existing records. [2025-11-04T19:50:34.283007] [WARN ] [MAIN ] KREF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:50:34.283210] [INFO ] [MAIN ] KREF: Wiping existing primary data before full refresh. [2025-11-04T19:50:34.283257] [INFO ] [LOCAL ] KREF: Starting deletion of primary data. [2025-11-04T19:50:34.283429] [DEBUG] [LOCAL ] KREF: Deleted primary data file. [2025-11-04T19:50:34.283466] [INFO ] [MAIN ] KREF: Starting full primary refresh. [2025-11-04T19:50:34.572222] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-04. [2025-11-04T19:50:34.637776] [INFO ] [REMOTE ] KREF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:50:35.269074] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-11-04T19:50:35.367037] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-06. [2025-11-04T19:50:35.776360] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T19:50:36.167050] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-01. [2025-11-04T19:50:36.776043] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-11-04T19:50:36.967011] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-07. [2025-11-04T19:50:37.037300] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T19:50:37.767016] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-02. [2025-11-04T19:50:38.432000] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T19:50:38.579162] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-11-04T19:50:38.578018] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-08. [2025-11-04T19:50:39.367065] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-07. [2025-11-04T19:50:40.167058] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-03. [2025-11-04T19:50:40.251736] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-11-04T19:50:40.647776] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-11-04T19:50:40.967079] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-08. [2025-11-04T19:50:41.031852] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-11-04T19:50:41.767087] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-04. [2025-11-04T19:50:42.270910] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T19:50:42.567047] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-05. [2025-11-04T19:50:42.953864] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-11-04T19:50:43.367056] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-05. [2025-11-04T19:50:43.733380] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T19:50:44.167099] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-09. [2025-11-04T19:50:44.349252] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-11-04T19:50:44.967078] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-06. [2025-11-04T19:50:45.767018] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-06. [2025-11-04T19:50:45.846199] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-11-04T19:50:45.886090] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-11-04T19:50:46.567031] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-10. [2025-11-04T19:50:46.850474] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T19:50:47.367039] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-07. [2025-11-04T19:50:48.167033] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-07. [2025-11-04T19:50:48.274028] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-11-04T19:50:48.356418] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-11-04T19:50:48.967074] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-11. [2025-11-04T19:50:49.231816] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-11-04T19:50:49.306790] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T19:50:49.308939] [INFO ] [REMOTE ] KR: Full fetch complete (77844 records). [2025-11-04T19:50:49.309042] [DEBUG] [VALIDATOR ] KR: Validating primary data. [2025-11-04T19:50:49.704884] [DEBUG] [VALIDATOR ] KR: All expected trading days are present. [2025-11-04T19:50:49.704967] [DEBUG] [VALIDATOR ] KR: Primary data validation passed. [2025-11-04T19:50:49.705923] [INFO ] [MAIN ] KR: Fetched primary data is valid. [2025-11-04T19:50:49.705986] [DEBUG] [LOCAL ] KR: Saving primary data. [2025-11-04T19:50:49.707467] [DEBUG] [LOCAL ] KR: Trimmed 77844 records to 53681 (keeping 200 days of data) [2025-11-04T19:50:49.769158] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-09. [2025-11-04T19:50:50.031250] [DEBUG] [LOCAL ] KR: Primary data saved (53681 records). [2025-11-04T19:50:50.031395] [INFO ] [MAIN ] KR: Full history saved. [2025-11-04T19:50:50.031474] [INFO ] [MAIN ] Update Progress: 1214/2367 symbols completed. KR took 37.8s. Estimated remaining: 2h 42m 12s. Rate: 7.1 symbols/min [2025-11-04T19:50:50.201077] [INFO ] [MAIN ] KRG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:50:50.243090] [DEBUG] [LOCAL ] KRG: Loaded 49194 existing records from primary storage. [2025-11-04T19:50:50.243382] [INFO ] [MAIN ] KRG: Loaded 49194 existing records. [2025-11-04T19:50:50.415392] [WARN ] [MAIN ] KRG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:50:50.415682] [INFO ] [MAIN ] KRG: Wiping existing primary data before full refresh. [2025-11-04T19:50:50.415741] [INFO ] [LOCAL ] KRG: Starting deletion of primary data. [2025-11-04T19:50:50.415955] [DEBUG] [LOCAL ] KRG: Deleted primary data file. [2025-11-04T19:50:50.416020] [INFO ] [MAIN ] KRG: Starting full primary refresh. [2025-11-04T19:50:50.574989] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-08. [2025-11-04T19:50:50.773123] [INFO ] [REMOTE ] KRG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:50:51.054134] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-11-04T19:50:51.367050] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-08. [2025-11-04T19:50:51.624577] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-11-04T19:50:52.167043] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-09. [2025-11-04T19:50:52.506551] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-11-04T19:50:52.967021] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-09. [2025-11-04T19:50:53.227898] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T19:50:53.767054] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-10. [2025-11-04T19:50:54.159572] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T19:50:54.567043] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-10. [2025-11-04T19:50:55.200783] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T19:50:55.367067] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-01. [2025-11-04T19:50:55.684132] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-11-04T19:50:56.167098] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-10. [2025-11-04T19:50:56.611186] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-11-04T19:50:56.967046] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-11. [2025-11-04T19:50:57.410045] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-11-04T19:50:57.767064] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-11. [2025-11-04T19:50:57.905791] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-11 (0.94 seconds). [2025-11-04T19:50:57.907494] [INFO ] [REMOTE ] KREF: Full fetch complete (53491 records). [2025-11-04T19:50:57.907568] [DEBUG] [VALIDATOR ] KREF: Validating primary data. [2025-11-04T19:50:58.289265] [DEBUG] [VALIDATOR ] KREF: All expected trading days are present. [2025-11-04T19:50:58.289339] [DEBUG] [VALIDATOR ] KREF: Primary data validation passed. [2025-11-04T19:50:58.290113] [INFO ] [MAIN ] KREF: Fetched primary data is valid. [2025-11-04T19:50:58.290174] [DEBUG] [LOCAL ] KREF: Saving primary data. [2025-11-04T19:50:58.291198] [DEBUG] [LOCAL ] KREF: Trimmed 53491 records to 37862 (keeping 200 days of data) [2025-11-04T19:50:58.504578] [DEBUG] [LOCAL ] KREF: Primary data saved (37862 records). [2025-11-04T19:50:58.504736] [INFO ] [MAIN ] KREF: Full history saved. [2025-11-04T19:50:58.504820] [INFO ] [MAIN ] Update Progress: 1215/2367 symbols completed. KREF took 24.6s. Estimated remaining: 2h 42m 3s. Rate: 7.1 symbols/min [2025-11-04T19:50:58.579089] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-02. [2025-11-04T19:50:58.691601] [INFO ] [MAIN ] KRMN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:50:58.691791] [DEBUG] [LOCAL ] KRMN: No existing data file found in primary storage. [2025-11-04T19:50:58.691840] [WARN ] [MAIN ] KRMN: No existing data found, performing full refresh instead. [2025-11-04T19:50:58.691946] [INFO ] [MAIN ] KRMN: Wiping existing primary data before full refresh. [2025-11-04T19:50:58.691989] [INFO ] [LOCAL ] KRMN: Starting deletion of primary data. [2025-11-04T19:50:58.692041] [DEBUG] [LOCAL ] KRMN: No primary data file found to delete. [2025-11-04T19:50:58.692121] [INFO ] [MAIN ] KRMN: Starting full primary refresh. [2025-11-04T19:50:58.982560] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-11 (1.22 seconds). [2025-11-04T19:50:59.016766] [INFO ] [REMOTE ] KOS: Full fetch complete (77047 records). [2025-11-04T19:50:59.035443] [DEBUG] [VALIDATOR ] KOS: Validating primary data. [2025-11-04T19:50:59.122152] [INFO ] [REMOTE ] KRMN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:50:59.372199] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-11. [2025-11-04T19:50:59.443329] [DEBUG] [VALIDATOR ] KOS: All expected trading days are present. [2025-11-04T19:50:59.443413] [DEBUG] [VALIDATOR ] KOS: Primary data validation passed. [2025-11-04T19:50:59.444360] [INFO ] [MAIN ] KOS: Fetched primary data is valid. [2025-11-04T19:50:59.444475] [DEBUG] [LOCAL ] KOS: Saving primary data. [2025-11-04T19:50:59.445883] [DEBUG] [LOCAL ] KOS: Trimmed 77047 records to 53087 (keeping 200 days of data) [2025-11-04T19:50:59.739859] [DEBUG] [LOCAL ] KOS: Primary data saved (53087 records). [2025-11-04T19:50:59.739981] [INFO ] [MAIN ] KOS: Full history saved. [2025-11-04T19:50:59.740057] [INFO ] [MAIN ] Update Progress: 1216/2367 symbols completed. KOS took 48.5s. Estimated remaining: 2h 41m 48s. Rate: 7.1 symbols/min [2025-11-04T19:50:59.888894] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-11-04T19:50:59.951673] [INFO ] [MAIN ] KRO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:50:59.976923] [DEBUG] [LOCAL ] KRO: Loaded 26393 existing records from primary storage. [2025-11-04T19:50:59.977185] [INFO ] [MAIN ] KRO: Loaded 26393 existing records. [2025-11-04T19:51:00.148827] [WARN ] [MAIN ] KRO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:51:00.149005] [INFO ] [MAIN ] KRO: Wiping existing primary data before full refresh. [2025-11-04T19:51:00.149051] [INFO ] [LOCAL ] KRO: Starting deletion of primary data. [2025-11-04T19:51:00.149265] [DEBUG] [LOCAL ] KRO: Deleted primary data file. [2025-11-04T19:51:00.149321] [INFO ] [MAIN ] KRO: Starting full primary refresh. [2025-11-04T19:51:00.172253] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-03. [2025-11-04T19:51:00.495509] [INFO ] [REMOTE ] KRO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:51:00.730486] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-11 (1.36 seconds). [2025-11-04T19:51:00.732304] [INFO ] [REMOTE ] KRC: Full fetch complete (70111 records). [2025-11-04T19:51:00.732389] [DEBUG] [VALIDATOR ] KRC: Validating primary data. [2025-11-04T19:51:00.972960] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-01. [2025-11-04T19:51:01.116835] [DEBUG] [VALIDATOR ] KRC: All expected trading days are present. [2025-11-04T19:51:01.116923] [DEBUG] [VALIDATOR ] KRC: Primary data validation passed. [2025-11-04T19:51:01.117843] [INFO ] [MAIN ] KRC: Fetched primary data is valid. [2025-11-04T19:51:01.117909] [DEBUG] [LOCAL ] KRC: Saving primary data. [2025-11-04T19:51:01.119346] [DEBUG] [LOCAL ] KRC: Trimmed 70111 records to 47835 (keeping 200 days of data) [2025-11-04T19:51:01.221835] [DEBUG] [REMOTE ] KRMN: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-11-04T19:51:01.226003] [WARN ] [REMOTE ] KRMN: No data returned for month 2025-01 [2025-11-04T19:51:01.228845] [FATAL] [REMOTE ] KRMN: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:51:01.231367] [FATAL] [MAIN ] KRMN: Failed to fetch primary data. [2025-11-04T19:51:01.233003] [WARN ] [MAIN ] KRMN: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:51:01.234369] [INFO ] [LOCAL ] KRMN: Starting deletion of primary data. [2025-11-04T19:51:01.237727] [DEBUG] [LOCAL ] KRMN: No primary data file found to delete. [2025-11-04T19:51:01.241381] [INFO ] [LOCAL ] KRMN: Starting deletion of derivative data. [2025-11-04T19:51:01.245794] [DEBUG] [LOCAL ] KRMN: No derivative data file found to delete. [2025-11-04T19:51:01.252792] [INFO ] [MAIN ] Update Progress: 1217/2367 symbols completed. KRMN took 2.7s. Estimated remaining: 2h 41m 33s. Rate: 7.1 symbols/min [2025-11-04T19:51:01.535069] [INFO ] [MAIN ] KRP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:51:01.627946] [DEBUG] [LOCAL ] KRP: Loaded 35387 existing records from primary storage. [2025-11-04T19:51:01.630148] [INFO ] [MAIN ] KRP: Loaded 35387 existing records. [2025-11-04T19:51:01.769152] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-01. [2025-11-04T19:51:01.797257] [DEBUG] [LOCAL ] KRC: Primary data saved (47835 records). [2025-11-04T19:51:01.804103] [INFO ] [MAIN ] KRC: Full history saved. [2025-11-04T19:51:01.806356] [INFO ] [MAIN ] Update Progress: 1218/2367 symbols completed. KRC took 42.7s. Estimated remaining: 2h 41m 17s. Rate: 7.1 symbols/min [2025-11-04T19:51:01.850305] [WARN ] [MAIN ] KRP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:51:01.873281] [INFO ] [MAIN ] KRP: Wiping existing primary data before full refresh. [2025-11-04T19:51:01.921207] [INFO ] [LOCAL ] KRP: Starting deletion of primary data. [2025-11-04T19:51:01.945510] [DEBUG] [LOCAL ] KRP: Deleted primary data file. [2025-11-04T19:51:01.974931] [INFO ] [MAIN ] KRP: Starting full primary refresh. [2025-11-04T19:51:02.092991] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-11-04T19:51:02.136714] [INFO ] [MAIN ] KSS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:51:02.489807] [DEBUG] [LOCAL ] KSS: Loaded 53560 existing records from primary storage. [2025-11-04T19:51:02.488278] [INFO ] [REMOTE ] KRP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:51:02.490834] [INFO ] [MAIN ] KSS: Loaded 53560 existing records. [2025-11-04T19:51:02.571557] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-04. [2025-11-04T19:51:02.676514] [WARN ] [MAIN ] KSS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:51:02.687521] [INFO ] [MAIN ] KSS: Wiping existing primary data before full refresh. [2025-11-04T19:51:02.687950] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-11-04T19:51:02.688050] [INFO ] [LOCAL ] KSS: Starting deletion of primary data. [2025-11-04T19:51:02.692373] [DEBUG] [LOCAL ] KSS: Deleted primary data file. [2025-11-04T19:51:02.692425] [INFO ] [MAIN ] KSS: Starting full primary refresh. [2025-11-04T19:51:03.041510] [INFO ] [REMOTE ] KSS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:51:03.367051] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-02. [2025-11-04T19:51:03.919877] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:51:04.038429] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-11-04T19:51:04.167002] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-03. [2025-11-04T19:51:04.807177] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-11-04T19:51:04.967055] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-01. [2025-11-04T19:51:05.767031] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-05. [2025-11-04T19:51:06.166589] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T19:51:06.567042] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-04. [2025-11-04T19:51:06.923084] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T19:51:07.270378] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-11-04T19:51:07.367003] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-05. [2025-11-04T19:51:08.093801] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T19:51:08.167077] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-06. [2025-11-04T19:51:08.967120] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-02. [2025-11-04T19:51:09.077179] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T19:51:09.767070] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-07. [2025-11-04T19:51:09.928001] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T19:51:10.528170] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-11-04T19:51:10.567016] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-08. [2025-11-04T19:51:11.367085] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-01. [2025-11-04T19:51:11.603510] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T19:51:12.167104] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-06. [2025-11-04T19:51:12.938212] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-11-04T19:51:12.967006] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-09. [2025-11-04T19:51:13.389085] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-11-04T19:51:13.767026] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-02. [2025-11-04T19:51:13.810065] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-11-04T19:51:14.567087] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-07. [2025-11-04T19:51:15.110975] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-11-04T19:51:15.367125] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-03. [2025-11-04T19:51:15.895095] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-11-04T19:51:16.167054] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-08. [2025-11-04T19:51:16.967019] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-10. [2025-11-04T19:51:17.095548] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-11-04T19:51:17.364999] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T19:51:17.767060] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-03. [2025-11-04T19:51:17.840506] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-11-04T19:51:18.567119] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-09. [2025-11-04T19:51:18.757939] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-11-04T19:51:19.367121] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-11. [2025-11-04T19:51:19.709466] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-11-04T19:51:20.167050] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-10. [2025-11-04T19:51:20.212091] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T19:51:20.213720] [INFO ] [REMOTE ] KRO: Full fetch complete (35861 records). [2025-11-04T19:51:20.213819] [DEBUG] [VALIDATOR ] KRO: Validating primary data. [2025-11-04T19:51:20.593769] [DEBUG] [VALIDATOR ] KRO: All expected trading days are present. [2025-11-04T19:51:20.593873] [DEBUG] [VALIDATOR ] KRO: Primary data validation passed. [2025-11-04T19:51:20.594666] [INFO ] [MAIN ] KRO: Fetched primary data is valid. [2025-11-04T19:51:20.594732] [DEBUG] [LOCAL ] KRO: Saving primary data. [2025-11-04T19:51:20.595814] [DEBUG] [LOCAL ] KRO: Trimmed 35861 records to 26409 (keeping 200 days of data) [2025-11-04T19:51:20.764895] [DEBUG] [LOCAL ] KRO: Primary data saved (26409 records). [2025-11-04T19:51:20.765038] [INFO ] [MAIN ] KRO: Full history saved. [2025-11-04T19:51:20.765123] [INFO ] [MAIN ] Update Progress: 1219/2367 symbols completed. KRO took 21.0s. Estimated remaining: 2h 41m 19s. Rate: 7.1 symbols/min [2025-11-04T19:51:20.935618] [INFO ] [MAIN ] KT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:51:20.976783] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-04. [2025-11-04T19:51:20.979617] [DEBUG] [LOCAL ] KT: Loaded 44932 existing records from primary storage. [2025-11-04T19:51:20.979890] [INFO ] [MAIN ] KT: Loaded 44932 existing records. [2025-11-04T19:51:21.156093] [WARN ] [MAIN ] KT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:51:21.156324] [INFO ] [MAIN ] KT: Wiping existing primary data before full refresh. [2025-11-04T19:51:21.156370] [INFO ] [LOCAL ] KT: Starting deletion of primary data. [2025-11-04T19:51:21.156551] [DEBUG] [LOCAL ] KT: Deleted primary data file. [2025-11-04T19:51:21.156588] [INFO ] [MAIN ] KT: Starting full primary refresh. [2025-11-04T19:51:21.510132] [INFO ] [REMOTE ] KT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:51:21.540156] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-11-04T19:51:21.767059] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-04. [2025-11-04T19:51:22.068740] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-11-04T19:51:22.567053] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-05. [2025-11-04T19:51:23.296667] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-11-04T19:51:23.367061] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-01. [2025-11-04T19:51:23.584280] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-11-04T19:51:24.167067] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-11. [2025-11-04T19:51:24.733897] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-11-04T19:51:24.967080] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-02. [2025-11-04T19:51:25.055950] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-11 (0.89 seconds). [2025-11-04T19:51:25.058762] [INFO ] [REMOTE ] KRG: Full fetch complete (71502 records). [2025-11-04T19:51:25.058912] [DEBUG] [VALIDATOR ] KRG: Validating primary data. [2025-11-04T19:51:25.469404] [DEBUG] [VALIDATOR ] KRG: All expected trading days are present. [2025-11-04T19:51:25.469493] [DEBUG] [VALIDATOR ] KRG: Primary data validation passed. [2025-11-04T19:51:25.470415] [INFO ] [MAIN ] KRG: Fetched primary data is valid. [2025-11-04T19:51:25.470479] [DEBUG] [LOCAL ] KRG: Saving primary data. [2025-11-04T19:51:25.471873] [DEBUG] [LOCAL ] KRG: Trimmed 71502 records to 49185 (keeping 200 days of data) [2025-11-04T19:51:25.767263] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-05. [2025-11-04T19:51:25.770528] [DEBUG] [LOCAL ] KRG: Primary data saved (49185 records). [2025-11-04T19:51:25.770706] [INFO ] [MAIN ] KRG: Full history saved. [2025-11-04T19:51:25.770799] [INFO ] [MAIN ] Update Progress: 1220/2367 symbols completed. KRG took 35.7s. Estimated remaining: 2h 41m 7s. Rate: 7.1 symbols/min [2025-11-04T19:51:25.942296] [INFO ] [MAIN ] KTB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:51:25.977497] [DEBUG] [LOCAL ] KTB: Loaded 39714 existing records from primary storage. [2025-11-04T19:51:25.977805] [INFO ] [MAIN ] KTB: Loaded 39714 existing records. [2025-11-04T19:51:26.166098] [WARN ] [MAIN ] KTB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:51:26.166489] [INFO ] [MAIN ] KTB: Wiping existing primary data before full refresh. [2025-11-04T19:51:26.166607] [INFO ] [LOCAL ] KTB: Starting deletion of primary data. [2025-11-04T19:51:26.166911] [DEBUG] [LOCAL ] KTB: Deleted primary data file. [2025-11-04T19:51:26.167050] [INFO ] [MAIN ] KTB: Starting full primary refresh. [2025-11-04T19:51:26.325358] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-11-04T19:51:26.574194] [INFO ] [REMOTE ] KTB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:51:26.574301] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-03. [2025-11-04T19:51:27.305350] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-11-04T19:51:27.367133] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-06. [2025-11-04T19:51:27.738544] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-11-04T19:51:28.167074] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-06. [2025-11-04T19:51:28.613852] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-11-04T19:51:28.967053] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-07. [2025-11-04T19:51:29.100447] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-11-04T19:51:29.767197] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-01. [2025-11-04T19:51:30.453442] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-11-04T19:51:30.567012] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-08. [2025-11-04T19:51:30.751563] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-11-04T19:51:31.367010] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-07. [2025-11-04T19:51:32.132847] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-11-04T19:51:32.167015] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-04. [2025-11-04T19:51:32.767302] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-11-04T19:51:32.967051] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-02. [2025-11-04T19:51:33.430147] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-11-04T19:51:33.767046] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-09. [2025-11-04T19:51:34.016952] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-11-04T19:51:34.567090] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-03. [2025-11-04T19:51:35.368153] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-11-04T19:51:35.367005] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-05. [2025-11-04T19:51:35.683157] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T19:51:36.167043] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-08. [2025-11-04T19:51:36.612766] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-11-04T19:51:36.967026] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-10. [2025-11-04T19:51:37.262382] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-11-04T19:51:37.767010] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-09. [2025-11-04T19:51:38.567032] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-06. [2025-11-04T19:51:38.666762] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-11-04T19:51:38.711812] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-11-04T19:51:39.367085] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-11. [2025-11-04T19:51:39.479155] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-11-04T19:51:40.167044] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-07. [2025-11-04T19:51:40.663165] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-11 (1.30 seconds). [2025-11-04T19:51:40.665353] [INFO ] [REMOTE ] KSS: Full fetch complete (77726 records). [2025-11-04T19:51:40.665489] [DEBUG] [VALIDATOR ] KSS: Validating primary data. [2025-11-04T19:51:40.973013] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-04. [2025-11-04T19:51:41.094231] [DEBUG] [VALIDATOR ] KSS: All expected trading days are present. [2025-11-04T19:51:41.094319] [DEBUG] [VALIDATOR ] KSS: Primary data validation passed. [2025-11-04T19:51:41.095320] [INFO ] [MAIN ] KSS: Fetched primary data is valid. [2025-11-04T19:51:41.095387] [DEBUG] [LOCAL ] KSS: Saving primary data. [2025-11-04T19:51:41.096941] [DEBUG] [LOCAL ] KSS: Trimmed 77726 records to 53560 (keeping 200 days of data) [2025-11-04T19:51:41.419385] [DEBUG] [LOCAL ] KSS: Primary data saved (53560 records). [2025-11-04T19:51:41.419598] [INFO ] [MAIN ] KSS: Full history saved. [2025-11-04T19:51:41.419851] [INFO ] [MAIN ] Update Progress: 1221/2367 symbols completed. KSS took 39.6s. Estimated remaining: 2h 41m 6s. Rate: 7.1 symbols/min [2025-11-04T19:51:41.572838] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T19:51:41.639099] [INFO ] [MAIN ] KTF: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:51:41.651358] [DEBUG] [LOCAL ] KTF: Loaded 11425 existing records from primary storage. [2025-11-04T19:51:41.651566] [INFO ] [MAIN ] KTF: Loaded 11425 existing records. [2025-11-04T19:51:41.772272] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-08. [2025-11-04T19:51:41.829701] [WARN ] [MAIN ] KTF: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:51:41.829872] [INFO ] [MAIN ] KTF: Wiping existing primary data before full refresh. [2025-11-04T19:51:41.829920] [INFO ] [LOCAL ] KTF: Starting deletion of primary data. [2025-11-04T19:51:41.830158] [DEBUG] [LOCAL ] KTF: Deleted primary data file. [2025-11-04T19:51:41.830234] [INFO ] [MAIN ] KTF: Starting full primary refresh. [2025-11-04T19:51:42.182499] [INFO ] [REMOTE ] KTF: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:51:42.208861] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-11-04T19:51:42.567049] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-01. [2025-11-04T19:51:42.995060] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T19:51:43.339745] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-11-04T19:51:43.366992] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-02. [2025-11-04T19:51:43.872149] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-11-04T19:51:44.167099] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-03. [2025-11-04T19:51:44.847765] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-11-04T19:51:44.967039] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-04. [2025-11-04T19:51:45.566420] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-11-04T19:51:45.767072] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-10. [2025-11-04T19:51:46.567018] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-05. [2025-11-04T19:51:46.762881] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-11-04T19:51:47.367059] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-05. [2025-11-04T19:51:47.566808] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-11-04T19:51:47.865357] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-11-04T19:51:48.167045] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-11. [2025-11-04T19:51:48.948570] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-11 (0.78 seconds). [2025-11-04T19:51:48.950186] [INFO ] [REMOTE ] KRP: Full fetch complete (54403 records). [2025-11-04T19:51:48.950262] [DEBUG] [VALIDATOR ] KRP: Validating primary data. [2025-11-04T19:51:48.972241] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-06. [2025-11-04T19:51:49.326590] [DEBUG] [VALIDATOR ] KRP: All expected trading days are present. [2025-11-04T19:51:49.326928] [DEBUG] [VALIDATOR ] KRP: Primary data validation passed. [2025-11-04T19:51:49.327775] [INFO ] [MAIN ] KRP: Fetched primary data is valid. [2025-11-04T19:51:49.327835] [DEBUG] [LOCAL ] KRP: Saving primary data. [2025-11-04T19:51:49.328910] [DEBUG] [LOCAL ] KRP: Trimmed 54403 records to 35387 (keeping 200 days of data) [2025-11-04T19:51:49.535526] [DEBUG] [LOCAL ] KRP: Primary data saved (35387 records). [2025-11-04T19:51:49.535680] [INFO ] [MAIN ] KRP: Full history saved. [2025-11-04T19:51:49.535775] [INFO ] [MAIN ] Update Progress: 1222/2367 symbols completed. KRP took 48.3s. Estimated remaining: 2h 40m 57s. Rate: 7.1 symbols/min [2025-11-04T19:51:49.699300] [INFO ] [MAIN ] KVUE: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:51:49.747789] [DEBUG] [LOCAL ] KVUE: Loaded 53667 existing records from primary storage. [2025-11-04T19:51:49.748132] [INFO ] [MAIN ] KVUE: Loaded 53667 existing records. [2025-11-04T19:51:49.772285] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-09. [2025-11-04T19:51:49.933459] [WARN ] [MAIN ] KVUE: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:51:49.933801] [INFO ] [MAIN ] KVUE: Wiping existing primary data before full refresh. [2025-11-04T19:51:49.933921] [INFO ] [LOCAL ] KVUE: Starting deletion of primary data. [2025-11-04T19:51:49.934312] [DEBUG] [LOCAL ] KVUE: Deleted primary data file. [2025-11-04T19:51:49.934369] [INFO ] [MAIN ] KVUE: Starting full primary refresh. [2025-11-04T19:51:50.300466] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-11-04T19:51:50.326562] [INFO ] [REMOTE ] KVUE: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:51:50.567012] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-01. [2025-11-04T19:51:51.069423] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T19:51:51.367037] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-07. [2025-11-04T19:51:51.671954] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-11-04T19:51:52.167031] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-06. [2025-11-04T19:51:52.478012] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-11-04T19:51:52.708678] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-11-04T19:51:52.967053] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-08. [2025-11-04T19:51:53.767051] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-10. [2025-11-04T19:51:54.173418] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T19:51:54.567036] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-07. [2025-11-04T19:51:54.917302] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-11-04T19:51:54.964733] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-11-04T19:51:55.367087] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-08. [2025-11-04T19:51:55.811791] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-11-04T19:51:56.167067] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-09. [2025-11-04T19:51:56.967104] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-09. [2025-11-04T19:51:57.230478] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T19:51:57.455423] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-11-04T19:51:57.767047] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-10. [2025-11-04T19:51:58.209142] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-11-04T19:51:58.567023] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-11. [2025-11-04T19:51:59.182274] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-11 (0.62 seconds). [2025-11-04T19:51:59.183809] [INFO ] [REMOTE ] KTF: Full fetch complete (18012 records). [2025-11-04T19:51:59.183882] [DEBUG] [VALIDATOR ] KTF: Validating primary data. [2025-11-04T19:51:59.367872] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-10. [2025-11-04T19:51:59.572371] [DEBUG] [VALIDATOR ] KTF: All expected trading days are present. [2025-11-04T19:51:59.572532] [DEBUG] [VALIDATOR ] KTF: Primary data validation passed. [2025-11-04T19:51:59.573382] [INFO ] [MAIN ] KTF: Fetched primary data is valid. [2025-11-04T19:51:59.573449] [DEBUG] [LOCAL ] KTF: Saving primary data. [2025-11-04T19:51:59.574092] [DEBUG] [LOCAL ] KTF: Trimmed 18012 records to 11643 (keeping 200 days of data) [2025-11-04T19:51:59.654750] [DEBUG] [LOCAL ] KTF: Primary data saved (11643 records). [2025-11-04T19:51:59.654920] [INFO ] [MAIN ] KTF: Full history saved. [2025-11-04T19:51:59.655026] [INFO ] [MAIN ] Update Progress: 1223/2367 symbols completed. KTF took 18.2s. Estimated remaining: 2h 40m 50s. Rate: 7.1 symbols/min [2025-11-04T19:51:59.854374] [INFO ] [MAIN ] KVYO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:51:59.901171] [DEBUG] [LOCAL ] KVYO: Loaded 50976 existing records from primary storage. [2025-11-04T19:51:59.901530] [INFO ] [MAIN ] KVYO: Loaded 50976 existing records. [2025-11-04T19:52:00.087718] [WARN ] [MAIN ] KVYO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:52:00.087939] [INFO ] [MAIN ] KVYO: Wiping existing primary data before full refresh. [2025-11-04T19:52:00.087988] [INFO ] [LOCAL ] KVYO: Starting deletion of primary data. [2025-11-04T19:52:00.088185] [DEBUG] [LOCAL ] KVYO: Deleted primary data file. [2025-11-04T19:52:00.088241] [INFO ] [MAIN ] KVYO: Starting full primary refresh. [2025-11-04T19:52:00.172285] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-11. [2025-11-04T19:52:00.473406] [INFO ] [REMOTE ] KVYO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:52:00.497101] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-11-04T19:52:00.967048] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-11. [2025-11-04T19:52:01.277046] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-11 (1.10 seconds). [2025-11-04T19:52:01.278856] [INFO ] [REMOTE ] KT: Full fetch complete (66364 records). [2025-11-04T19:52:01.278943] [DEBUG] [VALIDATOR ] KT: Validating primary data. [2025-11-04T19:52:01.694447] [DEBUG] [VALIDATOR ] KT: All expected trading days are present. [2025-11-04T19:52:01.694540] [DEBUG] [VALIDATOR ] KT: Primary data validation passed. [2025-11-04T19:52:01.695595] [INFO ] [MAIN ] KT: Fetched primary data is valid. [2025-11-04T19:52:01.695717] [DEBUG] [LOCAL ] KT: Saving primary data. [2025-11-04T19:52:01.697201] [DEBUG] [LOCAL ] KT: Trimmed 66364 records to 44929 (keeping 200 days of data) [2025-11-04T19:52:01.784902] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-02. [2025-11-04T19:52:01.971131] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T19:52:01.978815] [INFO ] [REMOTE ] KTB: Full fetch complete (58446 records). [2025-11-04T19:52:01.982122] [DEBUG] [VALIDATOR ] KTB: Validating primary data. [2025-11-04T19:52:01.982002] [DEBUG] [LOCAL ] KT: Primary data saved (44929 records). [2025-11-04T19:52:01.982294] [INFO ] [MAIN ] KT: Full history saved. [2025-11-04T19:52:01.982428] [INFO ] [MAIN ] Update Progress: 1224/2367 symbols completed. KT took 41.2s. Estimated remaining: 2h 40m 36s. Rate: 7.1 symbols/min [2025-11-04T19:52:02.365693] [INFO ] [MAIN ] KW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:52:02.567307] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-01. [2025-11-04T19:52:02.649890] [DEBUG] [VALIDATOR ] KTB: All expected trading days are present. [2025-11-04T19:52:02.650057] [DEBUG] [VALIDATOR ] KTB: Primary data validation passed. [2025-11-04T19:52:02.651211] [INFO ] [MAIN ] KTB: Fetched primary data is valid. [2025-11-04T19:52:02.666239] [DEBUG] [LOCAL ] KTB: Saving primary data. [2025-11-04T19:52:02.669441] [DEBUG] [LOCAL ] KW: Loaded 39919 existing records from primary storage. [2025-11-04T19:52:02.670404] [INFO ] [MAIN ] KW: Loaded 39919 existing records. [2025-11-04T19:52:02.688729] [DEBUG] [LOCAL ] KTB: Trimmed 58446 records to 39764 (keeping 200 days of data) [2025-11-04T19:52:02.970952] [WARN ] [MAIN ] KW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:52:02.972932] [INFO ] [MAIN ] KW: Wiping existing primary data before full refresh. [2025-11-04T19:52:02.973687] [INFO ] [LOCAL ] KW: Starting deletion of primary data. [2025-11-04T19:52:02.976157] [DEBUG] [LOCAL ] KW: Deleted primary data file. [2025-11-04T19:52:02.977007] [INFO ] [MAIN ] KW: Starting full primary refresh. [2025-11-04T19:52:03.489566] [INFO ] [REMOTE ] KW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:52:03.515962] [DEBUG] [LOCAL ] KTB: Primary data saved (39764 records). [2025-11-04T19:52:03.516453] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-01. [2025-11-04T19:52:03.520124] [INFO ] [MAIN ] KTB: Full history saved. [2025-11-04T19:52:03.520433] [INFO ] [MAIN ] Update Progress: 1225/2367 symbols completed. KTB took 37.7s. Estimated remaining: 2h 40m 21s. Rate: 7.1 symbols/min [2025-11-04T19:52:03.551553] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-11-04T19:52:03.705811] [INFO ] [MAIN ] KWR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:52:03.733317] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T19:52:03.754071] [DEBUG] [LOCAL ] KWR: Loaded 15357 existing records from primary storage. [2025-11-04T19:52:03.754311] [INFO ] [MAIN ] KWR: Loaded 15357 existing records. [2025-11-04T19:52:03.932470] [WARN ] [MAIN ] KWR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:52:03.932706] [INFO ] [MAIN ] KWR: Wiping existing primary data before full refresh. [2025-11-04T19:52:03.932763] [INFO ] [LOCAL ] KWR: Starting deletion of primary data. [2025-11-04T19:52:03.932919] [DEBUG] [LOCAL ] KWR: Deleted primary data file. [2025-11-04T19:52:03.932956] [INFO ] [MAIN ] KWR: Starting full primary refresh. [2025-11-04T19:52:04.172263] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-03. [2025-11-04T19:52:04.287302] [INFO ] [REMOTE ] KWR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:52:04.609183] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-11-04T19:52:04.967029] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-02. [2025-11-04T19:52:05.582421] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-11-04T19:52:05.767041] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-04. [2025-11-04T19:52:05.973054] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-11-04T19:52:06.567056] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-03. [2025-11-04T19:52:07.129783] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-11-04T19:52:07.367062] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-02. [2025-11-04T19:52:07.740419] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-11-04T19:52:08.134728] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-11-04T19:52:08.167008] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-03. [2025-11-04T19:52:08.967053] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-04. [2025-11-04T19:52:09.215415] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-11-04T19:52:09.767089] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-01. [2025-11-04T19:52:10.141926] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-11-04T19:52:10.431534] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-11-04T19:52:10.567046] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-05. [2025-11-04T19:52:11.367074] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-04. [2025-11-04T19:52:11.930598] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-11-04T19:52:12.167050] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-02. [2025-11-04T19:52:12.426122] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-11-04T19:52:12.796238] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-11-04T19:52:12.967040] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-05. [2025-11-04T19:52:13.767077] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-03. [2025-11-04T19:52:14.132921] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-11-04T19:52:14.500819] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-11-04T19:52:14.567009] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-04. [2025-11-04T19:52:15.330749] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-11-04T19:52:15.367010] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-05. [2025-11-04T19:52:16.130990] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-11-04T19:52:16.167032] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-06. [2025-11-04T19:52:16.967098] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-06. [2025-11-04T19:52:17.242482] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T19:52:17.656620] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-11-04T19:52:17.767047] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-07. [2025-11-04T19:52:18.567074] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-05. [2025-11-04T19:52:18.727494] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-11-04T19:52:19.367049] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-08. [2025-11-04T19:52:19.907333] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T19:52:20.167049] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-06. [2025-11-04T19:52:20.372031] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T19:52:20.967050] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-07. [2025-11-04T19:52:21.585327] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-11-04T19:52:21.664085] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-11-04T19:52:21.767109] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-07. [2025-11-04T19:52:22.567090] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-09. [2025-11-04T19:52:23.294665] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-11-04T19:52:23.367086] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-08. [2025-11-04T19:52:23.628103] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T19:52:24.075046] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-11-04T19:52:24.167049] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-10. [2025-11-04T19:52:24.967080] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-08. [2025-11-04T19:52:25.232542] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T19:52:25.767034] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-06. [2025-11-04T19:52:26.133592] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-11-04T19:52:26.567069] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-11. [2025-11-04T19:52:26.866993] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-11-04T19:52:27.367120] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-09. [2025-11-04T19:52:27.471251] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T19:52:27.473234] [INFO ] [REMOTE ] KW: Full fetch complete (57158 records). [2025-11-04T19:52:27.473341] [DEBUG] [VALIDATOR ] KW: Validating primary data. [2025-11-04T19:52:27.900870] [DEBUG] [VALIDATOR ] KW: All expected trading days are present. [2025-11-04T19:52:27.900998] [DEBUG] [VALIDATOR ] KW: Primary data validation passed. [2025-11-04T19:52:27.902519] [INFO ] [MAIN ] KW: Fetched primary data is valid. [2025-11-04T19:52:27.902619] [DEBUG] [LOCAL ] KW: Saving primary data. [2025-11-04T19:52:27.904460] [DEBUG] [LOCAL ] KW: Trimmed 57158 records to 39860 (keeping 200 days of data) [2025-11-04T19:52:28.154071] [DEBUG] [LOCAL ] KW: Primary data saved (39860 records). [2025-11-04T19:52:28.154301] [INFO ] [MAIN ] KW: Full history saved. [2025-11-04T19:52:28.154427] [INFO ] [MAIN ] Update Progress: 1226/2367 symbols completed. KW took 26.2s. Estimated remaining: 2h 40m 28s. Rate: 7.1 symbols/min [2025-11-04T19:52:28.176150] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-07. [2025-11-04T19:52:28.348077] [INFO ] [MAIN ] KYN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:52:28.382971] [DEBUG] [LOCAL ] KYN: Loaded 29675 existing records from primary storage. [2025-11-04T19:52:28.383397] [INFO ] [MAIN ] KYN: Loaded 29675 existing records. [2025-11-04T19:52:28.581330] [WARN ] [MAIN ] KYN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:52:28.581713] [INFO ] [MAIN ] KYN: Wiping existing primary data before full refresh. [2025-11-04T19:52:28.581802] [INFO ] [LOCAL ] KYN: Starting deletion of primary data. [2025-11-04T19:52:28.582083] [DEBUG] [LOCAL ] KYN: Deleted primary data file. [2025-11-04T19:52:28.582143] [INFO ] [MAIN ] KYN: Starting full primary refresh. [2025-11-04T19:52:28.978140] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-09. [2025-11-04T19:52:28.978030] [INFO ] [REMOTE ] KYN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:52:29.019785] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-11-04T19:52:29.379824] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-11-04T19:52:29.659265] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-11-04T19:52:29.767081] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-10. [2025-11-04T19:52:30.567071] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-08. [2025-11-04T19:52:31.367037] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-01. [2025-11-04T19:52:31.706234] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-11-04T19:52:31.783159] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-11-04T19:52:32.119968] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T19:52:32.167009] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-11. [2025-11-04T19:52:32.967238] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-02. [2025-11-04T19:52:33.767027] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-10. [2025-11-04T19:52:33.832602] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T19:52:33.854269] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-11 (1.69 seconds). [2025-11-04T19:52:33.856347] [INFO ] [REMOTE ] KVUE: Full fetch complete (77846 records). [2025-11-04T19:52:33.856475] [DEBUG] [VALIDATOR ] KVUE: Validating primary data. [2025-11-04T19:52:34.253249] [DEBUG] [VALIDATOR ] KVUE: All expected trading days are present. [2025-11-04T19:52:34.253334] [DEBUG] [VALIDATOR ] KVUE: Primary data validation passed. [2025-11-04T19:52:34.254318] [INFO ] [MAIN ] KVUE: Fetched primary data is valid. [2025-11-04T19:52:34.254383] [DEBUG] [LOCAL ] KVUE: Saving primary data. [2025-11-04T19:52:34.256780] [DEBUG] [LOCAL ] KVUE: Trimmed 77846 records to 53667 (keeping 200 days of data) [2025-11-04T19:52:34.317896] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-11-04T19:52:34.569539] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-11. [2025-11-04T19:52:34.596162] [DEBUG] [LOCAL ] KVUE: Primary data saved (53667 records). [2025-11-04T19:52:34.596312] [INFO ] [MAIN ] KVUE: Full history saved. [2025-11-04T19:52:34.596399] [INFO ] [MAIN ] Update Progress: 1227/2367 symbols completed. KVUE took 45.1s. Estimated remaining: 2h 40m 17s. Rate: 7.1 symbols/min [2025-11-04T19:52:34.772434] [INFO ] [MAIN ] L: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:52:34.809223] [DEBUG] [LOCAL ] L: Loaded 40811 existing records from primary storage. [2025-11-04T19:52:34.809515] [INFO ] [MAIN ] L: Loaded 40811 existing records. [2025-11-04T19:52:34.991224] [WARN ] [MAIN ] L: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:52:34.991459] [INFO ] [MAIN ] L: Wiping existing primary data before full refresh. [2025-11-04T19:52:34.991509] [INFO ] [LOCAL ] L: Starting deletion of primary data. [2025-11-04T19:52:34.991712] [DEBUG] [LOCAL ] L: Deleted primary data file. [2025-11-04T19:52:34.991767] [INFO ] [MAIN ] L: Starting full primary refresh. [2025-11-04T19:52:35.280390] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-11 (0.71 seconds). [2025-11-04T19:52:35.302316] [INFO ] [REMOTE ] KWR: Full fetch complete (23049 records). [2025-11-04T19:52:35.325483] [DEBUG] [VALIDATOR ] KWR: Validating primary data. [2025-11-04T19:52:35.380203] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-09. [2025-11-04T19:52:35.437352] [INFO ] [REMOTE ] L: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:52:35.731960] [DEBUG] [VALIDATOR ] KWR: All expected trading days are present. [2025-11-04T19:52:35.732033] [DEBUG] [VALIDATOR ] KWR: Primary data validation passed. [2025-11-04T19:52:35.732463] [INFO ] [MAIN ] KWR: Fetched primary data is valid. [2025-11-04T19:52:35.732534] [DEBUG] [LOCAL ] KWR: Saving primary data. [2025-11-04T19:52:35.733140] [DEBUG] [LOCAL ] KWR: Trimmed 23049 records to 15346 (keeping 200 days of data) [2025-11-04T19:52:35.824991] [DEBUG] [LOCAL ] KWR: Primary data saved (15346 records). [2025-11-04T19:52:35.825133] [INFO ] [MAIN ] KWR: Full history saved. [2025-11-04T19:52:35.825227] [INFO ] [MAIN ] Update Progress: 1228/2367 symbols completed. KWR took 32.3s. Estimated remaining: 2h 40m 2s. Rate: 7.1 symbols/min [2025-11-04T19:52:35.990356] [INFO ] [MAIN ] LAC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:52:36.036154] [DEBUG] [LOCAL ] LAC: Loaded 52653 existing records from primary storage. [2025-11-04T19:52:36.036440] [INFO ] [MAIN ] LAC: Loaded 52653 existing records. [2025-11-04T19:52:36.173277] [INFO ] [REMOTE ] L: Fetching primary data for 2025-01. [2025-11-04T19:52:36.212709] [WARN ] [MAIN ] LAC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:52:36.212890] [INFO ] [MAIN ] LAC: Wiping existing primary data before full refresh. [2025-11-04T19:52:36.212938] [INFO ] [LOCAL ] LAC: Starting deletion of primary data. [2025-11-04T19:52:36.213111] [DEBUG] [LOCAL ] LAC: Deleted primary data file. [2025-11-04T19:52:36.213147] [INFO ] [MAIN ] LAC: Starting full primary refresh. [2025-11-04T19:52:36.578967] [INFO ] [REMOTE ] LAC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:52:36.609399] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-11-04T19:52:36.967040] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-10. [2025-11-04T19:52:37.233780] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T19:52:37.767029] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-03. [2025-11-04T19:52:38.282015] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-11-04T19:52:38.561019] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-11-04T19:52:38.567132] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-01. [2025-11-04T19:52:39.367072] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-11. [2025-11-04T19:52:39.959062] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-11-04T19:52:40.167041] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-04. [2025-11-04T19:52:40.459200] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T19:52:40.461323] [INFO ] [REMOTE ] KVYO: Full fetch complete (72686 records). [2025-11-04T19:52:40.461432] [DEBUG] [VALIDATOR ] KVYO: Validating primary data. [2025-11-04T19:52:40.862120] [DEBUG] [VALIDATOR ] KVYO: All expected trading days are present. [2025-11-04T19:52:40.862201] [DEBUG] [VALIDATOR ] KVYO: Primary data validation passed. [2025-11-04T19:52:40.863131] [INFO ] [MAIN ] KVYO: Fetched primary data is valid. [2025-11-04T19:52:40.863195] [DEBUG] [LOCAL ] KVYO: Saving primary data. [2025-11-04T19:52:40.864524] [DEBUG] [LOCAL ] KVYO: Trimmed 72686 records to 51035 (keeping 200 days of data) [2025-11-04T19:52:40.973725] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-02. [2025-11-04T19:52:41.186310] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T19:52:41.206047] [DEBUG] [LOCAL ] KVYO: Primary data saved (51035 records). [2025-11-04T19:52:41.206320] [INFO ] [MAIN ] KVYO: Full history saved. [2025-11-04T19:52:41.206949] [INFO ] [MAIN ] Update Progress: 1229/2367 symbols completed. KVYO took 41.6s. Estimated remaining: 2h 39m 51s. Rate: 7.1 symbols/min [2025-11-04T19:52:41.390274] [INFO ] [MAIN ] LAD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:52:41.416747] [DEBUG] [LOCAL ] LAD: Loaded 23382 existing records from primary storage. [2025-11-04T19:52:41.417114] [INFO ] [MAIN ] LAD: Loaded 23382 existing records. [2025-11-04T19:52:41.618486] [WARN ] [MAIN ] LAD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:52:41.618865] [INFO ] [MAIN ] LAD: Wiping existing primary data before full refresh. [2025-11-04T19:52:41.618928] [INFO ] [LOCAL ] LAD: Starting deletion of primary data. [2025-11-04T19:52:41.619202] [DEBUG] [LOCAL ] LAD: Deleted primary data file. [2025-11-04T19:52:41.619265] [INFO ] [MAIN ] LAD: Starting full primary refresh. [2025-11-04T19:52:41.772695] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-05. [2025-11-04T19:52:42.057469] [INFO ] [REMOTE ] LAD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:52:42.379612] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-11-04T19:52:42.567076] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-01. [2025-11-04T19:52:42.713686] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-11-04T19:52:43.367032] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-06. [2025-11-04T19:52:43.498597] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-11-04T19:52:44.167055] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-03. [2025-11-04T19:52:44.221557] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T19:52:44.967071] [INFO ] [REMOTE ] L: Fetching primary data for 2025-02. [2025-11-04T19:52:45.639995] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-11-04T19:52:45.767063] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-07. [2025-11-04T19:52:46.052825] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-11-04T19:52:46.567072] [INFO ] [REMOTE ] L: Fetching primary data for 2025-03. [2025-11-04T19:52:46.640370] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-11-04T19:52:47.367037] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-04. [2025-11-04T19:52:47.679156] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T19:52:48.167066] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-08. [2025-11-04T19:52:48.845674] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-11-04T19:52:48.967049] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-05. [2025-11-04T19:52:49.116370] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-11-04T19:52:49.767043] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-02. [2025-11-04T19:52:50.433488] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-11-04T19:52:50.569789] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-06. [2025-11-04T19:52:50.575411] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-11-04T19:52:51.367105] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-09. [2025-11-04T19:52:51.750259] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T19:52:52.167036] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-07. [2025-11-04T19:52:52.264838] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-11-04T19:52:52.967010] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-10. [2025-11-04T19:52:53.719194] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-11-04T19:52:53.759360] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-11-04T19:52:53.770484] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-11. [2025-11-04T19:52:54.494212] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-11 (0.72 seconds). [2025-11-04T19:52:54.495750] [INFO ] [REMOTE ] KYN: Full fetch complete (44184 records). [2025-11-04T19:52:54.495836] [DEBUG] [VALIDATOR ] KYN: Validating primary data. [2025-11-04T19:52:54.572114] [INFO ] [REMOTE ] L: Fetching primary data for 2025-04. [2025-11-04T19:52:54.907244] [DEBUG] [VALIDATOR ] KYN: All expected trading days are present. [2025-11-04T19:52:54.907320] [DEBUG] [VALIDATOR ] KYN: Primary data validation passed. [2025-11-04T19:52:54.908014] [INFO ] [MAIN ] KYN: Fetched primary data is valid. [2025-11-04T19:52:54.908075] [DEBUG] [LOCAL ] KYN: Saving primary data. [2025-11-04T19:52:54.909028] [DEBUG] [LOCAL ] KYN: Trimmed 44184 records to 29711 (keeping 200 days of data) [2025-11-04T19:52:55.094711] [DEBUG] [LOCAL ] KYN: Primary data saved (29711 records). [2025-11-04T19:52:55.094942] [INFO ] [MAIN ] KYN: Full history saved. [2025-11-04T19:52:55.095052] [INFO ] [MAIN ] Update Progress: 1230/2367 symbols completed. KYN took 26.9s. Estimated remaining: 2h 39m 48s. Rate: 7.1 symbols/min [2025-11-04T19:52:55.269351] [INFO ] [MAIN ] LADR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:52:55.304699] [DEBUG] [LOCAL ] LADR: Loaded 38894 existing records from primary storage. [2025-11-04T19:52:55.304971] [INFO ] [MAIN ] LADR: Loaded 38894 existing records. [2025-11-04T19:52:55.372219] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-03. [2025-11-04T19:52:55.484136] [WARN ] [MAIN ] LADR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:52:55.484329] [INFO ] [MAIN ] LADR: Wiping existing primary data before full refresh. [2025-11-04T19:52:55.484375] [INFO ] [LOCAL ] LADR: Starting deletion of primary data. [2025-11-04T19:52:55.484577] [DEBUG] [LOCAL ] LADR: Deleted primary data file. [2025-11-04T19:52:55.484635] [INFO ] [MAIN ] LADR: Starting full primary refresh. [2025-11-04T19:52:55.849481] [INFO ] [REMOTE ] LADR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:52:55.873578] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-11-04T19:52:56.167011] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-01. [2025-11-04T19:52:56.348357] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T19:52:56.967048] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-08. [2025-11-04T19:52:57.162849] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-11-04T19:52:57.767037] [INFO ] [REMOTE ] L: Fetching primary data for 2025-05. [2025-11-04T19:52:58.505545] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-11-04T19:52:58.567050] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-04. [2025-11-04T19:52:58.873109] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-11-04T19:52:59.367014] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-02. [2025-11-04T19:52:59.408289] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-11-04T19:53:00.167014] [INFO ] [REMOTE ] L: Fetching primary data for 2025-06. [2025-11-04T19:53:00.330473] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-11-04T19:53:00.967041] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-05. [2025-11-04T19:53:01.447245] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-11-04T19:53:01.714217] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-11-04T19:53:01.767043] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-06. [2025-11-04T19:53:02.452525] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-11-04T19:53:02.567056] [INFO ] [REMOTE ] L: Fetching primary data for 2025-07. [2025-11-04T19:53:03.367070] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-09. [2025-11-04T19:53:03.733938] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-11-04T19:53:04.167035] [INFO ] [REMOTE ] L: Fetching primary data for 2025-08. [2025-11-04T19:53:04.931107] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-11-04T19:53:04.967019] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-03. [2025-11-04T19:53:05.046531] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-11-04T19:53:05.767018] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-10. [2025-11-04T19:53:05.938037] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-11-04T19:53:06.567072] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-04. [2025-11-04T19:53:07.367044] [INFO ] [REMOTE ] L: Fetching primary data for 2025-09. [2025-11-04T19:53:07.499234] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-11-04T19:53:07.971174] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-11-04T19:53:08.167039] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-11. [2025-11-04T19:53:08.448210] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T19:53:08.967074] [INFO ] [REMOTE ] L: Fetching primary data for 2025-10. [2025-11-04T19:53:09.767066] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-07. [2025-11-04T19:53:10.142293] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T19:53:10.193303] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-11 (2.03 seconds). [2025-11-04T19:53:10.195528] [INFO ] [REMOTE ] LAC: Full fetch complete (76182 records). [2025-11-04T19:53:10.195636] [DEBUG] [VALIDATOR ] LAC: Validating primary data. [2025-11-04T19:53:10.569733] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-05. [2025-11-04T19:53:10.650820] [DEBUG] [VALIDATOR ] LAC: All expected trading days are present. [2025-11-04T19:53:10.650961] [DEBUG] [VALIDATOR ] LAC: Primary data validation passed. [2025-11-04T19:53:10.651992] [INFO ] [MAIN ] LAC: Fetched primary data is valid. [2025-11-04T19:53:10.652115] [DEBUG] [LOCAL ] LAC: Saving primary data. [2025-11-04T19:53:10.669204] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T19:53:10.675201] [DEBUG] [LOCAL ] LAC: Trimmed 76182 records to 52687 (keeping 200 days of data) [2025-11-04T19:53:10.966345] [DEBUG] [LOCAL ] LAC: Primary data saved (52687 records). [2025-11-04T19:53:10.966472] [INFO ] [MAIN ] LAC: Full history saved. [2025-11-04T19:53:10.966550] [INFO ] [MAIN ] Update Progress: 1231/2367 symbols completed. LAC took 35.1s. Estimated remaining: 2h 39m 46s. Rate: 7.1 symbols/min [2025-11-04T19:53:11.133777] [INFO ] [MAIN ] LANV: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:53:11.138514] [DEBUG] [LOCAL ] LANV: Loaded 2729 existing records from primary storage. [2025-11-04T19:53:11.138723] [INFO ] [MAIN ] LANV: Loaded 2729 existing records. [2025-11-04T19:53:11.312308] [WARN ] [MAIN ] LANV: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:53:11.312508] [INFO ] [MAIN ] LANV: Wiping existing primary data before full refresh. [2025-11-04T19:53:11.312554] [INFO ] [LOCAL ] LANV: Starting deletion of primary data. [2025-11-04T19:53:11.312711] [DEBUG] [LOCAL ] LANV: Deleted primary data file. [2025-11-04T19:53:11.312767] [INFO ] [MAIN ] LANV: Starting full primary refresh. [2025-11-04T19:53:11.372201] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-08. [2025-11-04T19:53:11.683876] [INFO ] [REMOTE ] LANV: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:53:11.703562] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-11-04T19:53:12.121482] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-11-04T19:53:12.167061] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-01. [2025-11-04T19:53:12.445051] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-01 (0.28 seconds). [2025-11-04T19:53:12.967035] [INFO ] [REMOTE ] L: Fetching primary data for 2025-11. [2025-11-04T19:53:13.767094] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-06. [2025-11-04T19:53:14.023516] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T19:53:14.025230] [INFO ] [REMOTE ] L: Full fetch complete (59402 records). [2025-11-04T19:53:14.025306] [DEBUG] [VALIDATOR ] L: Validating primary data. [2025-11-04T19:53:14.406257] [DEBUG] [VALIDATOR ] L: All expected trading days are present. [2025-11-04T19:53:14.406334] [DEBUG] [VALIDATOR ] L: Primary data validation passed. [2025-11-04T19:53:14.407159] [INFO ] [MAIN ] L: Fetched primary data is valid. [2025-11-04T19:53:14.407220] [DEBUG] [LOCAL ] L: Saving primary data. [2025-11-04T19:53:14.408322] [DEBUG] [LOCAL ] L: Trimmed 59402 records to 40868 (keeping 200 days of data) [2025-11-04T19:53:14.581967] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-02. [2025-11-04T19:53:14.657632] [DEBUG] [LOCAL ] L: Primary data saved (40868 records). [2025-11-04T19:53:14.657771] [INFO ] [MAIN ] L: Full history saved. [2025-11-04T19:53:14.657845] [INFO ] [MAIN ] Update Progress: 1232/2367 symbols completed. L took 40.1s. Estimated remaining: 2h 39m 33s. Rate: 7.1 symbols/min [2025-11-04T19:53:14.832364] [INFO ] [MAIN ] LAR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:53:14.832540] [DEBUG] [LOCAL ] LAR: No existing data file found in primary storage. [2025-11-04T19:53:14.832604] [WARN ] [MAIN ] LAR: No existing data found, performing full refresh instead. [2025-11-04T19:53:14.832844] [INFO ] [MAIN ] LAR: Wiping existing primary data before full refresh. [2025-11-04T19:53:14.833014] [INFO ] [LOCAL ] LAR: Starting deletion of primary data. [2025-11-04T19:53:14.833071] [DEBUG] [LOCAL ] LAR: No primary data file found to delete. [2025-11-04T19:53:14.840139] [INFO ] [MAIN ] LAR: Starting full primary refresh. [2025-11-04T19:53:14.917768] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T19:53:14.964164] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T19:53:15.220050] [INFO ] [REMOTE ] LAR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:53:15.367015] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-01. [2025-11-04T19:53:15.965574] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-11-04T19:53:16.167054] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-03. [2025-11-04T19:53:16.499501] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-11-04T19:53:16.967035] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-09. [2025-11-04T19:53:17.767046] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-07. [2025-11-04T19:53:17.811357] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-11-04T19:53:18.567037] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-10. [2025-11-04T19:53:18.918874] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-11-04T19:53:19.367003] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-08. [2025-11-04T19:53:19.507793] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-11-04T19:53:20.167028] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-11. [2025-11-04T19:53:20.323001] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-11-04T19:53:20.967080] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-02. [2025-11-04T19:53:21.007252] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-11 (0.84 seconds). [2025-11-04T19:53:21.008882] [INFO ] [REMOTE ] LAD: Full fetch complete (35950 records). [2025-11-04T19:53:21.008968] [DEBUG] [VALIDATOR ] LAD: Validating primary data. [2025-11-04T19:53:21.385210] [DEBUG] [VALIDATOR ] LAD: All expected trading days are present. [2025-11-04T19:53:21.385284] [DEBUG] [VALIDATOR ] LAD: Primary data validation passed. [2025-11-04T19:53:21.385915] [INFO ] [MAIN ] LAD: Fetched primary data is valid. [2025-11-04T19:53:21.385975] [DEBUG] [LOCAL ] LAD: Saving primary data. [2025-11-04T19:53:21.386728] [DEBUG] [LOCAL ] LAD: Trimmed 35950 records to 23528 (keeping 200 days of data) [2025-11-04T19:53:21.536891] [DEBUG] [LOCAL ] LAD: Primary data saved (23528 records). [2025-11-04T19:53:21.537017] [INFO ] [MAIN ] LAD: Full history saved. [2025-11-04T19:53:21.537092] [INFO ] [MAIN ] Update Progress: 1233/2367 symbols completed. LAD took 40.3s. Estimated remaining: 2h 39m 23s. Rate: 7.1 symbols/min [2025-11-04T19:53:21.705960] [INFO ] [MAIN ] LAW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:53:21.721711] [DEBUG] [LOCAL ] LAW: Loaded 15124 existing records from primary storage. [2025-11-04T19:53:21.721945] [INFO ] [MAIN ] LAW: Loaded 15124 existing records. [2025-11-04T19:53:21.775693] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-04. [2025-11-04T19:53:21.918978] [WARN ] [MAIN ] LAW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:53:21.919191] [INFO ] [MAIN ] LAW: Wiping existing primary data before full refresh. [2025-11-04T19:53:21.919238] [INFO ] [LOCAL ] LAW: Starting deletion of primary data. [2025-11-04T19:53:21.919407] [DEBUG] [LOCAL ] LAW: Deleted primary data file. [2025-11-04T19:53:21.919452] [INFO ] [MAIN ] LAW: Starting full primary refresh. [2025-11-04T19:53:22.146411] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-11-04T19:53:22.295260] [INFO ] [REMOTE ] LAW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:53:22.319785] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-11-04T19:53:22.567038] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-03. [2025-11-04T19:53:23.367043] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-09. [2025-11-04T19:53:23.767083] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-11-04T19:53:24.167055] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-04. [2025-11-04T19:53:24.449470] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-11-04T19:53:24.967051] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-05. [2025-11-04T19:53:25.183515] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-05 (0.22 seconds). [2025-11-04T19:53:25.392158] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-11-04T19:53:25.767037] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-06. [2025-11-04T19:53:26.101847] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-11-04T19:53:26.567056] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-07. [2025-11-04T19:53:26.865446] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-07 (0.30 seconds). [2025-11-04T19:53:27.367098] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-08. [2025-11-04T19:53:27.716801] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-11-04T19:53:28.167040] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-01. [2025-11-04T19:53:28.889660] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-11-04T19:53:28.967040] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-09. [2025-11-04T19:53:29.338996] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-11-04T19:53:29.767042] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-05. [2025-11-04T19:53:30.567052] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-10. [2025-11-04T19:53:30.841632] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-10 (0.27 seconds). [2025-11-04T19:53:30.870599] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-11-04T19:53:31.367043] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-10. [2025-11-04T19:53:32.167033] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-06. [2025-11-04T19:53:32.312209] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T19:53:32.967125] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-11. [2025-11-04T19:53:33.184967] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-11-04T19:53:33.306398] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-11 (0.34 seconds). [2025-11-04T19:53:33.307150] [INFO ] [REMOTE ] LANV: Full fetch complete (4056 records). [2025-11-04T19:53:33.307216] [DEBUG] [VALIDATOR ] LANV: Validating primary data. [2025-11-04T19:53:33.682537] [DEBUG] [VALIDATOR ] LANV: All expected trading days are present. [2025-11-04T19:53:33.682613] [DEBUG] [VALIDATOR ] LANV: Primary data validation passed. [2025-11-04T19:53:33.682760] [INFO ] [MAIN ] LANV: Fetched primary data is valid. [2025-11-04T19:53:33.682816] [DEBUG] [LOCAL ] LANV: Saving primary data. [2025-11-04T19:53:33.683156] [DEBUG] [LOCAL ] LANV: Trimmed 4056 records to 2739 (keeping 200 days of data) [2025-11-04T19:53:33.698083] [DEBUG] [LOCAL ] LANV: Primary data saved (2739 records). [2025-11-04T19:53:33.698154] [INFO ] [MAIN ] LANV: Full history saved. [2025-11-04T19:53:33.698224] [INFO ] [MAIN ] Update Progress: 1234/2367 symbols completed. LANV took 22.7s. Estimated remaining: 2h 39m 18s. Rate: 7.1 symbols/min [2025-11-04T19:53:33.772286] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-02. [2025-11-04T19:53:33.880336] [INFO ] [MAIN ] LAZ: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:53:33.924448] [DEBUG] [LOCAL ] LAZ: Loaded 44251 existing records from primary storage. [2025-11-04T19:53:33.924851] [INFO ] [MAIN ] LAZ: Loaded 44251 existing records. [2025-11-04T19:53:34.123596] [WARN ] [MAIN ] LAZ: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:53:34.123837] [INFO ] [MAIN ] LAZ: Wiping existing primary data before full refresh. [2025-11-04T19:53:34.123884] [INFO ] [LOCAL ] LAZ: Starting deletion of primary data. [2025-11-04T19:53:34.124076] [DEBUG] [LOCAL ] LAZ: Deleted primary data file. [2025-11-04T19:53:34.124131] [INFO ] [MAIN ] LAZ: Starting full primary refresh. [2025-11-04T19:53:34.485271] [INFO ] [REMOTE ] LAZ: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:53:34.486348] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-11-04T19:53:34.567014] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-01. [2025-11-04T19:53:35.367085] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-11. [2025-11-04T19:53:35.628501] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-11-04T19:53:36.167039] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-02. [2025-11-04T19:53:36.393045] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-11 (1.03 seconds). [2025-11-04T19:53:36.394790] [INFO ] [REMOTE ] LADR: Full fetch complete (55887 records). [2025-11-04T19:53:36.394887] [DEBUG] [VALIDATOR ] LADR: Validating primary data. [2025-11-04T19:53:36.784877] [DEBUG] [VALIDATOR ] LADR: All expected trading days are present. [2025-11-04T19:53:36.784976] [DEBUG] [VALIDATOR ] LADR: Primary data validation passed. [2025-11-04T19:53:36.785880] [INFO ] [MAIN ] LADR: Fetched primary data is valid. [2025-11-04T19:53:36.785971] [DEBUG] [LOCAL ] LADR: Saving primary data. [2025-11-04T19:53:36.787291] [DEBUG] [LOCAL ] LADR: Trimmed 55887 records to 38923 (keeping 200 days of data) [2025-11-04T19:53:36.990634] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-07. [2025-11-04T19:53:37.027187] [DEBUG] [LOCAL ] LADR: Primary data saved (38923 records). [2025-11-04T19:53:37.027314] [INFO ] [MAIN ] LADR: Full history saved. [2025-11-04T19:53:37.027396] [INFO ] [MAIN ] Update Progress: 1235/2367 symbols completed. LADR took 41.9s. Estimated remaining: 2h 39m 5s. Rate: 7.1 symbols/min [2025-11-04T19:53:37.206892] [INFO ] [MAIN ] LB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:53:37.234047] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-11-04T19:53:37.270349] [DEBUG] [LOCAL ] LB: Loaded 36285 existing records from primary storage. [2025-11-04T19:53:37.270691] [INFO ] [MAIN ] LB: Loaded 36285 existing records. [2025-11-04T19:53:37.458088] [WARN ] [MAIN ] LB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:53:37.458301] [INFO ] [MAIN ] LB: Wiping existing primary data before full refresh. [2025-11-04T19:53:37.458350] [INFO ] [LOCAL ] LB: Starting deletion of primary data. [2025-11-04T19:53:37.458552] [DEBUG] [LOCAL ] LB: Deleted primary data file. [2025-11-04T19:53:37.458609] [INFO ] [MAIN ] LB: Starting full primary refresh. [2025-11-04T19:53:37.772158] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-03. [2025-11-04T19:53:37.835119] [INFO ] [REMOTE ] LB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:53:38.086110] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-11-04T19:53:38.567034] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-03. [2025-11-04T19:53:38.903159] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-11-04T19:53:38.988920] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-11-04T19:53:39.367046] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-08. [2025-11-04T19:53:40.167046] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-04. [2025-11-04T19:53:40.528064] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-11-04T19:53:40.967041] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-01. [2025-11-04T19:53:41.367175] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-11-04T19:53:41.767055] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-09. [2025-11-04T19:53:42.200914] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-11-04T19:53:42.567091] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-05. [2025-11-04T19:53:43.034163] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-11-04T19:53:43.367041] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-02. [2025-11-04T19:53:43.691267] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-11-04T19:53:44.167073] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-06. [2025-11-04T19:53:44.448812] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T19:53:44.967030] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-04. [2025-11-04T19:53:45.174493] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-11-04T19:53:45.480028] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-11-04T19:53:45.767044] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-05. [2025-11-04T19:53:46.374585] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-11-04T19:53:46.567018] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-06. [2025-11-04T19:53:47.180726] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T19:53:47.367040] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-03. [2025-11-04T19:53:48.167034] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-07. [2025-11-04T19:53:48.486127] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T19:53:48.719243] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-11-04T19:53:48.967061] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-08. [2025-11-04T19:53:49.618853] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-11-04T19:53:49.767032] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-09. [2025-11-04T19:53:49.824041] [DEBUG] [REMOTE ] LAW: Error fetching primary data for 2025-09: 'API call returned an empty response.' [2025-11-04T19:53:49.824229] [WARN ] [REMOTE ] LAW: No data returned for month 2025-09 [2025-11-04T19:53:49.824350] [FATAL] [REMOTE ] LAW: API call failed - aborting fetch to avoid wasting further API calls [2025-11-04T19:53:49.824470] [FATAL] [MAIN ] LAW: Failed to fetch primary data. [2025-11-04T19:53:49.824556] [WARN ] [MAIN ] LAW: Deleting existing primary and derivative data due to fetch failure. [2025-11-04T19:53:49.824674] [INFO ] [LOCAL ] LAW: Starting deletion of primary data. [2025-11-04T19:53:49.824775] [DEBUG] [LOCAL ] LAW: No primary data file found to delete. [2025-11-04T19:53:49.824816] [INFO ] [LOCAL ] LAW: Starting deletion of derivative data. [2025-11-04T19:53:49.824949] [DEBUG] [LOCAL ] LAW: Deleted derivative data file. [2025-11-04T19:53:49.825031] [INFO ] [MAIN ] Update Progress: 1236/2367 symbols completed. LAW took 28.3s. Estimated remaining: 2h 39m 1s. Rate: 7.1 symbols/min [2025-11-04T19:53:49.997188] [INFO ] [MAIN ] LBRT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:53:50.045948] [DEBUG] [LOCAL ] LBRT: Loaded 53010 existing records from primary storage. [2025-11-04T19:53:50.046267] [INFO ] [MAIN ] LBRT: Loaded 53010 existing records. [2025-11-04T19:53:50.223201] [WARN ] [MAIN ] LBRT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:53:50.223387] [INFO ] [MAIN ] LBRT: Wiping existing primary data before full refresh. [2025-11-04T19:53:50.223435] [INFO ] [LOCAL ] LBRT: Starting deletion of primary data. [2025-11-04T19:53:50.223860] [DEBUG] [LOCAL ] LBRT: Deleted primary data file. [2025-11-04T19:53:50.223916] [INFO ] [MAIN ] LBRT: Starting full primary refresh. [2025-11-04T19:53:50.570790] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-07. [2025-11-04T19:53:50.584566] [INFO ] [REMOTE ] LBRT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:53:51.367080] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-04. [2025-11-04T19:53:51.704284] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-11-04T19:53:52.167046] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-01. [2025-11-04T19:53:52.391863] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T19:53:52.967050] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-08. [2025-11-04T19:53:53.615347] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-11-04T19:53:53.767041] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-02. [2025-11-04T19:53:53.844993] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-11-04T19:53:54.567047] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-09. [2025-11-04T19:53:55.025052] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-11-04T19:53:55.367096] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-05. [2025-11-04T19:53:55.657356] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-11-04T19:53:56.167023] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-10. [2025-11-04T19:53:56.240256] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T19:53:56.967099] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-03. [2025-11-04T19:53:57.740082] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-11-04T19:53:57.767011] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-11. [2025-11-04T19:53:58.306016] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-11-04T19:53:58.567036] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-10. [2025-11-04T19:53:59.130720] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-11 (1.36 seconds). [2025-11-04T19:53:59.133414] [INFO ] [REMOTE ] LAR: Full fetch complete (65832 records). [2025-11-04T19:53:59.133566] [DEBUG] [VALIDATOR ] LAR: Validating primary data. [2025-11-04T19:53:59.337371] [WARN ] [VALIDATOR ] LAR: Validation failed - insufficient historical data. Have 196 unique days, need 200 days [2025-11-04T19:53:59.338218] [FATAL] [MAIN ] LAR: Fetched primary data is invalid. Refusing to save it. [2025-11-04T19:53:59.338302] [WARN ] [MAIN ] LAR: Deleting existing primary and derivative data due to validation failure. [2025-11-04T19:53:59.338393] [INFO ] [LOCAL ] LAR: Starting deletion of primary data. [2025-11-04T19:53:59.338506] [DEBUG] [LOCAL ] LAR: No primary data file found to delete. [2025-11-04T19:53:59.338567] [INFO ] [LOCAL ] LAR: Starting deletion of derivative data. [2025-11-04T19:53:59.338614] [DEBUG] [LOCAL ] LAR: No derivative data file found to delete. [2025-11-04T19:53:59.338714] [INFO ] [MAIN ] Update Progress: 1237/2367 symbols completed. LAR took 44.7s. Estimated remaining: 2h 38m 53s. Rate: 7.1 symbols/min [2025-11-04T19:53:59.367055] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-06. [2025-11-04T19:53:59.634277] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-11-04T19:54:00.167018] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-11. [2025-11-04T19:54:00.370924] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-11-04T19:54:00.967077] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-07. [2025-11-04T19:54:01.255046] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-11 (1.09 seconds). [2025-11-04T19:54:01.256938] [INFO ] [REMOTE ] LAZ: Full fetch complete (65257 records). [2025-11-04T19:54:01.257058] [DEBUG] [VALIDATOR ] LAZ: Validating primary data. [2025-11-04T19:54:01.686366] [DEBUG] [VALIDATOR ] LAZ: All expected trading days are present. [2025-11-04T19:54:01.686462] [DEBUG] [VALIDATOR ] LAZ: Primary data validation passed. [2025-11-04T19:54:01.687449] [INFO ] [MAIN ] LAZ: Fetched primary data is valid. [2025-11-04T19:54:01.687525] [DEBUG] [LOCAL ] LAZ: Saving primary data. [2025-11-04T19:54:01.688889] [DEBUG] [LOCAL ] LAZ: Trimmed 65257 records to 44312 (keeping 200 days of data) [2025-11-04T19:54:01.785844] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-04. [2025-11-04T19:54:01.999193] [DEBUG] [LOCAL ] LAZ: Primary data saved (44312 records). [2025-11-04T19:54:01.999567] [INFO ] [MAIN ] LAZ: Full history saved. [2025-11-04T19:54:01.999736] [INFO ] [MAIN ] Update Progress: 1238/2367 symbols completed. LAZ took 28.3s. Estimated remaining: 2h 38m 40s. Rate: 7.1 symbols/min [2025-11-04T19:54:02.088564] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-11-04T19:54:02.567026] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-08. [2025-11-04T19:54:03.108856] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T19:54:03.367073] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-05. [2025-11-04T19:54:03.588331] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-11-04T19:54:04.167046] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-09. [2025-11-04T19:54:04.651311] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T19:54:04.967105] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-06. [2025-11-04T19:54:05.332866] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-11-04T19:54:05.767019] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-10. [2025-11-04T19:54:06.151103] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-11-04T19:54:06.567079] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-07. [2025-11-04T19:54:06.664769] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-11-04T19:54:07.367095] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-11. [2025-11-04T19:54:07.923169] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-11-04T19:54:08.165513] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-11 (0.80 seconds). [2025-11-04T19:54:08.166982] [INFO ] [REMOTE ] LB: Full fetch complete (54981 records). [2025-11-04T19:54:08.167024] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-08. [2025-11-04T19:54:08.167164] [DEBUG] [VALIDATOR ] LB: Validating primary data. [2025-11-04T19:54:08.584235] [DEBUG] [VALIDATOR ] LB: All expected trading days are present. [2025-11-04T19:54:08.584304] [DEBUG] [VALIDATOR ] LB: Primary data validation passed. [2025-11-04T19:54:08.585250] [INFO ] [MAIN ] LB: Fetched primary data is valid. [2025-11-04T19:54:08.585337] [DEBUG] [LOCAL ] LB: Saving primary data. [2025-11-04T19:54:08.586621] [DEBUG] [LOCAL ] LB: Trimmed 54981 records to 36295 (keeping 200 days of data) [2025-11-04T19:54:08.807441] [DEBUG] [LOCAL ] LB: Primary data saved (36295 records). [2025-11-04T19:54:08.807581] [INFO ] [MAIN ] LB: Full history saved. [2025-11-04T19:54:08.807688] [INFO ] [MAIN ] Update Progress: 1239/2367 symbols completed. LB took 31.8s. Estimated remaining: 2h 38m 30s. Rate: 7.1 symbols/min [2025-11-04T19:54:09.463587] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-11-04T19:54:09.474628] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-09. [2025-11-04T19:54:10.802210] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-11-04T19:54:10.813559] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-10. [2025-11-04T19:54:12.312989] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-11-04T19:54:12.325369] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-11. [2025-11-04T19:54:13.677754] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-11 (1.35 seconds). [2025-11-04T19:54:13.679786] [INFO ] [REMOTE ] LBRT: Full fetch complete (76946 records). [2025-11-04T19:54:13.679911] [DEBUG] [VALIDATOR ] LBRT: Validating primary data. [2025-11-04T19:54:14.062528] [DEBUG] [VALIDATOR ] LBRT: All expected trading days are present. [2025-11-04T19:54:14.062620] [DEBUG] [VALIDATOR ] LBRT: Primary data validation passed. [2025-11-04T19:54:14.063576] [INFO ] [MAIN ] LBRT: Fetched primary data is valid. [2025-11-04T19:54:14.063691] [DEBUG] [LOCAL ] LBRT: Saving primary data. [2025-11-04T19:54:14.065075] [DEBUG] [LOCAL ] LBRT: Trimmed 76946 records to 53008 (keeping 200 days of data) [2025-11-04T19:54:14.371770] [DEBUG] [LOCAL ] LBRT: Primary data saved (53008 records). [2025-11-04T19:54:14.371892] [INFO ] [MAIN ] LBRT: Full history saved. [2025-11-04T19:54:14.371967] [INFO ] [MAIN ] Update Progress: 1240/2367 symbols completed. LBRT took 24.5s. Estimated remaining: 2h 38m 19s. Rate: 7.1 symbols/min [2025-11-04T19:54:15.082008] [INFO ] [MAIN ] LDI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:54:15.083013] [INFO ] [MAIN ] LDOS: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:54:15.056757] [INFO ] [MAIN ] LC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:54:15.108268] [INFO ] [MAIN ] LCII: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:54:15.165296] [DEBUG] [LOCAL ] LCII: Loaded 26185 existing records from primary storage. [2025-11-04T19:54:15.165627] [INFO ] [MAIN ] LCII: Loaded 26185 existing records. [2025-11-04T19:54:15.288139] [DEBUG] [LOCAL ] LDOS: Loaded 44017 existing records from primary storage. [2025-11-04T19:54:15.301027] [DEBUG] [LOCAL ] LC: Loaded 48288 existing records from primary storage. [2025-11-04T19:54:15.312844] [INFO ] [MAIN ] LDOS: Loaded 44017 existing records. [2025-11-04T19:54:15.320135] [INFO ] [MAIN ] LC: Loaded 48288 existing records. [2025-11-04T19:54:15.366894] [DEBUG] [LOCAL ] LDI: Loaded 45671 existing records from primary storage. [2025-11-04T19:54:15.449416] [WARN ] [MAIN ] LCII: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:54:15.482056] [INFO ] [MAIN ] LDI: Loaded 45671 existing records. [2025-11-04T19:54:15.610890] [INFO ] [MAIN ] LCII: Wiping existing primary data before full refresh. [2025-11-04T19:54:15.648509] [INFO ] [LOCAL ] LCII: Starting deletion of primary data. [2025-11-04T19:54:15.721144] [DEBUG] [LOCAL ] LCII: Deleted primary data file. [2025-11-04T19:54:15.858857] [INFO ] [MAIN ] LCII: Starting full primary refresh. [2025-11-04T19:54:15.911566] [WARN ] [MAIN ] LDOS: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:54:15.919886] [WARN ] [MAIN ] LC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:54:16.000590] [INFO ] [MAIN ] LC: Wiping existing primary data before full refresh. [2025-11-04T19:54:15.970217] [INFO ] [MAIN ] LDOS: Wiping existing primary data before full refresh. [2025-11-04T19:54:16.009376] [INFO ] [LOCAL ] LC: Starting deletion of primary data. [2025-11-04T19:54:16.026046] [INFO ] [LOCAL ] LDOS: Starting deletion of primary data. [2025-11-04T19:54:16.066123] [WARN ] [MAIN ] LDI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:54:16.071299] [DEBUG] [LOCAL ] LC: Deleted primary data file. [2025-11-04T19:54:16.086993] [INFO ] [MAIN ] LDI: Wiping existing primary data before full refresh. [2025-11-04T19:54:16.086922] [INFO ] [MAIN ] LC: Starting full primary refresh. [2025-11-04T19:54:16.093507] [DEBUG] [LOCAL ] LDOS: Deleted primary data file. [2025-11-04T19:54:16.108749] [INFO ] [LOCAL ] LDI: Starting deletion of primary data. [2025-11-04T19:54:16.145320] [INFO ] [MAIN ] LDOS: Starting full primary refresh. [2025-11-04T19:54:16.255035] [DEBUG] [LOCAL ] LDI: Deleted primary data file. [2025-11-04T19:54:16.304418] [INFO ] [MAIN ] LDI: Starting full primary refresh. [2025-11-04T19:54:17.083559] [INFO ] [REMOTE ] LCII: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:54:17.175187] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-01. [2025-11-04T19:54:17.404266] [INFO ] [REMOTE ] LC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:54:17.398574] [INFO ] [REMOTE ] LDOS: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:54:17.415305] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-01. [2025-11-04T19:54:17.420721] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-01. [2025-11-04T19:54:17.486200] [INFO ] [REMOTE ] LDI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:54:17.486555] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-01. [2025-11-04T19:54:18.036686] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T19:54:18.041547] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-02. [2025-11-04T19:54:18.377543] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-11-04T19:54:18.384296] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-02. [2025-11-04T19:54:18.753305] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-11-04T19:54:18.763086] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-02. [2025-11-04T19:54:18.798842] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-11-04T19:54:18.808732] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-02. [2025-11-04T19:54:18.913978] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T19:54:18.919257] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-03. [2025-11-04T19:54:19.210581] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T19:54:19.218040] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-03. [2025-11-04T19:54:19.852452] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-11-04T19:54:19.859477] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-04. [2025-11-04T19:54:19.908874] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-11-04T19:54:19.967815] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T19:54:20.167003] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-03. [2025-11-04T19:54:20.469197] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-11-04T19:54:20.842411] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-11-04T19:54:20.967004] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-05. [2025-11-04T19:54:21.383021] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-11-04T19:54:21.767012] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-03. [2025-11-04T19:54:21.797430] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-11-04T19:54:22.567040] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-04. [2025-11-04T19:54:22.958587] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-11-04T19:54:23.367035] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-06. [2025-11-04T19:54:23.614728] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-11-04T19:54:24.167012] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-05. [2025-11-04T19:54:24.218901] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-11-04T19:54:24.967033] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-07. [2025-11-04T19:54:25.159623] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-11-04T19:54:25.767026] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-06. [2025-11-04T19:54:25.864808] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-11-04T19:54:26.567080] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-08. [2025-11-04T19:54:26.818866] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-11-04T19:54:27.367024] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-04. [2025-11-04T19:54:27.390267] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-11-04T19:54:28.167041] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-07. [2025-11-04T19:54:28.591055] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-11-04T19:54:28.967065] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-04. [2025-11-04T19:54:29.348802] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-11-04T19:54:29.767062] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-05. [2025-11-04T19:54:30.065330] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-11-04T19:54:30.567069] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-08. [2025-11-04T19:54:30.970096] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-11-04T19:54:31.367052] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-06. [2025-11-04T19:54:31.655708] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-11-04T19:54:32.167095] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-09. [2025-11-04T19:54:32.541977] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T19:54:32.967027] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-07. [2025-11-04T19:54:33.767073] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-05. [2025-11-04T19:54:33.880300] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-11-04T19:54:34.217520] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T19:54:34.567035] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-10. [2025-11-04T19:54:34.840255] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-11-04T19:54:35.367090] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-08. [2025-11-04T19:54:36.126148] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-11-04T19:54:36.167032] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-11. [2025-11-04T19:54:36.556768] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-11-04T19:54:36.967063] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-06. [2025-11-04T19:54:37.411106] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-11 (1.24 seconds). [2025-11-04T19:54:37.412977] [INFO ] [REMOTE ] LDI: Full fetch complete (63257 records). [2025-11-04T19:54:37.413074] [DEBUG] [VALIDATOR ] LDI: Validating primary data. [2025-11-04T19:54:37.774267] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-09. [2025-11-04T19:54:37.800851] [DEBUG] [VALIDATOR ] LDI: All expected trading days are present. [2025-11-04T19:54:37.800935] [DEBUG] [VALIDATOR ] LDI: Primary data validation passed. [2025-11-04T19:54:37.801857] [INFO ] [MAIN ] LDI: Fetched primary data is valid. [2025-11-04T19:54:37.801923] [DEBUG] [LOCAL ] LDI: Saving primary data. [2025-11-04T19:54:37.803270] [DEBUG] [LOCAL ] LDI: Trimmed 63257 records to 45708 (keeping 200 days of data) [2025-11-04T19:54:38.062982] [DEBUG] [LOCAL ] LDI: Primary data saved (45708 records). [2025-11-04T19:54:38.076754] [INFO ] [MAIN ] LDI: Full history saved. [2025-11-04T19:54:38.076915] [INFO ] [MAIN ] Update Progress: 1241/2367 symbols completed. LDI took 23.7s. Estimated remaining: 2h 38m 24s. Rate: 7.1 symbols/min [2025-11-04T19:54:38.078344] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-11-04T19:54:38.254437] [INFO ] [MAIN ] LDP: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:54:38.267173] [DEBUG] [LOCAL ] LDP: Loaded 12054 existing records from primary storage. [2025-11-04T19:54:38.267363] [INFO ] [MAIN ] LDP: Loaded 12054 existing records. [2025-11-04T19:54:38.441511] [WARN ] [MAIN ] LDP: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:54:38.441730] [INFO ] [MAIN ] LDP: Wiping existing primary data before full refresh. [2025-11-04T19:54:38.441805] [INFO ] [LOCAL ] LDP: Starting deletion of primary data. [2025-11-04T19:54:38.441953] [DEBUG] [LOCAL ] LDP: Deleted primary data file. [2025-11-04T19:54:38.441991] [INFO ] [MAIN ] LDP: Starting full primary refresh. [2025-11-04T19:54:38.572192] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-07. [2025-11-04T19:54:38.806605] [INFO ] [REMOTE ] LDP: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:54:38.976496] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-11-04T19:54:39.367078] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-01. [2025-11-04T19:54:39.977396] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-11-04T19:54:39.979385] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-11-04T19:54:40.167064] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-10. [2025-11-04T19:54:40.967028] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-02. [2025-11-04T19:54:41.271009] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T19:54:41.469147] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-11-04T19:54:41.767052] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-11. [2025-11-04T19:54:42.567044] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-08. [2025-11-04T19:54:43.337138] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-11 (1.57 seconds). [2025-11-04T19:54:43.339063] [INFO ] [REMOTE ] LDOS: Full fetch complete (66467 records). [2025-11-04T19:54:43.339194] [DEBUG] [VALIDATOR ] LDOS: Validating primary data. [2025-11-04T19:54:43.372515] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-03. [2025-11-04T19:54:43.749854] [DEBUG] [VALIDATOR ] LDOS: All expected trading days are present. [2025-11-04T19:54:43.749969] [DEBUG] [VALIDATOR ] LDOS: Primary data validation passed. [2025-11-04T19:54:43.751025] [INFO ] [MAIN ] LDOS: Fetched primary data is valid. [2025-11-04T19:54:43.751101] [DEBUG] [LOCAL ] LDOS: Saving primary data. [2025-11-04T19:54:43.752639] [DEBUG] [LOCAL ] LDOS: Trimmed 66467 records to 44024 (keeping 200 days of data) [2025-11-04T19:54:43.902611] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-11-04T19:54:44.073281] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-11-04T19:54:44.083841] [DEBUG] [LOCAL ] LDOS: Primary data saved (44024 records). [2025-11-04T19:54:44.083913] [INFO ] [MAIN ] LDOS: Full history saved. [2025-11-04T19:54:44.083983] [INFO ] [MAIN ] Update Progress: 1242/2367 symbols completed. LDOS took 29.7s. Estimated remaining: 2h 38m 13s. Rate: 7.1 symbols/min [2025-11-04T19:54:44.172129] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-09. [2025-11-04T19:54:44.290057] [INFO ] [MAIN ] LEA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:54:44.328382] [DEBUG] [LOCAL ] LEA: Loaded 40836 existing records from primary storage. [2025-11-04T19:54:44.328780] [INFO ] [MAIN ] LEA: Loaded 40836 existing records. [2025-11-04T19:54:44.509554] [WARN ] [MAIN ] LEA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:54:44.509793] [INFO ] [MAIN ] LEA: Wiping existing primary data before full refresh. [2025-11-04T19:54:44.509841] [INFO ] [LOCAL ] LEA: Starting deletion of primary data. [2025-11-04T19:54:44.510037] [DEBUG] [LOCAL ] LEA: Deleted primary data file. [2025-11-04T19:54:44.510092] [INFO ] [MAIN ] LEA: Starting full primary refresh. [2025-11-04T19:54:44.932705] [INFO ] [REMOTE ] LEA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:54:44.967110] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-01. [2025-11-04T19:54:45.236881] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-11-04T19:54:45.767060] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-10. [2025-11-04T19:54:46.004022] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-11-04T19:54:46.567073] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-02. [2025-11-04T19:54:47.117147] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-11-04T19:54:47.367039] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-09. [2025-11-04T19:54:47.587908] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-11-04T19:54:48.019349] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-11-04T19:54:48.167020] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-04. [2025-11-04T19:54:48.740699] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-11-04T19:54:48.967028] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-10. [2025-11-04T19:54:49.733442] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-11-04T19:54:49.767031] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-11. [2025-11-04T19:54:50.567027] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-11. [2025-11-04T19:54:50.584488] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-11 (0.82 seconds). [2025-11-04T19:54:50.585968] [INFO ] [REMOTE ] LCII: Full fetch complete (39455 records). [2025-11-04T19:54:50.586061] [DEBUG] [VALIDATOR ] LCII: Validating primary data. [2025-11-04T19:54:50.956540] [DEBUG] [VALIDATOR ] LCII: All expected trading days are present. [2025-11-04T19:54:50.956614] [DEBUG] [VALIDATOR ] LCII: Primary data validation passed. [2025-11-04T19:54:50.957282] [INFO ] [MAIN ] LCII: Fetched primary data is valid. [2025-11-04T19:54:50.957343] [DEBUG] [LOCAL ] LCII: Saving primary data. [2025-11-04T19:54:50.958147] [DEBUG] [LOCAL ] LCII: Trimmed 39455 records to 26101 (keeping 200 days of data) [2025-11-04T19:54:51.115925] [DEBUG] [LOCAL ] LCII: Primary data saved (26101 records). [2025-11-04T19:54:51.116068] [INFO ] [MAIN ] LCII: Full history saved. [2025-11-04T19:54:51.116149] [INFO ] [MAIN ] Update Progress: 1243/2367 symbols completed. LCII took 36.7s. Estimated remaining: 2h 38m 4s. Rate: 7.1 symbols/min [2025-11-04T19:54:51.281708] [INFO ] [MAIN ] LEG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:54:51.324058] [DEBUG] [LOCAL ] LEG: Loaded 49842 existing records from primary storage. [2025-11-04T19:54:51.324350] [INFO ] [MAIN ] LEG: Loaded 49842 existing records. [2025-11-04T19:54:51.377845] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-05. [2025-11-04T19:54:51.505821] [WARN ] [MAIN ] LEG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:54:51.506031] [INFO ] [MAIN ] LEG: Wiping existing primary data before full refresh. [2025-11-04T19:54:51.506078] [INFO ] [LOCAL ] LEG: Starting deletion of primary data. [2025-11-04T19:54:51.506269] [DEBUG] [LOCAL ] LEG: Deleted primary data file. [2025-11-04T19:54:51.506323] [INFO ] [MAIN ] LEG: Starting full primary refresh. [2025-11-04T19:54:51.765694] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-11 (1.20 seconds). [2025-11-04T19:54:51.800018] [INFO ] [REMOTE ] LC: Full fetch complete (70738 records). [2025-11-04T19:54:51.820946] [DEBUG] [VALIDATOR ] LC: Validating primary data. [2025-11-04T19:54:51.891075] [INFO ] [REMOTE ] LEG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:54:52.111158] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T19:54:52.176826] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-06. [2025-11-04T19:54:52.374441] [DEBUG] [VALIDATOR ] LC: All expected trading days are present. [2025-11-04T19:54:52.374544] [DEBUG] [VALIDATOR ] LC: Primary data validation passed. [2025-11-04T19:54:52.375579] [INFO ] [MAIN ] LC: Fetched primary data is valid. [2025-11-04T19:54:52.375662] [DEBUG] [LOCAL ] LC: Saving primary data. [2025-11-04T19:54:52.377008] [DEBUG] [LOCAL ] LC: Trimmed 70738 records to 48309 (keeping 200 days of data) [2025-11-04T19:54:52.654757] [DEBUG] [LOCAL ] LC: Primary data saved (48309 records). [2025-11-04T19:54:52.654885] [INFO ] [MAIN ] LC: Full history saved. [2025-11-04T19:54:52.654964] [INFO ] [MAIN ] Update Progress: 1244/2367 symbols completed. LC took 38.3s. Estimated remaining: 2h 37m 49s. Rate: 7.1 symbols/min [2025-11-04T19:54:52.772483] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-11-04T19:54:52.831337] [INFO ] [MAIN ] LEN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:54:52.880587] [DEBUG] [LOCAL ] LEN: Loaded 53204 existing records from primary storage. [2025-11-04T19:54:52.880925] [INFO ] [MAIN ] LEN: Loaded 53204 existing records. [2025-11-04T19:54:52.975297] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-07. [2025-11-04T19:54:53.056566] [WARN ] [MAIN ] LEN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:54:53.056846] [INFO ] [MAIN ] LEN: Wiping existing primary data before full refresh. [2025-11-04T19:54:53.056899] [INFO ] [LOCAL ] LEN: Starting deletion of primary data. [2025-11-04T19:54:53.057208] [DEBUG] [LOCAL ] LEN: Deleted primary data file. [2025-11-04T19:54:53.057262] [INFO ] [MAIN ] LEN: Starting full primary refresh. [2025-11-04T19:54:53.399624] [INFO ] [REMOTE ] LEN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:54:53.620563] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-11-04T19:54:53.767049] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-08. [2025-11-04T19:54:54.322299] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-11-04T19:54:54.567085] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-03. [2025-11-04T19:54:55.367062] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-01. [2025-11-04T19:54:55.867127] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-11-04T19:54:56.167050] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-04. [2025-11-04T19:54:56.522246] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T19:54:56.967025] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-01. [2025-11-04T19:54:57.285913] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-11-04T19:54:57.767045] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-02. [2025-11-04T19:54:58.322429] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-11-04T19:54:58.567102] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-05. [2025-11-04T19:54:58.924925] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-11-04T19:54:59.367058] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-02. [2025-11-04T19:54:59.498326] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-11-04T19:55:00.167105] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-09. [2025-11-04T19:55:00.790377] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-11-04T19:55:00.837733] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-11-04T19:55:00.967064] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-10. [2025-11-04T19:55:01.511840] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-11-04T19:55:01.767134] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-06. [2025-11-04T19:55:02.567066] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-11. [2025-11-04T19:55:02.932767] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T19:55:03.196355] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-11 (0.63 seconds). [2025-11-04T19:55:03.197353] [INFO ] [REMOTE ] LDP: Full fetch complete (18372 records). [2025-11-04T19:55:03.197426] [DEBUG] [VALIDATOR ] LDP: Validating primary data. [2025-11-04T19:55:03.372106] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-07. [2025-11-04T19:55:03.569813] [DEBUG] [VALIDATOR ] LDP: All expected trading days are present. [2025-11-04T19:55:03.569888] [DEBUG] [VALIDATOR ] LDP: Primary data validation passed. [2025-11-04T19:55:03.570265] [INFO ] [MAIN ] LDP: Fetched primary data is valid. [2025-11-04T19:55:03.570304] [DEBUG] [LOCAL ] LDP: Saving primary data. [2025-11-04T19:55:03.570865] [DEBUG] [LOCAL ] LDP: Trimmed 18372 records to 12065 (keeping 200 days of data) [2025-11-04T19:55:03.643662] [DEBUG] [LOCAL ] LDP: Primary data saved (12065 records). [2025-11-04T19:55:03.643806] [INFO ] [MAIN ] LDP: Full history saved. [2025-11-04T19:55:03.643887] [INFO ] [MAIN ] Update Progress: 1245/2367 symbols completed. LDP took 25.6s. Estimated remaining: 2h 37m 43s. Rate: 7.1 symbols/min [2025-11-04T19:55:03.820328] [INFO ] [MAIN ] LEO: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:55:03.836762] [DEBUG] [LOCAL ] LEO: Loaded 16168 existing records from primary storage. [2025-11-04T19:55:03.837014] [INFO ] [MAIN ] LEO: Loaded 16168 existing records. [2025-11-04T19:55:04.027854] [WARN ] [MAIN ] LEO: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:55:04.028066] [INFO ] [MAIN ] LEO: Wiping existing primary data before full refresh. [2025-11-04T19:55:04.028113] [INFO ] [LOCAL ] LEO: Starting deletion of primary data. [2025-11-04T19:55:04.028318] [DEBUG] [LOCAL ] LEO: Deleted primary data file. [2025-11-04T19:55:04.028376] [INFO ] [MAIN ] LEO: Starting full primary refresh. [2025-11-04T19:55:04.176501] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-03. [2025-11-04T19:55:04.415638] [INFO ] [REMOTE ] LEO: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:55:04.687005] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T19:55:04.967046] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-08. [2025-11-04T19:55:05.662780] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-11-04T19:55:05.767009] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-04. [2025-11-04T19:55:06.003899] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-11-04T19:55:06.567065] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-03. [2025-11-04T19:55:07.112937] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-11-04T19:55:07.367038] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-05. [2025-11-04T19:55:08.024856] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T19:55:08.167003] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-04. [2025-11-04T19:55:08.649800] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-11-04T19:55:08.967051] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-09. [2025-11-04T19:55:09.482299] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-11-04T19:55:09.767061] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-01. [2025-11-04T19:55:10.002335] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-11-04T19:55:10.461413] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-11-04T19:55:10.567053] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-10. [2025-11-04T19:55:11.367060] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-06. [2025-11-04T19:55:11.736439] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-11-04T19:55:12.167025] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-05. [2025-11-04T19:55:12.521309] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-11-04T19:55:12.967083] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-02. [2025-11-04T19:55:13.510681] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-11-04T19:55:13.664961] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-11-04T19:55:13.767044] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-03. [2025-11-04T19:55:14.539234] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-11-04T19:55:14.567007] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-04. [2025-11-04T19:55:15.367031] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-11. [2025-11-04T19:55:15.593093] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-11-04T19:55:16.167065] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-06. [2025-11-04T19:55:16.421884] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-11 (1.05 seconds). [2025-11-04T19:55:16.423581] [INFO ] [REMOTE ] LEA: Full fetch complete (60364 records). [2025-11-04T19:55:16.423700] [DEBUG] [VALIDATOR ] LEA: Validating primary data. [2025-11-04T19:55:16.819313] [DEBUG] [VALIDATOR ] LEA: All expected trading days are present. [2025-11-04T19:55:16.819393] [DEBUG] [VALIDATOR ] LEA: Primary data validation passed. [2025-11-04T19:55:16.820213] [INFO ] [MAIN ] LEA: Fetched primary data is valid. [2025-11-04T19:55:16.820276] [DEBUG] [LOCAL ] LEA: Saving primary data. [2025-11-04T19:55:16.821461] [DEBUG] [LOCAL ] LEA: Trimmed 60364 records to 40868 (keeping 200 days of data) [2025-11-04T19:55:16.992787] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-07. [2025-11-04T19:55:17.070020] [DEBUG] [LOCAL ] LEA: Primary data saved (40868 records). [2025-11-04T19:55:17.070151] [INFO ] [MAIN ] LEA: Full history saved. [2025-11-04T19:55:17.070225] [INFO ] [MAIN ] Update Progress: 1246/2367 symbols completed. LEA took 33.0s. Estimated remaining: 2h 37m 39s. Rate: 7.1 symbols/min [2025-11-04T19:55:17.248040] [INFO ] [MAIN ] LEVI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:55:17.295826] [DEBUG] [LOCAL ] LEVI: Loaded 51810 existing records from primary storage. [2025-11-04T19:55:17.296136] [INFO ] [MAIN ] LEVI: Loaded 51810 existing records. [2025-11-04T19:55:17.478878] [WARN ] [MAIN ] LEVI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:55:17.479100] [INFO ] [MAIN ] LEVI: Wiping existing primary data before full refresh. [2025-11-04T19:55:17.479150] [INFO ] [LOCAL ] LEVI: Starting deletion of primary data. [2025-11-04T19:55:17.479344] [DEBUG] [LOCAL ] LEVI: Deleted primary data file. [2025-11-04T19:55:17.479383] [INFO ] [MAIN ] LEVI: Starting full primary refresh. [2025-11-04T19:55:17.768637] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-05. [2025-11-04T19:55:17.848781] [INFO ] [REMOTE ] LEVI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:55:17.882489] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-11-04T19:55:18.304918] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-11-04T19:55:18.469110] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-11-04T19:55:18.567063] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-08. [2025-11-04T19:55:19.367057] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-07. [2025-11-04T19:55:19.777203] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-11-04T19:55:20.167106] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-09. [2025-11-04T19:55:20.967075] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-01. [2025-11-04T19:55:21.032102] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-11-04T19:55:21.361802] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-11-04T19:55:21.767081] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-10. [2025-11-04T19:55:22.134742] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T19:55:22.567093] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-02. [2025-11-04T19:55:23.196237] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-11-04T19:55:23.367016] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-11. [2025-11-04T19:55:23.787204] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-11-04T19:55:24.167072] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-03. [2025-11-04T19:55:24.645508] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-11 (1.28 seconds). [2025-11-04T19:55:24.647449] [INFO ] [REMOTE ] LEG: Full fetch complete (72781 records). [2025-11-04T19:55:24.647548] [DEBUG] [VALIDATOR ] LEG: Validating primary data. [2025-11-04T19:55:24.978870] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-06. [2025-11-04T19:55:25.037245] [DEBUG] [VALIDATOR ] LEG: All expected trading days are present. [2025-11-04T19:55:25.037338] [DEBUG] [VALIDATOR ] LEG: Primary data validation passed. [2025-11-04T19:55:25.038363] [INFO ] [MAIN ] LEG: Fetched primary data is valid. [2025-11-04T19:55:25.038430] [DEBUG] [LOCAL ] LEG: Saving primary data. [2025-11-04T19:55:25.039961] [DEBUG] [LOCAL ] LEG: Trimmed 72781 records to 49782 (keeping 200 days of data) [2025-11-04T19:55:25.339698] [DEBUG] [LOCAL ] LEG: Primary data saved (49782 records). [2025-11-04T19:55:25.339960] [INFO ] [MAIN ] LEG: Full history saved. [2025-11-04T19:55:25.340184] [INFO ] [MAIN ] Update Progress: 1247/2367 symbols completed. LEG took 34.2s. Estimated remaining: 2h 37m 30s. Rate: 7.1 symbols/min [2025-11-04T19:55:25.522939] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-11-04T19:55:25.551149] [INFO ] [MAIN ] LFT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:55:25.569739] [DEBUG] [LOCAL ] LFT: Loaded 18062 existing records from primary storage. [2025-11-04T19:55:25.569977] [INFO ] [MAIN ] LFT: Loaded 18062 existing records. [2025-11-04T19:55:25.748721] [WARN ] [MAIN ] LFT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:55:25.749070] [INFO ] [MAIN ] LFT: Wiping existing primary data before full refresh. [2025-11-04T19:55:25.749212] [INFO ] [LOCAL ] LFT: Starting deletion of primary data. [2025-11-04T19:55:25.749530] [DEBUG] [LOCAL ] LFT: Deleted primary data file. [2025-11-04T19:55:25.749587] [INFO ] [MAIN ] LFT: Starting full primary refresh. [2025-11-04T19:55:25.772145] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-08. [2025-11-04T19:55:25.868827] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-11-04T19:55:26.121175] [INFO ] [REMOTE ] LFT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:55:26.567080] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-07. [2025-11-04T19:55:27.254453] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-11-04T19:55:27.367015] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-08. [2025-11-04T19:55:27.526568] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-11-04T19:55:27.822623] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-11-04T19:55:28.167038] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-01. [2025-11-04T19:55:28.893482] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-11-04T19:55:28.967008] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-02. [2025-11-04T19:55:29.539795] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-11-04T19:55:29.767065] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-03. [2025-11-04T19:55:30.367387] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-11-04T19:55:30.567050] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-04. [2025-11-04T19:55:31.286123] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-11-04T19:55:31.367051] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-05. [2025-11-04T19:55:32.093689] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-11-04T19:55:32.167109] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-09. [2025-11-04T19:55:32.685431] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-11-04T19:55:32.967059] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-06. [2025-11-04T19:55:33.733225] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-11-04T19:55:33.767040] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-09. [2025-11-04T19:55:34.567074] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-04. [2025-11-04T19:55:35.367065] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-07. [2025-11-04T19:55:35.546528] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-11-04T19:55:36.051299] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-11-04T19:55:36.079289] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-11-04T19:55:36.167031] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-05. [2025-11-04T19:55:36.967052] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-08. [2025-11-04T19:55:37.598769] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-11-04T19:55:37.706217] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-11-04T19:55:37.767018] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-09. [2025-11-04T19:55:38.417454] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-11-04T19:55:38.567064] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-10. [2025-11-04T19:55:39.367011] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-10. [2025-11-04T19:55:40.010668] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-11-04T19:55:40.167038] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-10. [2025-11-04T19:55:40.491016] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-11-04T19:55:40.740436] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-11-04T19:55:40.967089] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-06. [2025-11-04T19:55:41.767097] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-11. [2025-11-04T19:55:42.206936] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-11-04T19:55:42.305906] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T19:55:42.307302] [INFO ] [REMOTE ] LFT: Full fetch complete (23874 records). [2025-11-04T19:55:42.307376] [DEBUG] [VALIDATOR ] LFT: Validating primary data. [2025-11-04T19:55:42.572780] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-07. [2025-11-04T19:55:42.673098] [DEBUG] [VALIDATOR ] LFT: All expected trading days are present. [2025-11-04T19:55:42.673180] [DEBUG] [VALIDATOR ] LFT: Primary data validation passed. [2025-11-04T19:55:42.673753] [INFO ] [MAIN ] LFT: Fetched primary data is valid. [2025-11-04T19:55:42.673814] [DEBUG] [LOCAL ] LFT: Saving primary data. [2025-11-04T19:55:42.674623] [DEBUG] [LOCAL ] LFT: Trimmed 23874 records to 18032 (keeping 200 days of data) [2025-11-04T19:55:42.775596] [DEBUG] [LOCAL ] LFT: Primary data saved (18032 records). [2025-11-04T19:55:42.775778] [INFO ] [MAIN ] LFT: Full history saved. [2025-11-04T19:55:42.775868] [INFO ] [MAIN ] Update Progress: 1248/2367 symbols completed. LFT took 17.4s. Estimated remaining: 2h 37m 30s. Rate: 7.1 symbols/min [2025-11-04T19:55:42.944281] [INFO ] [MAIN ] LGI: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:55:42.955925] [DEBUG] [LOCAL ] LGI: Loaded 9570 existing records from primary storage. [2025-11-04T19:55:42.956139] [INFO ] [MAIN ] LGI: Loaded 9570 existing records. [2025-11-04T19:55:43.131287] [WARN ] [MAIN ] LGI: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:55:43.131495] [INFO ] [MAIN ] LGI: Wiping existing primary data before full refresh. [2025-11-04T19:55:43.131542] [INFO ] [LOCAL ] LGI: Starting deletion of primary data. [2025-11-04T19:55:43.131748] [DEBUG] [LOCAL ] LGI: Deleted primary data file. [2025-11-04T19:55:43.131814] [INFO ] [MAIN ] LGI: Starting full primary refresh. [2025-11-04T19:55:43.372186] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-11. [2025-11-04T19:55:43.489897] [INFO ] [REMOTE ] LGI: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:55:44.028371] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T19:55:44.053231] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-11 (0.68 seconds). [2025-11-04T19:55:44.054471] [INFO ] [REMOTE ] LEO: Full fetch complete (26617 records). [2025-11-04T19:55:44.054543] [DEBUG] [VALIDATOR ] LEO: Validating primary data. [2025-11-04T19:55:44.172193] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-01. [2025-11-04T19:55:44.417327] [DEBUG] [VALIDATOR ] LEO: All expected trading days are present. [2025-11-04T19:55:44.417404] [DEBUG] [VALIDATOR ] LEO: Primary data validation passed. [2025-11-04T19:55:44.417996] [INFO ] [MAIN ] LEO: Fetched primary data is valid. [2025-11-04T19:55:44.418056] [DEBUG] [LOCAL ] LEO: Saving primary data. [2025-11-04T19:55:44.418670] [DEBUG] [LOCAL ] LEO: Trimmed 26617 records to 16172 (keeping 200 days of data) [2025-11-04T19:55:44.510984] [DEBUG] [LOCAL ] LEO: Primary data saved (16172 records). [2025-11-04T19:55:44.511155] [INFO ] [MAIN ] LEO: Full history saved. [2025-11-04T19:55:44.511247] [INFO ] [MAIN ] Update Progress: 1249/2367 symbols completed. LEO took 40.9s. Estimated remaining: 2h 37m 16s. Rate: 7.1 symbols/min [2025-11-04T19:55:44.680335] [INFO ] [MAIN ] LH: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:55:44.714946] [DEBUG] [LOCAL ] LH: Loaded 36901 existing records from primary storage. [2025-11-04T19:55:44.715241] [INFO ] [MAIN ] LH: Loaded 36901 existing records. [2025-11-04T19:55:44.905554] [WARN ] [MAIN ] LH: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:55:44.907000] [INFO ] [MAIN ] LH: Wiping existing primary data before full refresh. [2025-11-04T19:55:44.906842] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-11-04T19:55:44.907188] [INFO ] [LOCAL ] LH: Starting deletion of primary data. [2025-11-04T19:55:44.910055] [DEBUG] [LOCAL ] LH: Deleted primary data file. [2025-11-04T19:55:44.910122] [INFO ] [MAIN ] LH: Starting full primary refresh. [2025-11-04T19:55:44.972322] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-11. [2025-11-04T19:55:45.263674] [INFO ] [REMOTE ] LH: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:55:45.364639] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-11 (0.39 seconds). [2025-11-04T19:55:45.367373] [INFO ] [REMOTE ] LEN: Full fetch complete (77218 records). [2025-11-04T19:55:45.367471] [DEBUG] [VALIDATOR ] LEN: Validating primary data. [2025-11-04T19:55:45.748240] [DEBUG] [VALIDATOR ] LEN: All expected trading days are present. [2025-11-04T19:55:45.748328] [DEBUG] [VALIDATOR ] LEN: Primary data validation passed. [2025-11-04T19:55:45.749696] [INFO ] [MAIN ] LEN: Fetched primary data is valid. [2025-11-04T19:55:45.749760] [DEBUG] [LOCAL ] LEN: Saving primary data. [2025-11-04T19:55:45.751237] [DEBUG] [LOCAL ] LEN: Trimmed 77218 records to 53203 (keeping 200 days of data) [2025-11-04T19:55:45.805358] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-08. [2025-11-04T19:55:46.089164] [DEBUG] [LOCAL ] LEN: Primary data saved (53203 records). [2025-11-04T19:55:46.089341] [INFO ] [MAIN ] LEN: Full history saved. [2025-11-04T19:55:46.089448] [INFO ] [MAIN ] Update Progress: 1250/2367 symbols completed. LEN took 53.4s. Estimated remaining: 2h 37m 1s. Rate: 7.1 symbols/min [2025-11-04T19:55:46.270025] [INFO ] [MAIN ] LHX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:55:46.314347] [DEBUG] [LOCAL ] LHX: Loaded 48215 existing records from primary storage. [2025-11-04T19:55:46.314724] [INFO ] [MAIN ] LHX: Loaded 48215 existing records. [2025-11-04T19:55:46.508014] [WARN ] [MAIN ] LHX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:55:46.508234] [INFO ] [MAIN ] LHX: Wiping existing primary data before full refresh. [2025-11-04T19:55:46.508284] [INFO ] [LOCAL ] LHX: Starting deletion of primary data. [2025-11-04T19:55:46.508470] [DEBUG] [LOCAL ] LHX: Deleted primary data file. [2025-11-04T19:55:46.508548] [INFO ] [MAIN ] LHX: Starting full primary refresh. [2025-11-04T19:55:46.572220] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-01. [2025-11-04T19:55:46.887072] [INFO ] [REMOTE ] LHX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:55:47.009293] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T19:55:47.367079] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-02. [2025-11-04T19:55:47.694750] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-11-04T19:55:47.747862] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-11-04T19:55:48.167035] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-03. [2025-11-04T19:55:48.557733] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-11-04T19:55:48.967102] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-04. [2025-11-04T19:55:49.589241] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-11-04T19:55:49.767050] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-09. [2025-11-04T19:55:50.567102] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-05. [2025-11-04T19:55:51.123524] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-11-04T19:55:51.156576] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-11-04T19:55:51.367052] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-06. [2025-11-04T19:55:51.848630] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-11-04T19:55:52.167050] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-07. [2025-11-04T19:55:52.534142] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-11-04T19:55:52.967042] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-08. [2025-11-04T19:55:53.664910] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-11-04T19:55:53.767028] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-09. [2025-11-04T19:55:54.242036] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-11-04T19:55:54.567076] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-02. [2025-11-04T19:55:55.367118] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-10. [2025-11-04T19:55:55.708026] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-11-04T19:55:55.872576] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-11-04T19:55:56.167088] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-10. [2025-11-04T19:55:56.967092] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-01. [2025-11-04T19:55:57.767012] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-11. [2025-11-04T19:55:57.811205] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-11-04T19:55:58.213991] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-11 (0.45 seconds). [2025-11-04T19:55:58.214790] [INFO ] [REMOTE ] LGI: Full fetch complete (14267 records). [2025-11-04T19:55:58.214858] [DEBUG] [VALIDATOR ] LGI: Validating primary data. [2025-11-04T19:55:58.570182] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-03. [2025-11-04T19:55:58.617369] [DEBUG] [VALIDATOR ] LGI: All expected trading days are present. [2025-11-04T19:55:58.618429] [DEBUG] [VALIDATOR ] LGI: Primary data validation passed. [2025-11-04T19:55:58.618874] [INFO ] [MAIN ] LGI: Fetched primary data is valid. [2025-11-04T19:55:58.619033] [DEBUG] [LOCAL ] LGI: Saving primary data. [2025-11-04T19:55:58.620138] [DEBUG] [LOCAL ] LGI: Trimmed 14267 records to 9465 (keeping 200 days of data) [2025-11-04T19:55:58.680416] [DEBUG] [LOCAL ] LGI: Primary data saved (9465 records). [2025-11-04T19:55:58.680723] [INFO ] [MAIN ] LGI: Full history saved. [2025-11-04T19:55:58.680891] [INFO ] [MAIN ] Update Progress: 1251/2367 symbols completed. LGI took 15.9s. Estimated remaining: 2h 36m 56s. Rate: 7.1 symbols/min [2025-11-04T19:55:58.810298] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-11-04T19:55:58.886917] [INFO ] [MAIN ] LII: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:55:58.913112] [DEBUG] [LOCAL ] LII: Loaded 27774 existing records from primary storage. [2025-11-04T19:55:58.913356] [INFO ] [MAIN ] LII: Loaded 27774 existing records. [2025-11-04T19:55:59.089279] [WARN ] [MAIN ] LII: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:55:59.089502] [INFO ] [MAIN ] LII: Wiping existing primary data before full refresh. [2025-11-04T19:55:59.089549] [INFO ] [LOCAL ] LII: Starting deletion of primary data. [2025-11-04T19:55:59.089807] [DEBUG] [LOCAL ] LII: Deleted primary data file. [2025-11-04T19:55:59.089861] [INFO ] [MAIN ] LII: Starting full primary refresh. [2025-11-04T19:55:59.369159] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-11. [2025-11-04T19:55:59.448845] [INFO ] [REMOTE ] LII: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:55:59.968928] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T19:56:00.167070] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-01. [2025-11-04T19:56:00.872849] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-11 (1.50 seconds). [2025-11-04T19:56:00.874826] [INFO ] [REMOTE ] LEVI: Full fetch complete (75099 records). [2025-11-04T19:56:00.874917] [DEBUG] [VALIDATOR ] LEVI: Validating primary data. [2025-11-04T19:56:00.973630] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-04. [2025-11-04T19:56:01.285085] [DEBUG] [VALIDATOR ] LEVI: All expected trading days are present. [2025-11-04T19:56:01.285193] [DEBUG] [VALIDATOR ] LEVI: Primary data validation passed. [2025-11-04T19:56:01.286318] [INFO ] [MAIN ] LEVI: Fetched primary data is valid. [2025-11-04T19:56:01.286472] [DEBUG] [LOCAL ] LEVI: Saving primary data. [2025-11-04T19:56:01.303955] [DEBUG] [LOCAL ] LEVI: Trimmed 75099 records to 51808 (keeping 200 days of data) [2025-11-04T19:56:01.304470] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-11-04T19:56:01.627538] [DEBUG] [LOCAL ] LEVI: Primary data saved (51808 records). [2025-11-04T19:56:01.627694] [INFO ] [MAIN ] LEVI: Full history saved. [2025-11-04T19:56:01.627798] [INFO ] [MAIN ] Update Progress: 1252/2367 symbols completed. LEVI took 44.6s. Estimated remaining: 2h 36m 43s. Rate: 7.1 symbols/min [2025-11-04T19:56:01.768984] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-02. [2025-11-04T19:56:01.800598] [INFO ] [MAIN ] LION: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:56:01.842328] [DEBUG] [LOCAL ] LION: Loaded 48019 existing records from primary storage. [2025-11-04T19:56:01.842622] [INFO ] [MAIN ] LION: Loaded 48019 existing records. [2025-11-04T19:56:02.022743] [WARN ] [MAIN ] LION: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:56:02.022930] [INFO ] [MAIN ] LION: Wiping existing primary data before full refresh. [2025-11-04T19:56:02.022976] [INFO ] [LOCAL ] LION: Starting deletion of primary data. [2025-11-04T19:56:02.023165] [DEBUG] [LOCAL ] LION: Deleted primary data file. [2025-11-04T19:56:02.023249] [INFO ] [MAIN ] LION: Starting full primary refresh. [2025-11-04T19:56:02.384936] [INFO ] [REMOTE ] LION: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:56:02.409360] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-11-04T19:56:02.567084] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-05. [2025-11-04T19:56:02.810536] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-11-04T19:56:03.367102] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-01. [2025-11-04T19:56:03.715467] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-11-04T19:56:04.101814] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-11-04T19:56:04.167011] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-02. [2025-11-04T19:56:04.967070] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-06. [2025-11-04T19:56:05.547489] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-11-04T19:56:05.767045] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-02. [2025-11-04T19:56:06.037571] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-11-04T19:56:06.567012] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-07. [2025-11-04T19:56:06.660782] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-11-04T19:56:07.367039] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-03. [2025-11-04T19:56:07.702327] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-11-04T19:56:08.166999] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-08. [2025-11-04T19:56:08.343756] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-11-04T19:56:08.967025] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-04. [2025-11-04T19:56:09.163618] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-11-04T19:56:09.767046] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-03. [2025-11-04T19:56:10.011770] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-11-04T19:56:10.567067] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-09. [2025-11-04T19:56:10.979504] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-11-04T19:56:11.367040] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-04. [2025-11-04T19:56:11.488668] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T19:56:12.167068] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-10. [2025-11-04T19:56:12.676614] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-11-04T19:56:12.967068] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-03. [2025-11-04T19:56:13.184804] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-11-04T19:56:13.678888] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T19:56:13.767043] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-04. [2025-11-04T19:56:14.567036] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-05. [2025-11-04T19:56:14.753474] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-11-04T19:56:15.367079] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-05. [2025-11-04T19:56:15.943543] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-11-04T19:56:16.167045] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-06. [2025-11-04T19:56:16.522104] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-11-04T19:56:16.967139] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-11. [2025-11-04T19:56:17.503242] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T19:56:17.767093] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-07. [2025-11-04T19:56:17.970596] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-11 (1.00 seconds). [2025-11-04T19:56:17.972455] [INFO ] [REMOTE ] LH: Full fetch complete (55220 records). [2025-11-04T19:56:17.972535] [DEBUG] [VALIDATOR ] LH: Validating primary data. [2025-11-04T19:56:18.372721] [DEBUG] [VALIDATOR ] LH: All expected trading days are present. [2025-11-04T19:56:18.372832] [DEBUG] [VALIDATOR ] LH: Primary data validation passed. [2025-11-04T19:56:18.373728] [INFO ] [MAIN ] LH: Fetched primary data is valid. [2025-11-04T19:56:18.373794] [DEBUG] [LOCAL ] LH: Saving primary data. [2025-11-04T19:56:18.374972] [DEBUG] [LOCAL ] LH: Trimmed 55220 records to 36959 (keeping 200 days of data) [2025-11-04T19:56:18.587603] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-06. [2025-11-04T19:56:18.619877] [DEBUG] [LOCAL ] LH: Primary data saved (36959 records). [2025-11-04T19:56:18.620410] [INFO ] [MAIN ] LH: Full history saved. [2025-11-04T19:56:18.620704] [INFO ] [MAIN ] Update Progress: 1253/2367 symbols completed. LH took 34.1s. Estimated remaining: 2h 36m 42s. Rate: 7.1 symbols/min [2025-11-04T19:56:18.784571] [INFO ] [MAIN ] LITB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:56:18.790347] [DEBUG] [LOCAL ] LITB: Loaded 3602 existing records from primary storage. [2025-11-04T19:56:18.790511] [INFO ] [MAIN ] LITB: Loaded 3602 existing records. [2025-11-04T19:56:18.960861] [WARN ] [MAIN ] LITB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:56:18.961085] [INFO ] [MAIN ] LITB: Wiping existing primary data before full refresh. [2025-11-04T19:56:18.961133] [INFO ] [LOCAL ] LITB: Starting deletion of primary data. [2025-11-04T19:56:18.961280] [DEBUG] [LOCAL ] LITB: Deleted primary data file. [2025-11-04T19:56:18.961319] [INFO ] [MAIN ] LITB: Starting full primary refresh. [2025-11-04T19:56:19.330413] [INFO ] [REMOTE ] LITB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:56:19.364455] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-11-04T19:56:19.374586] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-05. [2025-11-04T19:56:19.759359] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T19:56:20.167015] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-08. [2025-11-04T19:56:20.247986] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T19:56:20.967071] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-07. [2025-11-04T19:56:21.528618] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-11-04T19:56:21.767032] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-09. [2025-11-04T19:56:22.221964] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-11-04T19:56:22.567036] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-01. [2025-11-04T19:56:22.888600] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-01 (0.32 seconds). [2025-11-04T19:56:23.067363] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T19:56:23.367085] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-02. [2025-11-04T19:56:23.684799] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-11-04T19:56:24.167130] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-03. [2025-11-04T19:56:24.594842] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-11-04T19:56:24.967106] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-06. [2025-11-04T19:56:25.751132] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-11-04T19:56:25.767024] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-07. [2025-11-04T19:56:26.567016] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-04. [2025-11-04T19:56:26.718261] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-11-04T19:56:27.053147] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-11-04T19:56:27.367041] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-08. [2025-11-04T19:56:28.167042] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-10. [2025-11-04T19:56:28.632945] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-11-04T19:56:28.967055] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-08. [2025-11-04T19:56:29.347357] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-11-04T19:56:29.777107] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-09. [2025-11-04T19:56:29.778468] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-11-04T19:56:30.567105] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-11. [2025-11-04T19:56:30.898141] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-11-04T19:56:31.367043] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-05. [2025-11-04T19:56:31.679170] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-11 (1.11 seconds). [2025-11-04T19:56:31.681124] [INFO ] [REMOTE ] LHX: Full fetch complete (70744 records). [2025-11-04T19:56:31.681227] [DEBUG] [VALIDATOR ] LHX: Validating primary data. [2025-11-04T19:56:31.785559] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-11-04T19:56:32.073832] [DEBUG] [VALIDATOR ] LHX: All expected trading days are present. [2025-11-04T19:56:32.073921] [DEBUG] [VALIDATOR ] LHX: Primary data validation passed. [2025-11-04T19:56:32.075002] [INFO ] [MAIN ] LHX: Fetched primary data is valid. [2025-11-04T19:56:32.075069] [DEBUG] [LOCAL ] LHX: Saving primary data. [2025-11-04T19:56:32.076478] [DEBUG] [LOCAL ] LHX: Trimmed 70744 records to 48226 (keeping 200 days of data) [2025-11-04T19:56:32.167078] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-10. [2025-11-04T19:56:32.375330] [DEBUG] [LOCAL ] LHX: Primary data saved (48226 records). [2025-11-04T19:56:32.375456] [INFO ] [MAIN ] LHX: Full history saved. [2025-11-04T19:56:32.375531] [INFO ] [MAIN ] Update Progress: 1254/2367 symbols completed. LHX took 46.3s. Estimated remaining: 2h 36m 38s. Rate: 7.1 symbols/min [2025-11-04T19:56:32.541305] [INFO ] [MAIN ] LLY: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:56:32.589442] [DEBUG] [LOCAL ] LLY: Loaded 53167 existing records from primary storage. [2025-11-04T19:56:32.589754] [INFO ] [MAIN ] LLY: Loaded 53167 existing records. [2025-11-04T19:56:32.772149] [WARN ] [MAIN ] LLY: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:56:32.772434] [INFO ] [MAIN ] LLY: Wiping existing primary data before full refresh. [2025-11-04T19:56:32.772501] [INFO ] [LOCAL ] LLY: Starting deletion of primary data. [2025-11-04T19:56:32.772760] [DEBUG] [LOCAL ] LLY: Deleted primary data file. [2025-11-04T19:56:32.772834] [INFO ] [MAIN ] LLY: Starting full primary refresh. [2025-11-04T19:56:32.978160] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-06. [2025-11-04T19:56:33.154797] [INFO ] [REMOTE ] LLY: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:56:33.363394] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-11-04T19:56:33.408497] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-11-04T19:56:33.767039] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-11. [2025-11-04T19:56:34.567045] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-09. [2025-11-04T19:56:34.847088] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-11 (1.08 seconds). [2025-11-04T19:56:34.849293] [INFO ] [REMOTE ] LION: Full fetch complete (59460 records). [2025-11-04T19:56:34.849424] [DEBUG] [VALIDATOR ] LION: Validating primary data. [2025-11-04T19:56:35.284292] [DEBUG] [VALIDATOR ] LION: All expected trading days are present. [2025-11-04T19:56:35.284403] [DEBUG] [VALIDATOR ] LION: Primary data validation passed. [2025-11-04T19:56:35.285806] [INFO ] [MAIN ] LION: Fetched primary data is valid. [2025-11-04T19:56:35.285874] [DEBUG] [LOCAL ] LION: Saving primary data. [2025-11-04T19:56:35.287737] [DEBUG] [LOCAL ] LION: Trimmed 59460 records to 48200 (keeping 200 days of data) [2025-11-04T19:56:35.370348] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-07. [2025-11-04T19:56:35.486552] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-11-04T19:56:35.611508] [DEBUG] [LOCAL ] LION: Primary data saved (48200 records). [2025-11-04T19:56:35.611739] [INFO ] [MAIN ] LION: Full history saved. [2025-11-04T19:56:35.611863] [INFO ] [MAIN ] Update Progress: 1255/2367 symbols completed. LION took 34.0s. Estimated remaining: 2h 36m 25s. Rate: 7.1 symbols/min [2025-11-04T19:56:35.728805] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-11-04T19:56:35.802944] [INFO ] [MAIN ] LMND: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:56:35.862639] [DEBUG] [LOCAL ] LMND: Loaded 52223 existing records from primary storage. [2025-11-04T19:56:35.863190] [INFO ] [MAIN ] LMND: Loaded 52223 existing records. [2025-11-04T19:56:36.068579] [WARN ] [MAIN ] LMND: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:56:36.068974] [INFO ] [MAIN ] LMND: Wiping existing primary data before full refresh. [2025-11-04T19:56:36.069030] [INFO ] [LOCAL ] LMND: Starting deletion of primary data. [2025-11-04T19:56:36.069339] [DEBUG] [LOCAL ] LMND: Deleted primary data file. [2025-11-04T19:56:36.069399] [INFO ] [MAIN ] LMND: Starting full primary refresh. [2025-11-04T19:56:36.172203] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-08. [2025-11-04T19:56:36.463296] [INFO ] [REMOTE ] LMND: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:56:36.557427] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-11-04T19:56:36.967079] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-09. [2025-11-04T19:56:37.424425] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-11-04T19:56:37.767041] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-01. [2025-11-04T19:56:38.567053] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-10. [2025-11-04T19:56:38.964800] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-11-04T19:56:39.367011] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-11. [2025-11-04T19:56:39.561006] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-11-04T19:56:39.784527] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T19:56:39.785447] [INFO ] [REMOTE ] LITB: Full fetch complete (5855 records). [2025-11-04T19:56:39.785523] [DEBUG] [VALIDATOR ] LITB: Validating primary data. [2025-11-04T19:56:40.161194] [DEBUG] [VALIDATOR ] LITB: All expected trading days are present. [2025-11-04T19:56:40.161318] [DEBUG] [VALIDATOR ] LITB: Primary data validation passed. [2025-11-04T19:56:40.161702] [INFO ] [MAIN ] LITB: Fetched primary data is valid. [2025-11-04T19:56:40.161768] [DEBUG] [LOCAL ] LITB: Saving primary data. [2025-11-04T19:56:40.162272] [DEBUG] [LOCAL ] LITB: Trimmed 5855 records to 3482 (keeping 200 days of data) [2025-11-04T19:56:40.172468] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-01. [2025-11-04T19:56:40.184438] [DEBUG] [LOCAL ] LITB: Primary data saved (3482 records). [2025-11-04T19:56:40.184692] [INFO ] [MAIN ] LITB: Full history saved. [2025-11-04T19:56:40.184806] [INFO ] [MAIN ] Update Progress: 1256/2367 symbols completed. LITB took 21.6s. Estimated remaining: 2h 36m 14s. Rate: 7.1 symbols/min [2025-11-04T19:56:40.364189] [INFO ] [MAIN ] LMT: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:56:40.412972] [DEBUG] [LOCAL ] LMT: Loaded 49412 existing records from primary storage. [2025-11-04T19:56:40.413427] [INFO ] [MAIN ] LMT: Loaded 49412 existing records. [2025-11-04T19:56:40.597552] [WARN ] [MAIN ] LMT: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:56:40.597887] [INFO ] [MAIN ] LMT: Wiping existing primary data before full refresh. [2025-11-04T19:56:40.597942] [INFO ] [LOCAL ] LMT: Starting deletion of primary data. [2025-11-04T19:56:40.598558] [DEBUG] [LOCAL ] LMT: Deleted primary data file. [2025-11-04T19:56:40.598619] [INFO ] [MAIN ] LMT: Starting full primary refresh. [2025-11-04T19:56:40.968072] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-02. [2025-11-04T19:56:40.994488] [INFO ] [REMOTE ] LMT: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:56:41.612484] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-11-04T19:56:41.767042] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-01. [2025-11-04T19:56:42.567050] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-10. [2025-11-04T19:56:42.742729] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-11-04T19:56:43.292524] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-11-04T19:56:43.367017] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-02. [2025-11-04T19:56:43.553344] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-11-04T19:56:44.167038] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-02. [2025-11-04T19:56:44.775133] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-11-04T19:56:44.967037] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-03. [2025-11-04T19:56:45.702284] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-11-04T19:56:45.767024] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-11. [2025-11-04T19:56:46.426519] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-11-04T19:56:46.567052] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-03. [2025-11-04T19:56:46.598088] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-11 (0.83 seconds). [2025-11-04T19:56:46.599749] [INFO ] [REMOTE ] LII: Full fetch complete (42264 records). [2025-11-04T19:56:46.599825] [DEBUG] [VALIDATOR ] LII: Validating primary data. [2025-11-04T19:56:46.968052] [DEBUG] [VALIDATOR ] LII: All expected trading days are present. [2025-11-04T19:56:46.968131] [DEBUG] [VALIDATOR ] LII: Primary data validation passed. [2025-11-04T19:56:46.968815] [INFO ] [MAIN ] LII: Fetched primary data is valid. [2025-11-04T19:56:46.968875] [DEBUG] [LOCAL ] LII: Saving primary data. [2025-11-04T19:56:46.969756] [DEBUG] [LOCAL ] LII: Trimmed 42264 records to 27899 (keeping 200 days of data) [2025-11-04T19:56:47.139378] [DEBUG] [LOCAL ] LII: Primary data saved (27899 records). [2025-11-04T19:56:47.139501] [INFO ] [MAIN ] LII: Full history saved. [2025-11-04T19:56:47.139576] [INFO ] [MAIN ] Update Progress: 1257/2367 symbols completed. LII took 48.4s. Estimated remaining: 2h 36m 4s. Rate: 7.1 symbols/min [2025-11-04T19:56:47.317831] [INFO ] [MAIN ] LNC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:56:47.361752] [DEBUG] [LOCAL ] LNC: Loaded 51253 existing records from primary storage. [2025-11-04T19:56:47.362046] [INFO ] [MAIN ] LNC: Loaded 51253 existing records. [2025-11-04T19:56:47.373131] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-03. [2025-11-04T19:56:47.541955] [WARN ] [MAIN ] LNC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:56:47.542163] [INFO ] [MAIN ] LNC: Wiping existing primary data before full refresh. [2025-11-04T19:56:47.542210] [INFO ] [LOCAL ] LNC: Starting deletion of primary data. [2025-11-04T19:56:47.543177] [DEBUG] [LOCAL ] LNC: Deleted primary data file. [2025-11-04T19:56:47.543239] [INFO ] [MAIN ] LNC: Starting full primary refresh. [2025-11-04T19:56:47.910604] [INFO ] [REMOTE ] LNC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:56:48.167049] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-01. [2025-11-04T19:56:48.427777] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-11-04T19:56:48.967011] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-04. [2025-11-04T19:56:49.102726] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-11-04T19:56:49.333728] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-11-04T19:56:49.767087] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-04. [2025-11-04T19:56:50.567051] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-04. [2025-11-04T19:56:50.761139] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-11-04T19:56:51.033165] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T19:56:51.367066] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-02. [2025-11-04T19:56:52.178362] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-05. [2025-11-04T19:56:52.203610] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-11-04T19:56:52.474508] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-11-04T19:56:52.967064] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-05. [2025-11-04T19:56:53.767055] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-05. [2025-11-04T19:56:54.187230] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-05 (2.01 seconds). [2025-11-04T19:56:54.406418] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-11-04T19:56:54.567021] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-06. [2025-11-04T19:56:55.215051] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-11-04T19:56:55.367046] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-03. [2025-11-04T19:56:56.135908] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-11-04T19:56:56.167016] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-07. [2025-11-04T19:56:56.695161] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-11-04T19:56:56.967054] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-06. [2025-11-04T19:56:57.723223] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-11-04T19:56:57.767031] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-08. [2025-11-04T19:56:58.567071] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-04. [2025-11-04T19:56:58.817976] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-11-04T19:56:59.307756] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-11-04T19:56:59.367062] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-07. [2025-11-04T19:56:59.896144] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-11-04T19:57:00.167034] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-06. [2025-11-04T19:57:00.967032] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-05. [2025-11-04T19:57:01.618548] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-11-04T19:57:01.767054] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-09. [2025-11-04T19:57:01.928104] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-11-04T19:57:02.377688] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-11-04T19:57:02.567085] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-08. [2025-11-04T19:57:03.339815] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-11-04T19:57:03.367008] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-06. [2025-11-04T19:57:04.167030] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-07. [2025-11-04T19:57:04.585531] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-11-04T19:57:04.601669] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-11-04T19:57:04.967042] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-10. [2025-11-04T19:57:05.767022] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-07. [2025-11-04T19:57:05.949588] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-11-04T19:57:06.567034] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-08. [2025-11-04T19:57:06.746783] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-11-04T19:57:07.065377] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-11-04T19:57:07.367068] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-09. [2025-11-04T19:57:08.111893] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-11-04T19:57:08.167015] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-09. [2025-11-04T19:57:08.967061] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-08. [2025-11-04T19:57:09.288899] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-11-04T19:57:09.767055] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-10. [2025-11-04T19:57:09.982434] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-11-04T19:57:10.242073] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-11-04T19:57:10.567054] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-11. [2025-11-04T19:57:11.367085] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-09. [2025-11-04T19:57:11.843852] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-11-04T19:57:12.068990] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-11 (1.50 seconds). [2025-11-04T19:57:12.071133] [INFO ] [REMOTE ] LMND: Full fetch complete (75508 records). [2025-11-04T19:57:12.071288] [DEBUG] [VALIDATOR ] LMND: Validating primary data. [2025-11-04T19:57:12.173803] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-10. [2025-11-04T19:57:12.535928] [DEBUG] [VALIDATOR ] LMND: All expected trading days are present. [2025-11-04T19:57:12.536070] [DEBUG] [VALIDATOR ] LMND: Primary data validation passed. [2025-11-04T19:57:12.537161] [INFO ] [MAIN ] LMND: Fetched primary data is valid. [2025-11-04T19:57:12.537227] [DEBUG] [LOCAL ] LMND: Saving primary data. [2025-11-04T19:57:12.538714] [DEBUG] [LOCAL ] LMND: Trimmed 75508 records to 52269 (keeping 200 days of data) [2025-11-04T19:57:12.863486] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-11-04T19:57:12.884840] [DEBUG] [LOCAL ] LMND: Primary data saved (52269 records). [2025-11-04T19:57:12.884939] [INFO ] [MAIN ] LMND: Full history saved. [2025-11-04T19:57:12.885016] [INFO ] [MAIN ] Update Progress: 1258/2367 symbols completed. LMND took 37.3s. Estimated remaining: 2h 36m 11s. Rate: 7.1 symbols/min [2025-11-04T19:57:12.974724] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-11. [2025-11-04T19:57:13.055005] [INFO ] [MAIN ] LND: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:57:13.061814] [DEBUG] [LOCAL ] LND: Loaded 4854 existing records from primary storage. [2025-11-04T19:57:13.061992] [INFO ] [MAIN ] LND: Loaded 4854 existing records. [2025-11-04T19:57:13.234285] [WARN ] [MAIN ] LND: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:57:13.234488] [INFO ] [MAIN ] LND: Wiping existing primary data before full refresh. [2025-11-04T19:57:13.234535] [INFO ] [LOCAL ] LND: Starting deletion of primary data. [2025-11-04T19:57:13.234721] [DEBUG] [LOCAL ] LND: Deleted primary data file. [2025-11-04T19:57:13.234778] [INFO ] [MAIN ] LND: Starting full primary refresh. [2025-11-04T19:57:13.515587] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-11 (0.54 seconds). [2025-11-04T19:57:13.523418] [INFO ] [REMOTE ] LLY: Full fetch complete (77183 records). [2025-11-04T19:57:13.533935] [DEBUG] [VALIDATOR ] LLY: Validating primary data. [2025-11-04T19:57:13.661334] [INFO ] [REMOTE ] LND: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:57:13.772374] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-10. [2025-11-04T19:57:13.991947] [DEBUG] [VALIDATOR ] LLY: All expected trading days are present. [2025-11-04T19:57:13.992182] [DEBUG] [VALIDATOR ] LLY: Primary data validation passed. [2025-11-04T19:57:13.993238] [INFO ] [MAIN ] LLY: Fetched primary data is valid. [2025-11-04T19:57:13.993330] [DEBUG] [LOCAL ] LLY: Saving primary data. [2025-11-04T19:57:13.995100] [DEBUG] [LOCAL ] LLY: Trimmed 77183 records to 53167 (keeping 200 days of data) [2025-11-04T19:57:14.026628] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-11-04T19:57:14.367257] [DEBUG] [LOCAL ] LLY: Primary data saved (53167 records). [2025-11-04T19:57:14.367401] [INFO ] [MAIN ] LLY: Full history saved. [2025-11-04T19:57:14.367479] [INFO ] [MAIN ] Update Progress: 1259/2367 symbols completed. LLY took 42.0s. Estimated remaining: 2h 35m 56s. Rate: 7.1 symbols/min [2025-11-04T19:57:14.533267] [INFO ] [MAIN ] LNG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:57:14.576382] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-01. [2025-11-04T19:57:14.579275] [DEBUG] [LOCAL ] LNG: Loaded 50954 existing records from primary storage. [2025-11-04T19:57:14.579666] [INFO ] [MAIN ] LNG: Loaded 50954 existing records. [2025-11-04T19:57:14.757815] [WARN ] [MAIN ] LNG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:57:14.758017] [INFO ] [MAIN ] LNG: Wiping existing primary data before full refresh. [2025-11-04T19:57:14.758066] [INFO ] [LOCAL ] LNG: Starting deletion of primary data. [2025-11-04T19:57:14.758935] [DEBUG] [LOCAL ] LNG: Deleted primary data file. [2025-11-04T19:57:14.758993] [INFO ] [MAIN ] LNG: Starting full primary refresh. [2025-11-04T19:57:15.124029] [INFO ] [REMOTE ] LNG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:57:15.128929] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-11-04T19:57:15.215115] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-11-04T19:57:15.367022] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-01. [2025-11-04T19:57:16.167031] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-02. [2025-11-04T19:57:16.664480] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-11-04T19:57:16.974460] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-03. [2025-11-04T19:57:16.980363] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-11-04T19:57:17.437983] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-11-04T19:57:17.767070] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-04. [2025-11-04T19:57:18.176817] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-11-04T19:57:18.567037] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-05. [2025-11-04T19:57:19.001469] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-11-04T19:57:19.367037] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-11. [2025-11-04T19:57:20.167036] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-06. [2025-11-04T19:57:20.502368] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-11 (1.14 seconds). [2025-11-04T19:57:20.504370] [INFO ] [REMOTE ] LNC: Full fetch complete (74549 records). [2025-11-04T19:57:20.504464] [DEBUG] [VALIDATOR ] LNC: Validating primary data. [2025-11-04T19:57:20.628375] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-11-04T19:57:20.921560] [DEBUG] [VALIDATOR ] LNC: All expected trading days are present. [2025-11-04T19:57:20.921639] [DEBUG] [VALIDATOR ] LNC: Primary data validation passed. [2025-11-04T19:57:20.922597] [INFO ] [MAIN ] LNC: Fetched primary data is valid. [2025-11-04T19:57:20.922639] [DEBUG] [LOCAL ] LNC: Saving primary data. [2025-11-04T19:57:20.924115] [DEBUG] [LOCAL ] LNC: Trimmed 74549 records to 51242 (keeping 200 days of data) [2025-11-04T19:57:20.975871] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-02. [2025-11-04T19:57:21.229246] [DEBUG] [LOCAL ] LNC: Primary data saved (51242 records). [2025-11-04T19:57:21.229372] [INFO ] [MAIN ] LNC: Full history saved. [2025-11-04T19:57:21.229447] [INFO ] [MAIN ] Update Progress: 1260/2367 symbols completed. LNC took 34.1s. Estimated remaining: 2h 35m 46s. Rate: 7.1 symbols/min [2025-11-04T19:57:21.397410] [INFO ] [MAIN ] LNN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:57:21.409561] [DEBUG] [LOCAL ] LNN: Loaded 11676 existing records from primary storage. [2025-11-04T19:57:21.409806] [INFO ] [MAIN ] LNN: Loaded 11676 existing records. [2025-11-04T19:57:21.588623] [WARN ] [MAIN ] LNN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:57:21.588831] [INFO ] [MAIN ] LNN: Wiping existing primary data before full refresh. [2025-11-04T19:57:21.588876] [INFO ] [LOCAL ] LNN: Starting deletion of primary data. [2025-11-04T19:57:21.589170] [DEBUG] [LOCAL ] LNN: Deleted primary data file. [2025-11-04T19:57:21.589226] [INFO ] [MAIN ] LNN: Starting full primary refresh. [2025-11-04T19:57:21.772403] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-07. [2025-11-04T19:57:21.946596] [INFO ] [REMOTE ] LNN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:57:22.296218] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-11-04T19:57:22.381254] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-11-04T19:57:22.567046] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-03. [2025-11-04T19:57:23.367047] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-08. [2025-11-04T19:57:23.788593] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-11-04T19:57:23.968257] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-11-04T19:57:24.167042] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-09. [2025-11-04T19:57:24.542105] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-11-04T19:57:24.967019] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-10. [2025-11-04T19:57:25.447419] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-11-04T19:57:25.767084] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-11. [2025-11-04T19:57:26.567061] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-01. [2025-11-04T19:57:27.100690] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-11-04T19:57:27.240348] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-11 (1.47 seconds). [2025-11-04T19:57:27.242261] [INFO ] [REMOTE ] LMT: Full fetch complete (72516 records). [2025-11-04T19:57:27.242349] [DEBUG] [VALIDATOR ] LMT: Validating primary data. [2025-11-04T19:57:27.372194] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-04. [2025-11-04T19:57:27.636725] [DEBUG] [VALIDATOR ] LMT: All expected trading days are present. [2025-11-04T19:57:27.636828] [DEBUG] [VALIDATOR ] LMT: Primary data validation passed. [2025-11-04T19:57:27.638506] [INFO ] [MAIN ] LMT: Fetched primary data is valid. [2025-11-04T19:57:27.638568] [DEBUG] [LOCAL ] LMT: Saving primary data. [2025-11-04T19:57:27.639907] [DEBUG] [LOCAL ] LMT: Trimmed 72516 records to 49424 (keeping 200 days of data) [2025-11-04T19:57:27.956830] [DEBUG] [LOCAL ] LMT: Primary data saved (49424 records). [2025-11-04T19:57:27.956985] [INFO ] [MAIN ] LMT: Full history saved. [2025-11-04T19:57:27.957097] [INFO ] [MAIN ] Update Progress: 1261/2367 symbols completed. LMT took 47.8s. Estimated remaining: 2h 35m 36s. Rate: 7.1 symbols/min [2025-11-04T19:57:28.126957] [INFO ] [MAIN ] LOAR: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:57:28.163601] [DEBUG] [LOCAL ] LOAR: Loaded 41332 existing records from primary storage. [2025-11-04T19:57:28.163904] [INFO ] [MAIN ] LOAR: Loaded 41332 existing records. [2025-11-04T19:57:28.175054] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-02. [2025-11-04T19:57:28.342674] [WARN ] [MAIN ] LOAR: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:57:28.342907] [INFO ] [MAIN ] LOAR: Wiping existing primary data before full refresh. [2025-11-04T19:57:28.342991] [INFO ] [LOCAL ] LOAR: Starting deletion of primary data. [2025-11-04T19:57:28.343180] [DEBUG] [LOCAL ] LOAR: Deleted primary data file. [2025-11-04T19:57:28.343218] [INFO ] [MAIN ] LOAR: Starting full primary refresh. [2025-11-04T19:57:28.702711] [INFO ] [REMOTE ] LOAR: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:57:28.750374] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-11-04T19:57:28.858551] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-11-04T19:57:28.967009] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-05. [2025-11-04T19:57:29.767032] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-11. [2025-11-04T19:57:30.248553] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-11 (0.48 seconds). [2025-11-04T19:57:30.249734] [INFO ] [REMOTE ] LND: Full fetch complete (7819 records). [2025-11-04T19:57:30.249819] [DEBUG] [VALIDATOR ] LND: Validating primary data. [2025-11-04T19:57:30.447715] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-11-04T19:57:30.576496] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-03. [2025-11-04T19:57:30.700533] [DEBUG] [VALIDATOR ] LND: All expected trading days are present. [2025-11-04T19:57:30.700624] [DEBUG] [VALIDATOR ] LND: Primary data validation passed. [2025-11-04T19:57:30.700871] [INFO ] [MAIN ] LND: Fetched primary data is valid. [2025-11-04T19:57:30.700918] [DEBUG] [LOCAL ] LND: Saving primary data. [2025-11-04T19:57:30.701297] [DEBUG] [LOCAL ] LND: Trimmed 7819 records to 4844 (keeping 200 days of data) [2025-11-04T19:57:30.728529] [DEBUG] [LOCAL ] LND: Primary data saved (4844 records). [2025-11-04T19:57:30.728639] [INFO ] [MAIN ] LND: Full history saved. [2025-11-04T19:57:30.728763] [INFO ] [MAIN ] Update Progress: 1262/2367 symbols completed. LND took 17.8s. Estimated remaining: 2h 35m 23s. Rate: 7.1 symbols/min [2025-11-04T19:57:30.896751] [INFO ] [MAIN ] LOB: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:57:30.921027] [DEBUG] [LOCAL ] LOB: Loaded 25890 existing records from primary storage. [2025-11-04T19:57:30.921263] [INFO ] [MAIN ] LOB: Loaded 25890 existing records. [2025-11-04T19:57:31.096274] [WARN ] [MAIN ] LOB: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:57:31.096520] [INFO ] [MAIN ] LOB: Wiping existing primary data before full refresh. [2025-11-04T19:57:31.096568] [INFO ] [LOCAL ] LOB: Starting deletion of primary data. [2025-11-04T19:57:31.096918] [DEBUG] [LOCAL ] LOB: Deleted primary data file. [2025-11-04T19:57:31.096976] [INFO ] [MAIN ] LOB: Starting full primary refresh. [2025-11-04T19:57:31.316122] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-11-04T19:57:31.372554] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-06. [2025-11-04T19:57:31.464352] [INFO ] [REMOTE ] LOB: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:57:32.167041] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-01. [2025-11-04T19:57:32.893177] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-11-04T19:57:32.916704] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-11-04T19:57:32.967005] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-07. [2025-11-04T19:57:33.767075] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-02. [2025-11-04T19:57:34.430961] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T19:57:34.557200] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-11-04T19:57:34.566997] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-03. [2025-11-04T19:57:35.367049] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-04. [2025-11-04T19:57:35.478280] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-11-04T19:57:36.057409] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-11-04T19:57:36.167034] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-05. [2025-11-04T19:57:36.606682] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-11-04T19:57:36.967054] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-06. [2025-11-04T19:57:37.567701] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-11-04T19:57:37.767062] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-01. [2025-11-04T19:57:38.569721] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-07. [2025-11-04T19:57:38.573704] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-11-04T19:57:39.326421] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-11-04T19:57:39.366997] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-08. [2025-11-04T19:57:39.883856] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-11-04T19:57:40.167007] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-09. [2025-11-04T19:57:40.671178] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-11-04T19:57:40.967180] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-04. [2025-11-04T19:57:41.767009] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-10. [2025-11-04T19:57:41.889767] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T19:57:42.528588] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-11-04T19:57:42.567060] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-11. [2025-11-04T19:57:43.226865] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-11 (0.66 seconds). [2025-11-04T19:57:43.227842] [INFO ] [REMOTE ] LNN: Full fetch complete (16598 records). [2025-11-04T19:57:43.227912] [DEBUG] [VALIDATOR ] LNN: Validating primary data. [2025-11-04T19:57:43.372183] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-08. [2025-11-04T19:57:43.607155] [DEBUG] [VALIDATOR ] LNN: All expected trading days are present. [2025-11-04T19:57:43.607239] [DEBUG] [VALIDATOR ] LNN: Primary data validation passed. [2025-11-04T19:57:43.607705] [INFO ] [MAIN ] LNN: Fetched primary data is valid. [2025-11-04T19:57:43.607768] [DEBUG] [LOCAL ] LNN: Saving primary data. [2025-11-04T19:57:43.608283] [DEBUG] [LOCAL ] LNN: Trimmed 16598 records to 11674 (keeping 200 days of data) [2025-11-04T19:57:43.683683] [DEBUG] [LOCAL ] LNN: Primary data saved (11674 records). [2025-11-04T19:57:43.683825] [INFO ] [MAIN ] LNN: Full history saved. [2025-11-04T19:57:43.683904] [INFO ] [MAIN ] Update Progress: 1263/2367 symbols completed. LNN took 22.5s. Estimated remaining: 2h 35m 18s. Rate: 7.1 symbols/min [2025-11-04T19:57:43.862201] [INFO ] [MAIN ] LOCL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:57:43.869400] [DEBUG] [LOCAL ] LOCL: Loaded 3217 existing records from primary storage. [2025-11-04T19:57:43.869563] [INFO ] [MAIN ] LOCL: Loaded 3217 existing records. [2025-11-04T19:57:44.049939] [WARN ] [MAIN ] LOCL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:57:44.050339] [INFO ] [MAIN ] LOCL: Wiping existing primary data before full refresh. [2025-11-04T19:57:44.050426] [INFO ] [LOCAL ] LOCL: Starting deletion of primary data. [2025-11-04T19:57:44.051000] [DEBUG] [LOCAL ] LOCL: Deleted primary data file. [2025-11-04T19:57:44.051058] [INFO ] [MAIN ] LOCL: Starting full primary refresh. [2025-11-04T19:57:44.172197] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-05. [2025-11-04T19:57:44.425109] [INFO ] [REMOTE ] LOCL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:57:44.794422] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T19:57:44.949199] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-11-04T19:57:44.967003] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-09. [2025-11-04T19:57:45.767025] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-06. [2025-11-04T19:57:46.266001] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-11-04T19:57:46.509150] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-11-04T19:57:46.567022] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-10. [2025-11-04T19:57:47.367082] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-01. [2025-11-04T19:57:47.652166] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-01 (0.29 seconds). [2025-11-04T19:57:47.983801] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-11-04T19:57:48.167106] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-07. [2025-11-04T19:57:48.967030] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-02. [2025-11-04T19:57:49.151089] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-11-04T19:57:49.767020] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-11. [2025-11-04T19:57:49.839525] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-11-04T19:57:50.185143] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-11 (0.42 seconds). [2025-11-04T19:57:50.187147] [INFO ] [REMOTE ] LNG: Full fetch complete (74608 records). [2025-11-04T19:57:50.187260] [DEBUG] [VALIDATOR ] LNG: Validating primary data. [2025-11-04T19:57:50.572930] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-02. [2025-11-04T19:57:50.574429] [DEBUG] [VALIDATOR ] LNG: All expected trading days are present. [2025-11-04T19:57:50.574483] [DEBUG] [VALIDATOR ] LNG: Primary data validation passed. [2025-11-04T19:57:50.575515] [INFO ] [MAIN ] LNG: Fetched primary data is valid. [2025-11-04T19:57:50.575706] [DEBUG] [LOCAL ] LNG: Saving primary data. [2025-11-04T19:57:50.577141] [DEBUG] [LOCAL ] LNG: Trimmed 74608 records to 50950 (keeping 200 days of data) [2025-11-04T19:57:50.880480] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-11-04T19:57:50.890056] [DEBUG] [LOCAL ] LNG: Primary data saved (50950 records). [2025-11-04T19:57:50.890132] [INFO ] [MAIN ] LNG: Full history saved. [2025-11-04T19:57:50.890199] [INFO ] [MAIN ] Update Progress: 1264/2367 symbols completed. LNG took 36.5s. Estimated remaining: 2h 35m 9s. Rate: 7.1 symbols/min [2025-11-04T19:57:51.056486] [INFO ] [MAIN ] LOMA: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:57:51.088494] [DEBUG] [LOCAL ] LOMA: Loaded 33555 existing records from primary storage. [2025-11-04T19:57:51.088814] [INFO ] [MAIN ] LOMA: Loaded 33555 existing records. [2025-11-04T19:57:51.265450] [WARN ] [MAIN ] LOMA: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:57:51.265723] [INFO ] [MAIN ] LOMA: Wiping existing primary data before full refresh. [2025-11-04T19:57:51.265812] [INFO ] [LOCAL ] LOMA: Starting deletion of primary data. [2025-11-04T19:57:51.266101] [DEBUG] [LOCAL ] LOMA: Deleted primary data file. [2025-11-04T19:57:51.266156] [INFO ] [MAIN ] LOMA: Starting full primary refresh. [2025-11-04T19:57:51.372110] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-03. [2025-11-04T19:57:51.614135] [INFO ] [REMOTE ] LOMA: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:57:52.167042] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-08. [2025-11-04T19:57:52.387958] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-11-04T19:57:52.967011] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-03. [2025-11-04T19:57:53.076103] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-11-04T19:57:53.283999] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-11-04T19:57:53.767041] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-01. [2025-11-04T19:57:54.567024] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-04. [2025-11-04T19:57:54.707960] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-11-04T19:57:55.055505] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-11-04T19:57:55.367069] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-09. [2025-11-04T19:57:56.167048] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-05. [2025-11-04T19:57:56.246309] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-11-04T19:57:56.440077] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-05 (0.27 seconds). [2025-11-04T19:57:56.967037] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-06. [2025-11-04T19:57:57.263310] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-06 (0.30 seconds). [2025-11-04T19:57:57.767055] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-04. [2025-11-04T19:57:58.567100] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-07. [2025-11-04T19:57:58.897483] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T19:57:58.909524] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-11-04T19:57:59.367041] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-08. [2025-11-04T19:57:59.695806] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-11-04T19:58:00.167075] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-10. [2025-11-04T19:58:00.967050] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-05. [2025-11-04T19:58:01.112117] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T19:58:01.767064] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-02. [2025-11-04T19:58:02.201255] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-11-04T19:58:02.567009] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-06. [2025-11-04T19:58:02.600456] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-11-04T19:58:03.367078] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-09. [2025-11-04T19:58:03.739893] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-11-04T19:58:03.780461] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-11-04T19:58:04.167094] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-11. [2025-11-04T19:58:04.900723] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-11 (0.73 seconds). [2025-11-04T19:58:04.902021] [INFO ] [REMOTE ] LOB: Full fetch complete (38824 records). [2025-11-04T19:58:04.902109] [DEBUG] [VALIDATOR ] LOB: Validating primary data. [2025-11-04T19:58:04.972202] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-10. [2025-11-04T19:58:05.286095] [DEBUG] [VALIDATOR ] LOB: All expected trading days are present. [2025-11-04T19:58:05.286187] [DEBUG] [VALIDATOR ] LOB: Primary data validation passed. [2025-11-04T19:58:05.286957] [INFO ] [MAIN ] LOB: Fetched primary data is valid. [2025-11-04T19:58:05.287031] [DEBUG] [LOCAL ] LOB: Saving primary data. [2025-11-04T19:58:05.287957] [DEBUG] [LOCAL ] LOB: Trimmed 38824 records to 25848 (keeping 200 days of data) [2025-11-04T19:58:05.426421] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-11-04T19:58:05.444362] [DEBUG] [LOCAL ] LOB: Primary data saved (25848 records). [2025-11-04T19:58:05.444432] [INFO ] [MAIN ] LOB: Full history saved. [2025-11-04T19:58:05.444499] [INFO ] [MAIN ] Update Progress: 1265/2367 symbols completed. LOB took 34.7s. Estimated remaining: 2h 35m 6s. Rate: 7.1 symbols/min [2025-11-04T19:58:05.612839] [INFO ] [MAIN ] LOW: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:58:05.661071] [DEBUG] [LOCAL ] LOW: Loaded 52907 existing records from primary storage. [2025-11-04T19:58:05.661365] [INFO ] [MAIN ] LOW: Loaded 52907 existing records. [2025-11-04T19:58:05.772138] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-11. [2025-11-04T19:58:05.842685] [WARN ] [MAIN ] LOW: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:58:05.842885] [INFO ] [MAIN ] LOW: Wiping existing primary data before full refresh. [2025-11-04T19:58:05.842934] [INFO ] [LOCAL ] LOW: Starting deletion of primary data. [2025-11-04T19:58:05.843134] [DEBUG] [LOCAL ] LOW: Deleted primary data file. [2025-11-04T19:58:05.843191] [INFO ] [MAIN ] LOW: Starting full primary refresh. [2025-11-04T19:58:06.200607] [INFO ] [REMOTE ] LOW: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:58:06.253172] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-11 (0.48 seconds). [2025-11-04T19:58:06.253872] [INFO ] [REMOTE ] LOCL: Full fetch complete (4601 records). [2025-11-04T19:58:06.253939] [DEBUG] [VALIDATOR ] LOCL: Validating primary data. [2025-11-04T19:58:06.578454] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-03. [2025-11-04T19:58:06.620863] [DEBUG] [VALIDATOR ] LOCL: All expected trading days are present. [2025-11-04T19:58:06.620934] [DEBUG] [VALIDATOR ] LOCL: Primary data validation passed. [2025-11-04T19:58:06.621087] [INFO ] [MAIN ] LOCL: Fetched primary data is valid. [2025-11-04T19:58:06.621129] [DEBUG] [LOCAL ] LOCL: Saving primary data. [2025-11-04T19:58:06.621479] [DEBUG] [LOCAL ] LOCL: Trimmed 4601 records to 3060 (keeping 200 days of data) [2025-11-04T19:58:06.639034] [DEBUG] [LOCAL ] LOCL: Primary data saved (3060 records). [2025-11-04T19:58:06.639101] [INFO ] [MAIN ] LOCL: Full history saved. [2025-11-04T19:58:06.639171] [INFO ] [MAIN ] Update Progress: 1266/2367 symbols completed. LOCL took 23.0s. Estimated remaining: 2h 34m 51s. Rate: 7.1 symbols/min [2025-11-04T19:58:06.813822] [INFO ] [MAIN ] LPG: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:58:06.849228] [DEBUG] [LOCAL ] LPG: Loaded 38514 existing records from primary storage. [2025-11-04T19:58:06.849500] [INFO ] [MAIN ] LPG: Loaded 38514 existing records. [2025-11-04T19:58:07.035067] [WARN ] [MAIN ] LPG: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:58:07.035287] [INFO ] [MAIN ] LPG: Wiping existing primary data before full refresh. [2025-11-04T19:58:07.035340] [INFO ] [LOCAL ] LPG: Starting deletion of primary data. [2025-11-04T19:58:07.035525] [DEBUG] [LOCAL ] LPG: Deleted primary data file. [2025-11-04T19:58:07.035582] [INFO ] [MAIN ] LPG: Starting full primary refresh. [2025-11-04T19:58:07.372686] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-01. [2025-11-04T19:58:07.412307] [INFO ] [REMOTE ] LPG: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:58:07.576140] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-11-04T19:58:08.167061] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-04. [2025-11-04T19:58:08.780437] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-11-04T19:58:08.967016] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-02. [2025-11-04T19:58:09.088829] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-11-04T19:58:09.767027] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-01. [2025-11-04T19:58:10.163788] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-11-04T19:58:10.567040] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-05. [2025-11-04T19:58:10.929417] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-11-04T19:58:11.367052] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-03. [2025-11-04T19:58:11.436240] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-11-04T19:58:12.167089] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-02. [2025-11-04T19:58:12.757950] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-11-04T19:58:12.967057] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-04. [2025-11-04T19:58:13.248717] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T19:58:13.767024] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-03. [2025-11-04T19:58:14.427391] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-11-04T19:58:14.567035] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-05. [2025-11-04T19:58:14.833545] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-11-04T19:58:15.367039] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-07. [2025-11-04T19:58:16.147265] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-11-04T19:58:16.167008] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-06. [2025-11-04T19:58:16.410276] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-11-04T19:58:16.967040] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-06. [2025-11-04T19:58:17.504706] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-11-04T19:58:17.767012] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-07. [2025-11-04T19:58:17.838532] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-11-04T19:58:18.567035] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-07. [2025-11-04T19:58:19.225931] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-11-04T19:58:19.367063] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-08. [2025-11-04T19:58:19.429347] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-11-04T19:58:20.167096] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-08. [2025-11-04T19:58:20.789623] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-11-04T19:58:20.967053] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-08. [2025-11-04T19:58:21.291850] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-11-04T19:58:21.767026] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-04. [2025-11-04T19:58:21.869875] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-11-04T19:58:22.567040] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-09. [2025-11-04T19:58:22.787055] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-11-04T19:58:23.367038] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-09. [2025-11-04T19:58:23.637583] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-11-04T19:58:24.167041] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-10. [2025-11-04T19:58:24.926704] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-11-04T19:58:24.966992] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-05. [2025-11-04T19:58:25.120070] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-11-04T19:58:25.767062] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-09. [2025-11-04T19:58:26.003597] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-11-04T19:58:26.567037] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-06. [2025-11-04T19:58:26.984032] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-11-04T19:58:27.367061] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-10. [2025-11-04T19:58:27.647155] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T19:58:28.167092] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-07. [2025-11-04T19:58:28.577052] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-11-04T19:58:28.967037] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-11. [2025-11-04T19:58:29.234700] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-11-04T19:58:29.767017] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-08. [2025-11-04T19:58:29.816507] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-11 (0.85 seconds). [2025-11-04T19:58:29.818276] [INFO ] [REMOTE ] LOMA: Full fetch complete (48522 records). [2025-11-04T19:58:29.818357] [DEBUG] [VALIDATOR ] LOMA: Validating primary data. [2025-11-04T19:58:30.188380] [DEBUG] [VALIDATOR ] LOMA: All expected trading days are present. [2025-11-04T19:58:30.188460] [DEBUG] [VALIDATOR ] LOMA: Primary data validation passed. [2025-11-04T19:58:30.189201] [INFO ] [MAIN ] LOMA: Fetched primary data is valid. [2025-11-04T19:58:30.189261] [DEBUG] [LOCAL ] LOMA: Saving primary data. [2025-11-04T19:58:30.190268] [DEBUG] [LOCAL ] LOMA: Trimmed 48522 records to 33704 (keeping 200 days of data) [2025-11-04T19:58:30.379209] [DEBUG] [LOCAL ] LOMA: Primary data saved (33704 records). [2025-11-04T19:58:30.379344] [INFO ] [MAIN ] LOMA: Full history saved. [2025-11-04T19:58:30.379459] [INFO ] [MAIN ] Update Progress: 1267/2367 symbols completed. LOMA took 39.5s. Estimated remaining: 2h 34m 56s. Rate: 7.1 symbols/min [2025-11-04T19:58:30.553851] [INFO ] [MAIN ] LPL: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:58:30.568993] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-11. [2025-11-04T19:58:30.583136] [DEBUG] [LOCAL ] LPL: Loaded 29628 existing records from primary storage. [2025-11-04T19:58:30.583438] [INFO ] [MAIN ] LPL: Loaded 29628 existing records. [2025-11-04T19:58:30.773700] [WARN ] [MAIN ] LPL: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:58:30.774077] [INFO ] [MAIN ] LPL: Wiping existing primary data before full refresh. [2025-11-04T19:58:30.774793] [INFO ] [LOCAL ] LPL: Starting deletion of primary data. [2025-11-04T19:58:30.774983] [DEBUG] [LOCAL ] LPL: Deleted primary data file. [2025-11-04T19:58:30.775022] [INFO ] [MAIN ] LPL: Starting full primary refresh. [2025-11-04T19:58:31.083134] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-11-04T19:58:31.169958] [INFO ] [REMOTE ] LPL: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:58:31.367028] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-01. [2025-11-04T19:58:31.469038] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-11 (0.90 seconds). [2025-11-04T19:58:31.470781] [INFO ] [REMOTE ] LOAR: Full fetch complete (56082 records). [2025-11-04T19:58:31.470875] [DEBUG] [VALIDATOR ] LOAR: Validating primary data. [2025-11-04T19:58:31.951886] [DEBUG] [VALIDATOR ] LOAR: All expected trading days are present. [2025-11-04T19:58:31.951980] [DEBUG] [VALIDATOR ] LOAR: Primary data validation passed. [2025-11-04T19:58:31.953112] [INFO ] [MAIN ] LOAR: Fetched primary data is valid. [2025-11-04T19:58:31.953176] [DEBUG] [LOCAL ] LOAR: Saving primary data. [2025-11-04T19:58:31.954366] [DEBUG] [LOCAL ] LOAR: Trimmed 56082 records to 41262 (keeping 200 days of data) [2025-11-04T19:58:32.178504] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-09. [2025-11-04T19:58:32.223220] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-11-04T19:58:32.241200] [DEBUG] [LOCAL ] LOAR: Primary data saved (41262 records). [2025-11-04T19:58:32.245895] [INFO ] [MAIN ] LOAR: Full history saved. [2025-11-04T19:58:32.245995] [INFO ] [MAIN ] Update Progress: 1268/2367 symbols completed. LOAR took 1m 4s. Estimated remaining: 2h 34m 42s. Rate: 7.1 symbols/min [2025-11-04T19:58:32.417760] [INFO ] [MAIN ] LPX: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:58:32.455638] [DEBUG] [LOCAL ] LPX: Loaded 39961 existing records from primary storage. [2025-11-04T19:58:32.455962] [INFO ] [MAIN ] LPX: Loaded 39961 existing records. [2025-11-04T19:58:32.634990] [WARN ] [MAIN ] LPX: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:58:32.635201] [INFO ] [MAIN ] LPX: Wiping existing primary data before full refresh. [2025-11-04T19:58:32.635246] [INFO ] [LOCAL ] LPX: Starting deletion of primary data. [2025-11-04T19:58:32.635449] [DEBUG] [LOCAL ] LPX: Deleted primary data file. [2025-11-04T19:58:32.635506] [INFO ] [MAIN ] LPX: Starting full primary refresh. [2025-11-04T19:58:32.983073] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-10. [2025-11-04T19:58:32.991543] [INFO ] [REMOTE ] LPX: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:58:33.167329] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-11-04T19:58:33.767045] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-01. [2025-11-04T19:58:34.567018] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-02. [2025-11-04T19:58:34.706166] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-11-04T19:58:34.964768] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-11-04T19:58:35.216369] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-11-04T19:58:35.367046] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-03. [2025-11-04T19:58:36.078509] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-11-04T19:58:36.167021] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-10. [2025-11-04T19:58:36.967009] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-04. [2025-11-04T19:58:37.168983] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-11-04T19:58:37.719341] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-11-04T19:58:37.767007] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-02. [2025-11-04T19:58:38.567044] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-05. [2025-11-04T19:58:38.851038] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-11-04T19:58:39.350304] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-11-04T19:58:39.366999] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-06. [2025-11-04T19:58:39.976333] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-11-04T19:58:40.167042] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-03. [2025-11-04T19:58:40.967046] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-11. [2025-11-04T19:58:41.229847] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-11-04T19:58:41.324238] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-11 (0.36 seconds). [2025-11-04T19:58:41.326258] [INFO ] [REMOTE ] LOW: Full fetch complete (76670 records). [2025-11-04T19:58:41.326382] [DEBUG] [VALIDATOR ] LOW: Validating primary data. [2025-11-04T19:58:41.726554] [DEBUG] [VALIDATOR ] LOW: All expected trading days are present. [2025-11-04T19:58:41.726672] [DEBUG] [VALIDATOR ] LOW: Primary data validation passed. [2025-11-04T19:58:41.727895] [INFO ] [MAIN ] LOW: Fetched primary data is valid. [2025-11-04T19:58:41.727958] [DEBUG] [LOCAL ] LOW: Saving primary data. [2025-11-04T19:58:41.729403] [DEBUG] [LOCAL ] LOW: Trimmed 76670 records to 52914 (keeping 200 days of data) [2025-11-04T19:58:41.783711] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-04. [2025-11-04T19:58:42.068792] [DEBUG] [LOCAL ] LOW: Primary data saved (52914 records). [2025-11-04T19:58:42.068953] [INFO ] [MAIN ] LOW: Full history saved. [2025-11-04T19:58:42.069054] [INFO ] [MAIN ] Update Progress: 1269/2367 symbols completed. LOW took 36.6s. Estimated remaining: 2h 34m 34s. Rate: 7.1 symbols/min [2025-11-04T19:58:42.237612] [INFO ] [MAIN ] LRN: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:58:42.276482] [DEBUG] [LOCAL ] LRN: Loaded 39473 existing records from primary storage. [2025-11-04T19:58:42.276841] [INFO ] [MAIN ] LRN: Loaded 39473 existing records. [2025-11-04T19:58:42.459292] [WARN ] [MAIN ] LRN: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:58:42.459507] [INFO ] [MAIN ] LRN: Wiping existing primary data before full refresh. [2025-11-04T19:58:42.459555] [INFO ] [LOCAL ] LRN: Starting deletion of primary data. [2025-11-04T19:58:42.459756] [DEBUG] [LOCAL ] LRN: Deleted primary data file. [2025-11-04T19:58:42.459814] [INFO ] [MAIN ] LRN: Starting full primary refresh. [2025-11-04T19:58:42.572198] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-07. [2025-11-04T19:58:42.820505] [INFO ] [REMOTE ] LRN: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:58:42.913214] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T19:58:43.367015] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-05. [2025-11-04T19:58:43.463481] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-11-04T19:58:44.167040] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-11. [2025-11-04T19:58:44.468627] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-11-04T19:58:44.967103] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-08. [2025-11-04T19:58:45.094048] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-11 (0.93 seconds). [2025-11-04T19:58:45.095687] [INFO ] [REMOTE ] LPG: Full fetch complete (58243 records). [2025-11-04T19:58:45.095774] [DEBUG] [VALIDATOR ] LPG: Validating primary data. [2025-11-04T19:58:45.487797] [DEBUG] [VALIDATOR ] LPG: All expected trading days are present. [2025-11-04T19:58:45.487881] [DEBUG] [VALIDATOR ] LPG: Primary data validation passed. [2025-11-04T19:58:45.488861] [INFO ] [MAIN ] LPG: Fetched primary data is valid. [2025-11-04T19:58:45.488937] [DEBUG] [LOCAL ] LPG: Saving primary data. [2025-11-04T19:58:45.490086] [DEBUG] [LOCAL ] LPG: Trimmed 58243 records to 38475 (keeping 200 days of data) [2025-11-04T19:58:45.715913] [DEBUG] [LOCAL ] LPG: Primary data saved (38475 records). [2025-11-04T19:58:45.716052] [INFO ] [MAIN ] LPG: Full history saved. [2025-11-04T19:58:45.716132] [INFO ] [MAIN ] Update Progress: 1270/2367 symbols completed. LPG took 39.1s. Estimated remaining: 2h 34m 22s. Rate: 7.1 symbols/min [2025-11-04T19:58:45.772198] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-01. [2025-11-04T19:58:45.890585] [INFO ] [MAIN ] LSPD: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:58:45.928917] [DEBUG] [LOCAL ] LSPD: Loaded 41810 existing records from primary storage. [2025-11-04T19:58:45.929204] [INFO ] [MAIN ] LSPD: Loaded 41810 existing records. [2025-11-04T19:58:46.122485] [WARN ] [MAIN ] LSPD: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:58:46.131629] [INFO ] [MAIN ] LSPD: Wiping existing primary data before full refresh. [2025-11-04T19:58:46.131805] [INFO ] [LOCAL ] LSPD: Starting deletion of primary data. [2025-11-04T19:58:46.132270] [DEBUG] [LOCAL ] LSPD: Deleted primary data file. [2025-11-04T19:58:46.132343] [INFO ] [MAIN ] LSPD: Starting full primary refresh. [2025-11-04T19:58:46.166201] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-11-04T19:58:46.493330] [INFO ] [REMOTE ] LSPD: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:58:46.567054] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-09. [2025-11-04T19:58:46.950501] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-11-04T19:58:47.367071] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-02. [2025-11-04T19:58:47.699475] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-11-04T19:58:48.167089] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-06. [2025-11-04T19:58:48.295544] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-11-04T19:58:48.967014] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-10. [2025-11-04T19:58:49.155095] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-11-04T19:58:49.767028] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-07. [2025-11-04T19:58:50.066020] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-11-04T19:58:50.567036] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-01. [2025-11-04T19:58:50.788622] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-11-04T19:58:51.367014] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-03. [2025-11-04T19:58:51.557320] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-11-04T19:58:52.167043] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-08. [2025-11-04T19:58:52.487064] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-11-04T19:58:52.967070] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-11. [2025-11-04T19:58:53.131281] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-11-04T19:58:53.767030] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-09. [2025-11-04T19:58:54.025601] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-11 (1.06 seconds). [2025-11-04T19:58:54.027505] [INFO ] [REMOTE ] LPL: Full fetch complete (39057 records). [2025-11-04T19:58:54.027591] [DEBUG] [VALIDATOR ] LPL: Validating primary data. [2025-11-04T19:58:54.416330] [DEBUG] [VALIDATOR ] LPL: All expected trading days are present. [2025-11-04T19:58:54.416407] [DEBUG] [VALIDATOR ] LPL: Primary data validation passed. [2025-11-04T19:58:54.417071] [INFO ] [MAIN ] LPL: Fetched primary data is valid. [2025-11-04T19:58:54.417154] [DEBUG] [LOCAL ] LPL: Saving primary data. [2025-11-04T19:58:54.417987] [DEBUG] [LOCAL ] LPL: Trimmed 39057 records to 29813 (keeping 200 days of data) [2025-11-04T19:58:54.571252] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-04. [2025-11-04T19:58:54.576896] [DEBUG] [LOCAL ] LPL: Primary data saved (29813 records). [2025-11-04T19:58:54.577929] [INFO ] [MAIN ] LPL: Full history saved. [2025-11-04T19:58:54.578024] [INFO ] [MAIN ] Update Progress: 1271/2367 symbols completed. LPL took 24.2s. Estimated remaining: 2h 34m 14s. Rate: 7.1 symbols/min [2025-11-04T19:58:54.754314] [INFO ] [MAIN ] LTC: Starting incremental primary update from 2025-10-27 to 2025-11-04. [2025-11-04T19:58:54.783685] [DEBUG] [LOCAL ] LTC: Loaded 30595 existing records from primary storage. [2025-11-04T19:58:54.783978] [INFO ] [MAIN ] LTC: Loaded 30595 existing records. [2025-11-04T19:58:54.965001] [WARN ] [MAIN ] LTC: Insufficient historical data. Have data from 2025-04-17, but need from 2025-01-21 for 200 day retention. Performing full refresh. [2025-11-04T19:58:54.965278] [INFO ] [MAIN ] LTC: Wiping existing primary data before full refresh. [2025-11-04T19:58:54.965386] [INFO ] [LOCAL ] LTC: Starting deletion of primary data. [2025-11-04T19:58:54.975388] [DEBUG] [LOCAL ] LTC: Deleted primary data file. [2025-11-04T19:58:54.981193] [INFO ] [MAIN ] LTC: Starting full primary refresh. [2025-11-04T19:58:55.061187] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-11-04T19:58:55.365904] [INFO ] [REMOTE ] LTC: Fetching primary data covering 200 trading days from 2025-01-21 to 2025-11-04 (requires 11 months of data pull). [2025-11-04T19:58:55.367005] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-01. [2025-11-04T19:58:55.842552] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-11-04T19:58:56.167000] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-05. [2025-11-04T19:58:56.240676] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-11-04T19:58:56.967035] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-02. [2025-11-04T19:58:57.309791] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-11-04T19:58:57.767052] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-02. [2025-11-04T19:58:57.822061] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-11-04T19:58:58.567044] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-03. [2025-11-04T19:58:58.780902] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-11-04T19:58:59.368414] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-10. [2025-11-04T19:58:59.369847] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-11-04T19:59:00.167074] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-04. [2025-11-04T19:59:00.710036] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-11-04T19:59:00.967093] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-03. [2025-11-04T19:59:01.299505] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-11-04T19:59:01.767020] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-05. [2025-11-04T19:59:02.078412] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-11-04T19:59:02.567026] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-06. [2025-11-04T19:59:02.699798] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-11-04T19:59:03.367057] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-06. [2025-11-04T19:59:03.651276] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-11-04T19:59:04.167022] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-07. [2025-11-0